Image updating method, image updating system, and image updating program

Abstract

[Problem] To rapidly update a captured image of an area of interest. [Solution] In this method for updating a wide-area integrated image generated by integrating a plurality of mutually different partial images obtained by imaging, from the sky, partial areas obtained by subdividing an area of interest F, a computer executes: an integrated image generation step S102 for integrating the plurality of partial images to generate the wide-area integrated image; and, when a newly-acquired image of a partial area is newly acquired after the wide-area integrated image has been generated, an updating step S106 for updating a corresponding image, which corresponds to the partial area included in the newly-acquired image among the plurality of partial images constituting the wide-area integrated image, to the newly-acquired image.

Description

Image updating method, image updating system, and image updating program

[0001] The present invention relates to an image updating method, an image updating system, and an image updating program.

[0002] Patent Document 1 discloses a method for generating ground surface image data, which involves determining defective areas where clouds are captured from a static satellite image and partial areas where the ground surface is captured, obtaining static satellite images of the defective areas at different dates and times, saving them as complementary images, and synthesizing the satellite image of the partial area with the complementary image.

[0003] Patent No. 4365887

[0004] When wide-area integrated images, which are created by integrating multiple images taken from the air, are to be used to confirm the situation when a disaster or other emergency occurs, it is necessary to generate and update the wide-area integrated images quickly.

[0005] In this regard, in the method described in Patent Document 1, it is necessary to acquire the ground surface image again by a geostationary satellite when re-photographing the defective area, and it is not possible to generate and update the ground surface image data quickly.

[0006] The present invention has been made in consideration of the above-mentioned problems, and has an object to quickly generate or update a photographed image of a target area.

[0007] In order to achieve the above-mentioned object, an image updating method according to one aspect of the present invention is a method for updating a wide-area integrated image generated by integrating a plurality of different partial images photographed from above of partial areas into which a target area is subdivided, in which a computer executes an integrated image generation step of integrating the plurality of partial images to generate the wide-area integrated image, and an update step of, when a newly acquired image of the partial area is newly acquired after the generation of the wide-area integrated image, updating a corresponding image of the plurality of partial images constituting the wide-area integrated image that corresponds to the partial area included in the newly acquired image to the newly acquired image.

[0008] The update step may include an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and the update step may be configured to integrate the newly acquired image into the wide-area integrated image if the acquisition time of the newly acquired image is a predetermined time or more later than the acquisition time of the corresponding image.

[0009] The update step may include a corresponding image determination step of determining whether the corresponding image corresponding to the newly acquired image has been integrated into the wide-area integrated image, and the update step may be configured to integrate the newly acquired image into the wide-area integrated image if the corresponding image has not been integrated into the wide-area integrated image.

[0010] The update step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image, and the update step may be configured to integrate the newly acquired image into the wide-area integrated image if the acquisition time of the newly acquired image is a predetermined time or more later than the acquisition time of the corresponding image and if the resolution of the newly acquired image is equal to or greater than the resolution of the corresponding image.

[0011] The update step may include an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image, and in the update step, if the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount or more, the newly acquired image may be integrated into the wide-area integrated image.

[0012] In the updating step, the wide-area integrated image may be partially updated with the newly acquired image acquired by an image capturing means at an altitude different from that of the corresponding image.

[0013] The integrated image generation step may include an image suitability determination step for determining whether a plurality of partial images acquired at a first shooting altitude are suitable for inclusion in the wide-area integrated image, a complementation processing step for extracting areas of the target area for which usable partial images have not been acquired and using, in those areas, partial images taken at a second shooting altitude different from the first shooting altitude as complementation images, and an image integration step for integrating the complementation images into the wide-area integrated image.

[0014] The integrated image generation step may include an image suitability determination step for determining whether the acquired multiple partial images are suitable for inclusion in the wide-area integrated image; a complementation processing step for extracting areas of the target area for which an adoptable partial image has not been acquired, and for those areas using a partial image or geographic data acquired earlier than the partial image as a complement image; and an image integration step for integrating the complement image into the wide-area integrated image.

[0015] The multiple partial images that are integrated in the integrated image generation step may have overlapping areas in which adjacent partial images capture the same area, and the integrated image generation step may compare the qualities of the adjacent partial images and use the partial image with the better quality as the image in the overlapping area.

[0016] An update request receiving step may further be executed to receive input of a priority update area in the wide-area integrated image where the image is to be updated with priority.

[0017] The update request receiving step may receive input of at least one of an acquisition means, an image capturing altitude, and a resolution of a newly acquired image used to update the priority update area.

[0018] The method may further include a measurement request step of generating a measurement command for the priority update area, or a measurement plan generation step of generating a measurement plan, and a display control step of displaying the measurement command or the measurement plan on a display device.

[0019] When the update of the priority update area is completed, a notification step of notifying that the update has been completed may be further executed.

[0020] A display control step may further be executed to display, on a display device, at least one of the scheduled update time at which the newly acquired image photographed for the partial area is to be reflected in the wide-area integrated image, the acquisition means for the newly acquired image, or the shooting altitude of the newly acquired image, for each partial area based on a future image acquisition plan for the target area.

[0021] A display control step may further be executed to display, for each partial area, at least one of the update time of the displayed partial image, the acquisition means of the partial image, or the shooting altitude of the partial image on a display device.

[0022] The method may further include a display condition receiving step of receiving input of display conditions for the partial image to be displayed from a user, the display conditions including at least one of the acquisition time, acquisition means, acquisition altitude, or resolution of the partial image to be displayed, and a display control step of displaying on a display device the partial image that matches the display conditions received in the display condition receiving step.

[0023] In order to achieve the above-mentioned object, an image updating system according to another aspect of the present invention is an image updating system that updates a wide-area integrated image generated by integrating a plurality of different partial images taken from the air of subdivided partial areas of a target area, and includes an integrated image generation unit that integrates the plurality of partial images to generate the wide-area integrated image, and an image updating unit that, when a newly acquired image of the partial area is acquired after the generation of the wide-area integrated image, extracts a partial image taken of the partial area included in the newly acquired image from the plurality of partial images that constitute the wide-area integrated image as a corresponding image, and updates the corresponding image in the wide-area integrated image to the newly acquired image.

[0024] In order to achieve the above-mentioned object, an image updating program according to yet another aspect of the present invention is an image updating program for a wide-area integrated image generated by integrating a plurality of different partial images photographed from the air of subdivided partial areas of a target area, and causes a computer to execute an integrated image generation command for integrating the plurality of partial images to generate the wide-area integrated image, a corresponding image extraction command for, when a newly acquired image of the partial area is acquired after the generation of the wide-area integrated image, extracting a partial image photographed of the partial area included in the newly acquired image from the plurality of partial images constituting the wide-area integrated image as a corresponding image, and an update command for updating the corresponding image in the wide-area integrated image to the newly acquired image.

[0025] The computer program can be provided by being stored on various data-readable recording media, or can be provided so as to be downloadable via a network such as the Internet.

[0026] According to the present invention, it is possible to quickly update the captured images of the target area.

[0027] FIG. 1 is an overall configuration diagram of an image updating system according to an embodiment of the present invention. FIG. 1 is a conceptual diagram showing how a measuring instrument communicating with the image updating system measures a target area. FIG. 2 is a schematic diagram showing how images necessary for generating a wide-area integrated image are acquired by a plurality of measurement methods. FIG. 2 is a functional configuration diagram of a satellite measurement system provided in a satellite, which is a first example of the measuring instrument. FIG. 3 is a functional configuration diagram of an aircraft measurement system provided in an aircraft, which is a second example of the measuring instrument. FIG. 4 is a functional configuration diagram of an aerial data sensing system configured to be able to communicate with the image updating system. FIG. 5 is a functional configuration diagram of a spatial data processing system, which is an example of the image updating system. FIG. 6 is a flowchart showing a series of processes by which the image updating system acquires spatial data and updates a wide-area integrated image. FIG. 7 is a flowchart showing a detailed process flow of a process by which the image updating system generates a wide-area integrated image. FIG. 8 is a flowchart showing a detailed process flow of a process by which the image updating system creates and displays an update plan based on an update request. FIG. 9 is a flowchart showing a process flow by which the image updating system updates a wide-area integrated image based on an update request. FIG. 10 is a diagram showing a first example of a screen displayed by the image updating system. FIG. 11 is a diagram showing a second example of a screen displayed by the image updating system. FIG. 12 is a diagram showing a third example of a screen displayed by the image updating system.

[0028] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0029] <A. One embodiment> [A-1. Configuration] (A-1-1. Overview)

[0030] The spatial data processing system 4000 integrates multiple pieces of measurement data partially acquired by the measuring instrument 110 for the target area F (FIG. 2) to generate a wide-area integrated image. The target area F is a two-dimensional area from which data is acquired by the measuring instrument 110, and may be, for example, on water or land. The measuring instrument 110 may be located on the target area F, or may capture images from outside the target area F. The measurement data is appropriate data representing the condition of the earth's surface, such as an image, but may also be point cloud data.

[0031] As shown in Fig. 1, an image update system 4000 (hereinafter also referred to as "spatial data processing system 4000") according to one embodiment of the present invention is connected to an airborne data sensing system 1000, an equipment management system 5000, and a reservation system 6000 via a network NW. The airborne data sensing system 1000 is also connected to a measuring device 110. The measuring device 110 and the network NW may be connected to an appropriate artificial satellite by wireless.

[0032] The aviation data sensing system 1000 is a system that controls the measuring device 110. The aviation data sensing system 1000 controls each part of the measuring device 110, causes the measuring device 110 to operate along a predetermined route, and causes the measuring device 110 to photograph a target area.

[0033] The spatial data processing system 4000 is a system that accumulates measurement data such as images or point cloud data acquired by the measuring instrument 110. The spatial data processing system 4000 has a function of performing integration processing with map data and other information and publishing a wide-area integrated image. The spatial data processing system 4000 may be mounted on equipment fixed to the ground surface or on a mobile object such as a vehicle.

[0034] The facility and equipment management system 5000 is a system that manages the facilities and equipment at bases where aircraft 112 take off and land.

[0035] The reservation system 6000 is a system that manages reservations for base stations, for example, managing reservations for resources for various tasks at the base stations. Resources include all resources necessary for data collection, such as receiver equipment used for communication with and maintenance of aircraft 112, landing areas, or the number of workers. The reservation system 6000 may also store information on the time periods during which resources are occupied or the times when resources are released according to reservations. The flight operations management unit 1100, which will be described later, creates a measurement plan taking into account the reservation status of the base stations, etc. The reservation system 6000 also updates the reservation information for the base stations based on the measurement plan generated by the flight operations management unit 1100.

[0036] 1 may be connected to each other via a communication network such as the Internet or a communication method such as LTE. Each component may also be connected to a communication network via satellite communication via an artificial satellite. Each wireless communication may be a dedicated wireless communication network or may utilize an existing wireless infrastructure.

[0037] (A-1-2. Measuring device 110)

[0038] As shown in Figure 2, the measuring device 110 is a device that acquires measurement data of a partial area F110 that is a division of the target area F. The measuring device 110 is an example of an imaging means. The measuring device 110 moves, particularly relative to the ground surface, and captures images of the target area F while changing the partial area F110 to be captured as it moves. The measuring device 110 may also capture images of the target area F by changing the orientation of the sensor to change the partial area F110.

[0039] Multiple measuring instruments 110 are included in a single aviation data sensing system 1000, and the aviation data sensing system 1000 can simultaneously photograph multiple partial areas F110 by flying multiple measuring instruments 110 simultaneously over a single target area F to acquire measurement data.

[0040] The measurement devices 110 include multiple aircraft that perform measurements at different measurement altitudes. The measurement devices 110 are, for example, satellites 111 or aircraft 112 orbiting the Earth. The aircraft 112 may include a rotary-wing aircraft 112a and a fixed-wing aircraft 112b. In this case, the satellite 111 performs measurements at a higher measurement altitude than the aircraft 112. The rotary-wing aircraft 112a performs measurements at a relatively low altitude. The fixed-wing aircraft 112b performs measurements at an altitude equal to or higher than the rotary-wing aircraft 112a, but lower than the satellite 111. The altitude range of each measurement device 110 is predetermined depending on the type of aircraft. The altitude of each measurement device 110 may be variable based on commands from the aviation data sensing system 1000. (A-1-2-1. Satellite 111) The satellite 111 performs measurements of the surface of the target area from an orbit around the Earth. Measurement may involve not only image capture but also point cloud data acquisition using a laser sensor (LiDAR). The artificial satellite 111 may be a satellite used exclusively by the present system, or may be a general-purpose artificial satellite 111. The artificial satellite 111 orbits the Earth on a predetermined path, for example, a low-earth orbit satellite, but may also be a geostationary satellite or a medium-earth orbit satellite. Although only one artificial satellite 111 is depicted in FIGS. 1 and 2, multiple artificial satellites 111 may be connected to the spatial data processing system 4000. Multiple artificial satellites 111 at different altitudes may be connected to the spatial data processing system 4000.

[0041] (A-1-2-2. Aircraft 112) The aircraft 112 is, for example, an unmanned aircraft. In this specification, the term "aircraft" refers to any flying object that has the function of autonomously controlling its attitude, regardless of the power means (electric power, prime mover, etc.), the control method (wireless or wired, and whether fully autonomous flight or partially manual flight, etc.), and whether it is manned or unmanned. Furthermore, aircraft may also be referred to as unmanned aerial vehicles (UAVs), flying objects, multicopters, RPASs (remote piloted aircraft systems), UASs (unmanned aircraft systems), etc.

[0042] The aircraft 112 is not limited to the illustrated embodiment and may have any suitable configuration. For example, the aircraft 112 may be a rotary-wing aircraft 112a or a fixed-wing aircraft 112b, or a vertical take-off and landing aircraft (VTOL) equipped with both fixed and rotary wings. Furthermore, if the aircraft is equipped with rotary wings, a propeller guard (not shown) may be provided to prevent the rotors from interfering with obstacles. The measuring device 110 may also have an alarm device, such as a warning light and a speaker, that issues a warning to people around the measuring device 110.

[0043] 3 is a schematic diagram showing the paths of multiple types of measuring instruments 110 and the images acquired by the measuring instruments 110. The arrows on each of the paths R111a, R112a, and R112b indicate the movement path of the measuring instruments 110. In the example shown in the figure, when the artificial satellite 111 moves along the path R111a, it acquires an image P111a having a width H111a along the path R111a. Meanwhile, another artificial satellite 111 moves along the path R111b and acquires an image P111b along the path R111b. Because the artificial satellite 111 acquires images by moving along the satellite orbit, there are partial areas from which measurement data cannot be acquired.

[0044] In this regard, the aircraft 112 can fly to any position and acquire images by generating a flight path using the airborne data sensing system 1000. In the example shown in the figure, the rotary-wing aircraft 112a flies along a path R112a and acquires an image P112a having a width H112a along the path R112a. The fixed-wing aircraft 112b, which flies at a higher altitude than the rotary-wing aircraft 112a, flies along a path R112b and acquires an image P112b having a width H112b along the path R112b.

[0045] The width of the acquired image increases as the measurement altitude increases. That is, the imaging width H111a of the satellite 111 is the widest, followed by the imaging width H112b of the fixed-wing aircraft 112b, and the imaging width H112a of the rotary-wing aircraft 112a is the narrowest. Because the satellite 111 flies at an ultra-high altitude, it can acquire a wide image simply by passing over the target area F. On the other hand, the fixed-wing aircraft 112b flying at a high altitude flies back and forth over the target area F at a round-trip width H112a that ensures no measurement omissions, thereby acquiring an image of the target area F. Because the rotary-wing aircraft 112a flying at a low altitude has a low altitude, it flies back and forth at a narrower imaging width H112a than the fixed-wing aircraft 112b flying at a high altitude, thereby acquiring an image of the target area F without omissions. Furthermore, because of its low altitude, it can acquire high-resolution images. The rotary-wing aircraft 112a has the highest image resolution, followed by the fixed-wing aircraft 112b, and the images acquired by the satellite 111 have the lowest resolution. In this way, the spatial data processing system 4000 can efficiently obtain a wide-area integrated image representing the target area F by measuring the target area using different measurement methods and then integrating the obtained images.

[0046] Furthermore, since the aircraft 112 can acquire images of a desired flight path, it can acquire images of a desired area more quickly than the artificial satellite 111. With this configuration, even when the situation in the target area changes from moment to moment, such as during a disaster, the user can quickly visually confirm the situation in the area.

[0047] (A-1-2-3. Functional blocks of the satellite measurement system 1110) As shown in Figure 4, the satellite measurement system 1110 comprises an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a result, the software configuration mainly includes a sensor unit 1111, a self-position and attitude determination unit 1112, a measurement target position determination unit 1113, a data recording unit 1114, and a communication unit 1115.

[0048] (A-1-2-3-1. Sensor unit 1111) The sensor unit 1111 is configured to measure the target area F. The sensor unit 1111 mainly includes a laser sensor 1111a and an optical camera 1111b. The laser sensor 1111a acquires point cloud data of the ground surface, for example, by LiDAR. The optical camera 1111b is configured to capture images of the ground surface. Note that the sensor unit 1111 may include only either the laser sensor 1111a or the optical camera 1111b. Furthermore, the sensor unit 1111 may acquire the time of acquisition of the measurement data along with the measurement data.

[0049] (A-1-2-3-2. Self-position and attitude determination unit 1112) The self-position and attitude determination unit 1112 is a functional unit that determines the position and attitude of the artificial satellite 111. The self-position and attitude determination unit 1112 may record the measured altitude of the artificial satellite 111.

[0050] (A-1-2-3-3. Measurement target position determination unit 1113) The measurement target position determination unit 1113 is a functional unit that determines the partial area (hereinafter also referred to as the "measurement target area") that the acquired measurement data is intended to measure. The measurement target position determination unit 1113 calculates the measurement target area from the position and attitude of the artificial satellite 111. The measurement target position determination unit 1113 calculates, for example, the absolute position coordinates of the area. Furthermore, in a mode in which partial areas obtained by dividing the target area F are stored in advance, the measurement target position determination unit 1113 may perform processing to identify the partial area that corresponds to the acquired measurement data.

[0051] (A-1-2-3-4. Data recording unit 1114) The data recording unit 1114 is a functional unit that records measurement data obtained by the measuring instrument 110 on a recording medium inside the measuring instrument 110. The data recording unit 1114 acquires measurement data obtained by the sensor unit 1111 and related information about the measurement data. The related information includes the time of acquisition of the measurement data, position coordinates and measurement altitude obtained by the self-position and attitude determination unit 1112, or position information of the measurement target area obtained by the measurement target position determination unit 1113. The recording medium may include, for example, a separable medium such as an SD card or RAM.

[0052] (A-1-2-3-5. Communication Unit 1115) The communication unit 1115 is capable of radio wave communication via the communication network NW and includes, for example, a radio wave communication module. The communication unit 1115 is capable of communication with the measuring instrument 110 and the like via the communication network NW. The communication unit 1115 has a communication function for wirelessly communicating with the measuring instrument 110 using, for example, Wi-Fi, 2.4 GHz, or a frequency band of 5.6 to 5.8 GHz. The communication unit 1115 also has a wireless communication function for communicating with the spatial data processing system 4000 via the communication network NW using a communication standard such as LTE (Long Term Evolution). The communication unit 1115 transmits measurement data, related information, and the like to the spatial data processing system 4000 and the like. The communication unit 1115 may perform real-time transmission or may transmit data after the fact.

[0053] (A-1-2-4. Functional blocks of the aircraft measurement system 1120) The aircraft measurement system 1120 is a system that controls the aircraft 112. As shown in Fig. 5 , the aircraft measurement system 1120 includes a calculation device such as a CPU for executing information processing, and storage devices such as RAM and ROM, and thus has a software configuration that mainly includes functional components such as a sensor unit 1121, a self-position and attitude determination unit 1122, a measurement target position determination unit 1123, a flight control unit 1124, a data recording unit 1125, and a communication unit 1126.

[0054] (A-1-2-4-1. Sensor Unit 1121) The sensor unit 1121 is a sensor that measures the target area F and acquires measurement data. The sensor unit 1121 is held, for example, below or in front of the aircraft 112's body. The aircraft may be equipped with multiple sensor units 1121. The sensor unit 1121 is, for example, a camera that acquires an image of the target area F, but may also be a sensor such as a LiDAR (Light Detection and Ranging) or a microphone. Furthermore, the sensor unit 1121 may be an IR camera in addition to a visible light camera. That is, the sensing data acquired by the sensor unit 1121 may include, for example, an image, an IR image, point cloud data, or position data.

[0055] The orientation of the sensor unit 1121 can be adjusted by a sensor actuator (not shown). The sensor unit 1121 may have a function for automatically controlling parameters such as exposure, contrast, or ISO. The holder of the sensor unit 1121 may have a so-called gimbal control mechanism that suppresses the transmission of shaking or vibration of the aircraft to the sensor unit 1121. The sensor unit 1121 controls the sensor unit 1121 and the holder to adjust the shooting range of the camera, etc.

[0056] The measurement data acquired by the sensor unit 1121 is transmitted to the spatial data processing system 4000. The measurement data may also be stored in the data recording unit 1125 of the measuring device 110 itself.

[0057] (A-1-2-4-2. Self-position and attitude determination unit 1122) The self-position and attitude determination unit 1122 receives signals from artificial satellites and measures the position (absolute position) of the aircraft 112 based on the signals. The self-position and attitude determination unit 1122 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. As a method for measuring its position, for example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information.

[0058] (A-1-2-4-3. Measurement target position determination unit 1123) The measurement target position determination unit 1123 is a functional unit that determines the measurement target area of ​​the acquired measurement data. The measurement target position determination unit 1123 calculates the measurement target area from the position and attitude of the aircraft 112. The measurement target position determination unit 1123 calculates, for example, the absolute position coordinates of the area. Furthermore, in a mode in which partial areas obtained by dividing the target area F are stored in advance, the measurement target position determination unit 1123 may perform processing to identify the partial area corresponding to the acquired measurement data.

[0059] (A-1-2-4-4. Flight control unit 1124) The flight control unit 1124 is a mechanism and functional unit that operates and flies the aircraft 112, and generates thrust in the airframe to lift off and move the aircraft 112 in a desired direction. The flight control unit 1124 mainly has a motor control unit 1124a and a motor 1124b.

[0060] Flight control section 1124 includes a processing unit, also referred to as a flight controller. The processing unit may include one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to memory. The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps.

[0061] The processing unit includes a control module configured to control the airframe state of the aircraft 112. For example, the control module adjusts the spatial configuration, attitude angles, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the aircraft 112, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). That is, the flight control unit 1124 controls the attitude angle control and flight operations of the aircraft 112, from takeoff through flight and landing, by causing the aircraft 112 to perform operations such as takeoff, forward flight, turning, and landing.

[0062] The flight control unit 1124 can control the flight of the aircraft 112 based on, for example, an autonomous flight program acquired from the aircraft flight operating unit 1300. The flight control unit 1124 can also control the flight of the aircraft 112 by controlling the motor 1124b based on various information such as the target area F, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, current position information, attitude information (heading information), speed information, and acceleration information of the aircraft 112, and any combination of these.

[0063] The flight control unit 1124 may control the orientation and zoom amount of the sensor unit 1121. Regarding the orientation of the sensor unit 1121, the flight control unit 1124 may control either or both of the pitch angle with respect to the horizontal and the yaw angle with respect to a predetermined reference direction.

[0064] (A-1-2-4-5. Data Recording Unit 1125) The data recording unit 1125 stores, for example, measurement data measured by the aircraft 112 and related information.

[0065] (A-1-2-4-6. Communication Unit 1126) The communication unit 1126 has a modem or the like (not shown) and is capable of communicating with the aviation data sensing system 1000 or the spatial data processing system 4000 or the like via the communication network NW.

[0066] (A-1-3. Aviation Data Sensing System 1000) (A-1-3-1. Functional Blocks of Aviation Data Sensing System 1000) Fig. 6 is a functional configuration diagram of the aviation data sensing system 1000 of this embodiment. The aviation data sensing system 1000 is connected to the measuring device 110, and manages or controls the flight and photography of the measuring device 110.

[0067] The aviation data sensing system 1000 includes a computing device such as a CPU for executing information processing, and storage devices such as RAM and ROM. This software configuration primarily comprises functional blocks such as an aircraft flight management unit 1100, an airspace surveillance and control unit 1200, an aircraft flight operation unit 1300, an acquired data management unit 1400, and a communication infrastructure management unit 1500. (A-1-3-2. Flight Management Unit 1100) The flight management unit 1100 is a system that makes decisions and gives instructions regarding the operation of the measurement device 110. The flight management unit 1100 formulates plans for aircraft 112 operations, including measurement and flight, and transmits the plans to the aircraft 112 via the communication infrastructure management unit 1500. The flight management unit 1100 may formulate plans for multiple aircraft 112 and transmit information about the plans to each aircraft 112. The flight management unit 1100 may also prioritize the plans for the multiple aircraft 112 and determine a work plan according to the priorities. The flight management unit 1100 also receives control requests for the work airspace required for measuring the target area F, and coordinates with external systems by referring to information on the work airspace environment and other aircraft.

[0068] When the operation management unit 1100 receives an update request at a predetermined resolution from the update request receiving unit 4310 of the spatial data processing system 4000 (described later), the operation management unit 1100 may select an aircraft that can capture images at that resolution. In this case, the operation management unit 1100 may store controllable aircraft in association with the resolution at which images can be captured, the measurement altitude at which images can be captured, and the like, and may refer to this to select an aircraft.

[0069] In this embodiment, the operation management unit 1100 is described as being included in the aviation data sensing system 1000, but it may also be configured to be included in the measuring device 110 or the spatial data processing system 4000.

[0070] (A-1-3-3. Airspace Surveillance Control Unit 1200) The airspace surveillance control unit 1200 communicates with other aircraft in the operational airspace to determine the position of the aircraft. The aircraft may include manned aircraft and unmanned aircraft. This configuration enables the aircraft 112 to avoid contact with other aircraft and fly safely.

[0071] (A-1-3-4. Aircraft flight operating unit 1300) The aircraft flight operating unit 1300 is a functional unit that generates missions for the aircraft 112 and controls the movement of the aircraft 112. The mission is, for example, a movement plan that includes the movement route and movement speed of the aircraft 112. The aircraft flight operating unit 1300 transmits control signals to the aircraft 112 via the communication infrastructure management unit 1500 in order to operate the aircraft 112 automatically.

[0072] The aircraft operation unit 1300 may also determine the measurement altitude. Because the resolution of the measurement data acquired by each measuring device 110 varies depending on the measurement altitude, the aircraft operation unit 1300 determines the measurement altitude based on the resolution accepted by the update request acceptance unit 4310, for example. The resolution also varies depending on the settings of the installed sensor unit 1121 or the lens configuration. Therefore, the aircraft operation unit 1300 may change the settings of the sensor unit 1121 of the measuring device 110 so that an image at that resolution can be acquired.

[0073] (A-1-3-5. Acquired Data Management Unit 1400) The acquired data management unit 1400 is a functional unit that manages acquired data including video or images. The acquired data management unit 1400 determines the processing to be performed on the huge amount of acquired data for each piece of acquired data, and transmits a processing command via the communication infrastructure management unit 1500 to the aviation data sensing system 1000, the spatial data processing system 4000, the facility and equipment management system 5000, the reservation system 6000, or the like. In addition, the acquired data management unit 1400 records related information for each piece of acquired measurement data, such as the measured partial area, measurement time, resolution, measurement method, and measurement altitude, in association with the measurement data.

[0074] (A-1-3-6. Communication infrastructure management unit 1500) The communication infrastructure management unit 1500 is a system that manages communication means for transmitting and receiving data between the aviation data sensing system 1000 and the spatial data processing system 4000. For example, the communication infrastructure management unit 1500 transmits information generated by the measurement request unit 4320 to the aviation data sensing system 1000. In addition, the communication infrastructure management unit 1500 transmits the measurement plan generated by the operation management unit 1100 to the spatial data processing system 4000, and the measurement plan acquisition unit 4330 acquires this.

[0075] The communication infrastructure management unit 1500 also manages a plurality of different types of communication means. The communication infrastructure management unit 1500 may monitor the availability of communication through an existing infrastructure that is set to be used as a general rule, and the communication speed of communication through a communication means other than the existing infrastructure, and may determine the data transmission / reception means after determining the speed or urgency of data transmission / reception. Options for the data transmission / reception means may include, for example, communication methods such as parallel transmission and switched transmission, as well as a mode in which an operator physically transports a recording memory.

[0076] The configuration of the aviation data sensing system 1000 may be realized as a single device, or may be realized by multiple devices (e.g., measuring device 110, flight management unit 1100) partially or entirely connected via a communication network NW. Furthermore, each functional unit of the aviation data sensing system 1000 may be logically realized by cloud computing.

[0077] A plurality of aviation data sensing systems 1000 may be connected to one measuring device 110 via a plurality of communication networks NW, i.e., the system may be made redundant. In this case, even if an abnormality occurs in the flight management unit 1100 or the communication network, the measurement data acquisition process by the measuring device 110 can be continued by the other redundant flight management units 1100 or communication networks NW, thereby improving the reliability of the spatial data processing system 4000.

[0078] (A-1-4. Spatial Data Processing System 4000) The spatial data processing system 4000 shown in Figure 7 is a system that generates a wide-area integrated image of a target area by integrating partial images as measurement data, and also updates the wide-area integrated image partially using the acquired measurement data as needed, allowing a user to visually confirm the status of the target area. The spatial data processing system 4000 may also perform output processing to ensure compatibility with an external system, and output information included in the wide-area integrated image to the external system. The external system may be any business support GIS (Geographic Information System) such as PasCAL (registered trademark), OPEN GIS, or Arc GIS.

[0079] The spatial data processing system 4000 comprises an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a result, the software configuration mainly includes a spatial data acquisition unit 4100, an integrated image generation unit 4200, an update plan generation unit 4300, an integrated image update unit 4400, a spatial data information recording unit 4500, a display control unit 4600, and a notification unit 4700.

[0080] (A-1-4-1. Spatial data acquisition unit 4100) The spatial data acquisition unit 4100 is a functional unit that acquires data of the target area from the measuring instrument 110 or an appropriate external device. The spatial data acquisition unit 4100 mainly functionally includes a satellite measurement data acquisition unit 4110, an aircraft measurement data acquisition unit 4120, and a geographic data acquisition unit 4130.

[0081] The satellite measurement data acquisition unit 4110 acquires measurement data acquired by the artificial satellite 111. The aircraft measurement data acquisition unit 4120 is a functional unit that acquires measurement data acquired by the aircraft 112. This measurement data is used when the integrated image generation unit 4200 generates an initial wide-area integrated image and when the integrated image update unit 440 updates a portion of the wide-area integrated image. The satellite measurement data acquisition unit 4110 and the aircraft measurement data acquisition unit 4120 may also acquire related information about the measurement data. The related information includes, for example, measurement position coordinates, measurement time, resolution, measurement method, and measurement altitude. The acquired measurement data and related information are recorded in the measurement area recording unit 4510, measurement time recording unit 4520, measurement data resolution recording unit 4530, measurement method recording unit 4540, and measurement altitude recording unit 4550 of the spatial data information recording unit 4500, respectively.

[0082] The geographic data acquisition unit 4130 is a functional unit that acquires geographic data of the target area F. Unlike measurement data, geographic data is data that is almost universally linked to the target area F, such as map information or polygon data including three-dimensional land and building data. Note that the geographic data changes in response to changes in land shape and the renovation and demolition of buildings, and may be updated appropriately at a timing independent of the acquisition of measurement data. The acquired geographic data is stored in the geographic data recording unit 4560 of the spatial data information recording unit 4500.

[0083] (A-1-4-2. Integrated image generation unit 4200) The integrated image generation unit 4200 is a functional unit that integrates multiple partial images to generate a wide-area integrated image. The integrated image generation unit 4200 mainly has an integrated image selection unit 4210, an image suitability determination unit 4220, a complementary image selection unit 4230, and an image integration unit 4240.

[0084] The integrated image selection unit 4210 is a functional unit that selects an image to be integrated according to the measurement area, measurement time, or resolution of each acquired data.

[0085] The multiple partial images to be integrated have overlapping areas in which adjacent partial images capture the same area. Here, the integrated image selection unit 4210 compares the quality of the adjacent partial images and uses the partial image with the better quality as the image in the overlapping area. This configuration allows for the generation of a high-quality wide-area integrated image. Image quality may be determined, for example, by the state of sunlight reflection (presence or absence of overexposure, etc.), brightness, luminance, or radar intensity. Alternatively, an image with fewer defects and showing a larger area of ​​the ground surface may be determined to be of high quality.

[0086] The image suitability determination unit 4220 is a functional unit that determines whether each of the multiple partial images is suitable for use in the wide-area integrated image. The image suitability determination unit 4220, for example, determines whether each selected image has defects or measurement omissions. Examples of images with defects or measurement omissions include images with surface reflection, occlusion, low resolution, out-of-focus, blur, cloudiness, and out-of-focus. Images in which the ground surface area is not captured sufficiently due to clouds or foreign objects may also be considered defective. The image suitability determination unit 4220 determines the reflected area of ​​the ground surface, image blur, and the like, using an appropriate configuration. Clouds may be determined based on color tone or the difference between normal images. Clouds may also be determined using an appropriate learning algorithm. The image suitability determination unit 4220 may determine defects or measurement omissions by analyzing the histogram of the measured image. The image suitability determination unit 4220 may also identify partial areas from which an acceptable partial image has not been acquired.

[0087] The complementary image selection unit 4230 is a functional unit that selects a complementary image for a partial area for which the image suitability determination unit 4220 was unable to extract a suitable partial image. For example, in an area of ​​the target area F for which an acceptable partial image has not been acquired, the complementary image selection unit 4230 uses a partial image captured at a different altitude from the measurement data determined to be suitable by the integrated image selection unit 4210 as a complementary image. The images captured at different altitudes may have been captured by the same aircraft 112 or by different types of measuring instruments (e.g., the satellite 111 and the aircraft 112). Furthermore, if the measurement data determined to be suitable by the integrated image selection unit 4210 was acquired by a satellite 111, the complementary image selection unit 4230 may use measurement data acquired by a satellite 111 orbiting a different satellite 111 as a complementary image. This configuration is particularly suitable for removing images that include clouds.

[0088] In particular, the integrated image selection unit 4210 may use, as a complementary image, measurement data measured by a measuring instrument 110 of a type different from that of the measurement data for which suitability was determined. For example, the integrated image selection unit 4210 and the image suitability determination unit 4220 use a partial image acquired by the artificial satellite 111, and the complementary image selection unit 4230 uses an image acquired by the aircraft 112 for an area for which a partial image usable by the artificial satellite 111 has not been acquired. With this configuration, a wide-area integrated image can be efficiently generated by using data measured by different types of measuring instruments 110.

[0089] In particular, while the artificial satellite 111 can measure a wide area at once due to its high altitude, it can only measure from an orbit, making it difficult to measure a desired area. In contrast, the aircraft 112 can measure a desired area by control, but because it is at a lower altitude than the artificial satellite 111, the area it can measure at once is narrower, and it takes time to capture a wide area. In this regard, according to the configuration of the present application, images acquired by both the artificial satellite 111 and the aircraft 112 can be used and integrated, allowing for the rapid generation of a wide-area integrated image that combines images captured by multiple types of imaging means. Furthermore, since the artificial satellite 111 captures images from a high altitude, it acquires images with relatively low resolution, while the aircraft 112 captures images from a low altitude, it can acquire images with relatively high resolution. Therefore, by integrating images captured by multiple types of imaging means, it is possible to rapidly generate a wide-area integrated image in which a desired area is made high-resolution.

[0090] Furthermore, the complementary image selection unit 4230 may use a previous partial image acquired earlier than the partial image or geographic data as a complementary image for an area of ​​the target area F for which an acceptable partial image has not been acquired. A configuration that uses a previous partial image makes it possible to generate a wide-area integrated image that can grasp the situation to some extent even if there is no recently acquired image. Furthermore, a configuration that uses geographic data makes it possible to generate a wide-area integrated image even for an area for which no partial image exists.

[0091] The image integration unit 4240 is a functional unit that generates a wide-area integrated image. The image integration unit 4240 performs orthogonal transformation by orthorectification on the partial image selected by the integrated image selection unit 4210 or the complementary image selected by the complementary image selection unit 4230. The orthogonal transformation process may be performed on all pixels of the image, or on only some of the pixels. The image integration unit 4240 may identify the pixels to be subjected to the orthogonal transformation process based on the attitude, particularly the accuracy, of the measuring instrument 110 or the sensor units 1111 and 1121 when the image was measured.

[0092] The image integration unit 4240 inserts the partial image selected by the integrated image selection unit 4210 or the image selected by the complementary image selection unit 4230 into the partial region of interest and integrates them into a wide-area integrated image. Furthermore, the image integration unit 4240 inserts the image selected by the integrated image selection unit 4210 into the overlap region. The image integration unit 4240 performs the above-described image insertion process on each of the multiple partial regions that make up the target area F, thereby generating a wide-area integrated image of the target area F. Note that the image integration unit 4240 may be configured to directly insert the partial image or complementary image, or may be configured to associate a link indicating the storage location of the image with the partial region, thereby allowing the wide-area integrated image to be viewed.

[0093] With this configuration, images captured by different imaging means can be mixed and integrated, allowing for the rapid generation of a wide-area integrated image. Such a spatial data processing system 4000 is also useful for understanding the situation in areas where a disaster or other disaster has occurred.

[0094] (A-1-4-3. Update plan generation unit 4300) The update plan generation unit 4300 is a functional unit that generates an update plan for updating the wide-area integrated image, i.e., changing some of the partial images that make up the wide-area integrated image to newly acquired images. The update plan generation unit 4300 mainly functionally includes an update request reception unit 4310, a measurement request unit 4320, a measurement plan acquisition unit 4330, and an update plan creation unit 4340.

[0095] The update request receiving unit 4310 is a functional unit that receives input of a priority update area in the wide-area integrated image where image updating is to be performed preferentially. The update request receiving unit 4310 may receive a request for the resolution of the image to be acquired in the priority update area. The input of the resolution is not limited to an input of an absolute value, and may be received as a relative value from the current partial image. This may be, for example, a ratio or difference between the desired resolution and the resolution of the current partial image, or a qualitative request such as "an image with higher resolution" may be received. In response to a qualitative request, the update request receiving unit 4310 calculates the desired resolution value by applying the resolution of the current image to a predetermined formula or the like.

[0096] The update request receiving unit 4310 may receive input from multiple users. The users may include, for example, both a headquarters that specifies a relatively wide area and individual response teams that specify relatively localized areas. Furthermore, when a wide-area integrated image is published on a website or the like, the update request receiving unit 4310 may receive update requests from members of the public who browse the website. The priority update area may be a partial area that can be composed of a single partial image, or an area having an area composed of multiple partial images. The update request receiving unit 4310 receives input of, for example, at least one of the acquisition method, shooting altitude, and resolution of a newly acquired image to be used to update the priority update area. The shooting altitude input is not limited to absolute values, but may also be received as a relative value from the shooting altitude of the current partial image. For example, this may be "100 m lower than the current partial image," "half the altitude of the current partial image," or a qualitative request such as "lower altitude."

[0097] The update request receiving unit 4310 may also be configured to receive a confirmation of a measurement request for the received update request. For example, if there are multiple users, a measurement request command may be generated when another user confirms an update request entered by one user. This configuration may allow a response headquarters to be confirmed when an individual response team or a member of the public enters an update request. This configuration facilitates collaboration between multiple users.

[0098] The measurement request unit 4320 is a functional unit that generates a measurement request command for the priority update area. The measurement request command includes information such as the position coordinates of the partial area requiring measurement, the resolution of the update image to be acquired, the measurement method, or the measurement altitude. The measurement request unit 4320 transmits the measurement request command to the aviation data sensing system 1000. Based on this measurement request command, the flight management unit 1100 of the aviation data sensing system 1000 selects the type and individual of the measurement aircraft 110 that will perform the measurement, and makes operational decisions and gives instructions. In addition, the aircraft flight operation unit 1300 determines the mission of the measurement aircraft 110 that will perform the measurement. A measurement plan for the priority update area is determined through a series of processes by the flight management unit 1100 and the aircraft flight operation unit 1300.

[0099] The measurement plan acquisition unit 4330 is a functional unit that acquires a measurement plan (an example of an image acquisition plan) from the airborne data sensing system 1000. The measurement plan may include a scheduled acquisition time at which the measuring instrument 110 acquires a partial image of the priority update area.

[0100] The update plan creation unit 4340 is a functional unit that creates an update plan in accordance with the acquired measurement plan. The update plan creation unit 4340 refers to the scheduled time for each area where the measuring device 110 periodically patrols and acquires newly acquired images, and calculates the scheduled update time for each partial area at which the wide-area integrated image will be updated. The update plan creation unit 4340 also calculates the scheduled update time for the wide-area integrated image based on the scheduled acquisition time at which the measuring device 110 acquires newly acquired images of the priority update area. The update plan creation unit 4340 may calculate the scheduled update time by taking into account the scheduled acquisition time of the newly acquired image, the time for the spatial data processing system 4000 to acquire the newly acquired image, the time for determining whether the newly acquired image is appropriate, and the time for updating the wide-area integrated image using the newly acquired image. The update plan creation unit 4340 calculates the scheduled update time for each partial area.

[0101] (A-1-4-4. Integrated image update unit 4400) The integrated image update unit 4400 is a functional unit that updates the image displayed in a predetermined partial area of ​​the wide-area integrated image to a newly acquired image. The integrated image update unit 4400 mainly functionally includes an updated image appropriateness determination unit 4410 and an image update unit 4420.

[0102] The update image suitability determination unit 4410 is a functional unit that determines whether a newly acquired image is suitable as a partial image to be used for updating, i.e., as an update image. Based on the information in the spatial data information recording unit 4500, the update image suitability determination unit 4410 determines whether the update image is suitable depending on whether the measurement target region of the update image is within the target area F, whether the measurement time is the latest, for example, after a predetermined time, whether the resolution is higher than a predetermined value, etc.

[0103] When a new acquired image of a partial area is acquired after the wide-area integrated image is generated, the image update unit 4420 changes, to the newly acquired image, an image (hereinafter also referred to as a "corresponding image") among the multiple partial images constituting the wide-area integrated image that corresponds to the partial area included in the newly acquired image. With this configuration, even if an image of the target area F is partially acquired, the wide-area integrated image can be partially updated, so that the situation of the target area F, which changes every moment, can be quickly reflected in the wide-area integrated image.

[0104] The image update unit 4420 partially updates the wide-area integrated image with a newly acquired image acquired by the measuring instrument 110 at a different altitude from that of the corresponding image. With this configuration, for example, a partial area configured by a partial image acquired by the artificial satellite 111 can be updated with a partial image acquired by the aircraft 112.

[0105] The image update unit 4420 sets priorities of measurement conditions for updating the wide-area integrated image. That is, for example, the image update unit 4420 may update an image with a first priority being that the image was acquired more recently than the corresponding image, or may update an image with a first priority being that the image has a higher resolution than the corresponding image. The prioritized measurement conditions may be set in advance, or may be selected by the user.

[0106] The image update unit 4420 compares the acquisition time of the newly acquired image with the acquisition time of the corresponding image. If the acquisition time of the newly acquired image is a predetermined time later than the acquisition time of the corresponding image, or if no image has been integrated into the partial area, the image update unit 4420 integrates the newly acquired image into the wide-area integrated image. With this configuration, new partial images can be reflected in the wide-area integrated image. Furthermore, even for partial areas where no partial images exist and geographic data is displayed, updates can be performed as partial images are acquired, allowing for the generation of detailed and up-to-date wide-area integrated images.

[0107] The image update unit 4420 may compare the resolution of the newly acquired image with the resolution of the corresponding image. If the acquisition time of the newly acquired image is a predetermined time or more later than the acquisition time of the corresponding image and the resolution of the newly acquired image is equal to or higher than the resolution of the corresponding image, the image update unit 4420 may integrate the newly acquired image into the wide-area integrated image. With this configuration, the wide-area integrated image can be made high-resolution according to the acquired measurement data.

[0108] In addition, the image update unit 4420 may integrate the newly acquired image into the wide-area integrated image if the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount.

[0109] (A-1-4-5. Spatial Data Information Recording Unit 4500) The spatial data information recording unit 4500 is a functional unit that records acquired spatial data. The spatial data includes measurement data acquired by the measuring instrument 110 as well as geographic data acquired from an appropriate database. The spatial data information recording unit 4500 accumulates measurement data that is adopted in the wide-area integrated image, as well as measurement data that was not adopted in the wide-area integrated image and measurement data that was adopted in the past. The spatial data information recording unit 4500 mainly functionally includes a measurement area recording unit 4510, a measurement time recording unit 4520, a measurement data resolution recording unit 4530, a measurement method recording unit 4540, a measurement altitude recording unit 4550, and a geographic data recording unit 4560.

[0110] The measurement area recording unit 4510 records the position coordinates at which the measurement data was measured. The measurement time recording unit 4520 records the measurement time at which the measurement data was measured. The measurement data resolution recording unit 4530 records the resolution of the measurement data. The measurement method recording unit 4540 records the type of measuring instrument 110 that measured the measurement data. The measurement method recording unit 4540 may also record identification information of the measuring instrument 110. The measurement altitude recording unit 4550 records the measurement altitude at which the measurement data was measured. The geographic data recording unit 4560 records geographic data obtained from the database together with the coordinates of the corresponding partial area.

[0111] (A-1-4-6. Display control unit 4600) The display control unit 4600 is a functional unit that controls the display of an appropriate display device that the user checks. The display device is, for example, a display connected via a network NW, and may be connected to a personal computer or a mobile terminal such as a smartphone or tablet terminal. The display control unit 4600 mainly functionally includes a display condition receiving unit 4610, an integrated image display unit 4620, an update plan display unit 4630, and an update image display unit 4640.

[0112] The display condition receiving unit 4610 receives input of display conditions for the partial image to be displayed from the user. The display conditions include, for example, at least one of the acquisition time, acquisition means, acquisition altitude, and resolution of the partial image to be displayed.

[0113] The integrated image display unit 4620 is a functional unit that displays the wide-area integrated image on a display device. When display conditions are received by the display condition receiving unit 4610, the integrated image display unit 4620 displays partial images that match the received display conditions on the display device. With this configuration, a wide-area integrated image that conforms to the user's desired display conditions can be displayed on the display device.

[0114] The update plan display unit 4630 displays on the display device the update plan for the wide-area integrated image generated by the update plan creation unit 4340. The update plan display unit 4630 may also display a measurement request command or a measurement plan. With this configuration, the update schedule for the wide-area integrated image is clear to the user, and the user can predict the area of ​​interest.

[0115] The updated image display unit 4640 displays the updated wide-area integrated image on the display device. The updated image display unit 4640 may also display the measurement conditions of the displayed partial image for each partial area on the display device. The measurement conditions include, for example, at least one of the update time, the means for acquiring the partial image, or the altitude at which the partial image was captured. For example, when the user selects a partial area of ​​interest by tapping or rolling on the area, the measurement conditions of that area are displayed.

[0116] (A-1-4-7. Notification Unit 4700) The notification unit 4700 is a functional unit that provides a predetermined notification to the user via a display device or the like. This notification may be displayed on the screen, or may be sound, vibration, or the like. For example, when the update of the priority update area is complete, the notification unit 4700 notifies the user that the update is complete. This notification may be displayed as a pop-up on the wide-area integrated image, or an icon may be displayed separately from the wide-area integrated image. The update completion notification for the priority update area may be sent only to the terminal on which the user inputs the priority update area. Furthermore, the notification unit 4700 may display on the display device, for each partial area, the planned update time and planned measurement conditions for reflecting newly acquired images captured of the partial area in the wide-area integrated image based on the measurement plan for the target area. The planned measurement conditions may be, for example, at least one of the acquisition means for the newly acquired image or the shooting altitude of the newly acquired image.

[0117] Furthermore, when the image update process involves work by multiple workers, the notification unit 4700 may estimate the work status of the worker or acquire the work status input by the worker, and notify the user of the work progress status for each worker's pre-set work allocation on the user's display device. The status may be, for example, "not yet assigned," "not yet started," "work in progress," or "work completed." Furthermore, the scheduled image update time may be estimated based on the work status or status, and displayed on the display device.

[0118] Note that some or all of the components of the spatial data processing system 4000 may be implemented in the airborne data sensing system 1000 or in the measuring device 110. When each component is implemented in the aircraft 112, the wide-area integrated image may be created or updated in parallel with the acquisition of measurement data.

[0119] (A-1-5. Flowchart) (A-1-5-1. Overview of Processing) Using Figure 8, an example of the flow of a series of processes executed when acquiring spatial data and updating a wide-area integrated image will be described. First, the spatial data acquisition unit 4100 of the spatial data processing system 4000 acquires measurement data measured by the measuring device 110 (step S101). Next, the integrated image generation unit 4200 generates a wide-area integrated image (step S102). The integrated image display unit 4620 of the display control unit 4600 displays the generated wide-area integrated image on the display device (step S103). Next, the update plan generation unit 4300 generates an update plan (step S104). Next, the spatial data acquisition unit 4100 acquires additional spatial data based on the update plan (step S105).

[0120] Next, the integrated image update unit 4400 updates the wide-area integrated image (step S106). Next, the updated image display unit 4640 of the display control unit 4600 displays the updated wide-area integrated image on the display device. In addition, the notification unit 4650 notifies the display device that the image has been updated (step S107).

[0121] (A-1-5-2. Wide-area Integrated Image Creation Process) Next, the process of creating a wide-area integrated image will be described with reference to Fig. 9. This process is an example of a subroutine of step S102 in Fig. 8.

[0122] First, the spatial data processing system 4000 sets a target area F for creating an integrated image (step S201). Next, the integrated image selection unit 4210 selects images that fit into the target area F (step S202). Next, the image suitability determination unit 4220 excludes images with defects from the selected images (step S203).

[0123] Next, it is determined whether there are multiple images corresponding to the same partial region (step S204). In this case, for example, if the partial regions overlap by a predetermined percentage or more, it may be assumed that they correspond to the same partial region. If there are multiple images (Y in step S204), the process proceeds to step S205. If there is only one image or no images (N in step S204), the process proceeds to step S208.

[0124] In step S205, the acquisition times of multiple partial images corresponding to the same partial region are compared. If the acquisition times of multiple partial images are the same (N in step S205), the partial image with the higher resolution is selected (step S206). If the acquisition times of multiple partial images are different (Y in step S205), the image with the most recent measurement time is selected (step S207). Then, the process proceeds to step S210.

[0125] In step S208, it is determined whether there are any partial regions with no images, i.e., missing areas, within the target area F. If there are missing areas (Y in step S208), a previously acquired partial image corresponding to the missing area or geographic data corresponding to that area is selected (step S209). If there are no missing areas in step S208 (N in step S208), the process proceeds to step S210. The processes of steps S204 to S209 are performed for all partial regions that make up the target area F, and the partial images or geographic data to be adopted are determined one by one.

[0126] In step S210, the adopted partial image is orthorectified, and then the adopted partial image or geographic data is integrated to generate a wide-area integrated image (step S211), and the process ends.

[0127] (A-1-5-3. Process for Generating an Update Plan) The process for generating an update plan will be described with reference to Fig. 10. This process is an example of a subroutine of step S104 in Fig. 8.

[0128] First, the update request receiving unit 4310 receives an update request from the user, such as an area that the user wants to be updated with priority (step S301). In this process, the update request receiving unit 4310 may also receive a designation of the image resolution, measurement method, or measurement altitude to be obtained by the update. Next, the measurement request unit 4320 sends a measurement request to the airborne data sensing system 1000 (step S302). Next, the measurement plan acquisition unit 4330 acquires the measurement plan generated by the airborne data sensing system 1000 (step S303). Next, the update plan creation unit 4340 creates an update plan based on the measurement plan acquired in step S303 (step S304). The update plan display unit 4630 of the display control unit 4600 displays the update plan on the display device (step S305).

[0129] (A-1-5-4. Wide-area Integrated Image Update Processing) An example of the processing flow for updating the wide-area integrated image will be described with reference to Fig. 11. This processing is an example of a subroutine of step S106 in Fig. 8.

[0130] First, the update image suitability determining unit 4410 selects images that satisfy the update request conditions (step S401), and then the update image suitability determining unit 4410 excludes images with defects from the selected images (step S402).

[0131] Next, the update image appropriateness determination unit 4410 determines whether the acquisition time of the additionally acquired additional image is newer than the acquisition time of the pre-update image of the same area (step S403). If the update time of the additional image is newer than the update time of the pre-update image of the same area (Y in step S403), the additional image is selected as the update image (step S404). If the acquisition time of the additional image is not newer than the acquisition time of the pre-update image of the same area, that is, if the additional image was acquired at the same time or before the pre-update image (N in step S403), the image with the higher resolution is determined to be the image to be used for the integrated image (step S405).

[0132] In the example shown in the figure, the process of selecting an image is described in which priority is given to the acquisition time. However, instead, the process of selecting an image may be performed in which priority is given to the resolution. That is, the resolution of the added image and the pre-update image may be compared first, and the image with the higher resolution may be used as the integrated image. If the resolutions of the added image and the pre-update image are the same, the image with the newer acquisition time may be used. Furthermore, whether priority is given to the acquisition time or the resolution when determining an image may be switched based on a user specification.

[0133] Next, it is determined whether there is a missing area where no image exists within the target area F (step S406). If there is a missing area (Y in step S406), the complementary image selection unit 4230 selects an image corresponding to the missing area from previously measured images, or adopts geographic data of the corresponding location (step S407), and the process proceeds to step S408. If there is no missing area in step S406 (N in step S406), the process proceeds to step S408 without going through step S407.

[0134] Next, the image adopted for the image update is orthorectified (step S408), and then the image update unit 4420 updates the image using the orthorectified image (step S409).

[0135] (A-1-6. Screen Example) FIG. 12 is a diagram showing an example of a screen G10 for inputting information about a priority update area received by the update request receiving unit 4310. A wide-area integrated image G11 is displayed on this screen G10. Furthermore, on the wide-area integrated image G11, the boundaries between multiple partial areas constituting the wide-area integrated image G11 are displayed with lines. Note that in this embodiment, the partial areas are rectangular, and therefore the lines representing the boundaries between the partial areas are formed in a grid pattern. However, the shape of the partial areas is not limited to rectangular. Partial images of different sizes may be mixed. The lines representing the boundaries are displayed appropriately depending on the shape of the partial areas. Furthermore, when geographic data is used as a complementary image and the geographic data is displayed in part of the wide-area integrated image, the display control unit 4600 may display the partial area in which the geographic data is displayed in a display mode different from that of the partial area in which the image is displayed. Furthermore, the display may be configured to be displayed in multiple display modes depending on the reliability of the data. For example, the display mode may be varied depending on the resolution or the acquisition time.

[0136] The update request receiving unit 4310 receives input of the priority update area for each partial area. More specifically, the user can input an update request for one or more partial areas on the screen G10 by tapping, clicking, or the like to select the partial areas. The user may also input an update request for the enclosed area by inputting an operation to partially surround the wide-area integrated image G11. In the same figure, the priority update area is displayed with semi-transparent shading, so that the current partial image and the fact that it is a priority update area can be visually recognized together.

[0137] When a predetermined operation is performed on the input priority update area, it may be possible to input measurement conditions for each priority update area. The predetermined operation may be, for example, a long press, a right click, or a predetermined key input. The same figure shows a screen after the predetermined operation has been input, and a rectangular pop-up screen G13 is displayed near the priority update area, allowing the input of the acquisition method (measurement method), acquisition altitude (measurement altitude), and resolution.

[0138] FIG. 13 is a diagram showing an example of a display screen G20 of a wide-area integrated image after an update. The screen G20 displays a wide-area integrated image G21. Furthermore, the screen G20 displays one or more updated partial regions G22 with semi-transparent shading. This configuration makes the updated partial regions clear. Note that this shading may disappear upon a predetermined operation or the passage of a certain period of time. Furthermore, when a predetermined operation is performed on the updated partial region G22, a pop-up screen G23 is displayed, displaying measurement conditions for the partial region G22, etc. The pop-up screen G23 displays, for example, the update date and time, the acquisition method (measurement method), and the acquired altitude (measured altitude) as measurement conditions. The pop-up screen G23 may also display the next scheduled update date and time. Furthermore, statuses indicating the progress of the update, such as "not yet measured," "measured," and "image processing," may be displayed.

[0139] 14 is a diagram showing an example of a screen G30 that is displayed when the display condition receiving unit 4610 receives display conditions for a partial area from the user. When the display condition receiving unit 4610 receives at least one of the conditions of image acquisition time, image acquisition means, image acquisition altitude, and image resolution as a display condition, an image G32 that matches the condition is selected and displayed. The set display condition may be displayed in a predetermined area G33. Furthermore, if there is no image that matches the display condition, a blank area G34 may be displayed. Note that geographic data may be displayed in the blank area G34.

[0140] The figure shows an example of a case where only partial images whose update date and time is after 12:30 on 03 / 09 / 2015 are extracted and displayed. This example illustrates a situation where a river flood situation needs to be quickly grasped, and the aircraft 112 frequently flies over the area around the river and acquires images, resulting in frequently updated images of the area around the river. Therefore, in the figure, images that match the display condition "Update date and time: after 12:30 on 03 / 09 / 2015" are the area around the river.

[0141] [A-2. Effects of this embodiment] According to this embodiment, it is possible to quickly update the captured images of the target area.

[0142] The present invention is not limited to the above-described embodiment, and it goes without saying that various configurations can be adopted based on the contents of the present specification.

[0143] The series of processes described in connection with the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the image updating system 1 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a communication network NW without using a recording medium.

[0144] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

[0145] 1 Aviation data sensing system 1000 Aircraft control system 1100 Flight management unit 1200 Airspace surveillance and control unit 1300 Aircraft flight operation unit 1400 Acquired data management unit 1500 Communication infrastructure management unit 110 Measuring instrument 111 Satellite 112 Aircraft 4000 Spatial data processing system (image update system) 4100 Spatial data acquisition unit 4200 Integrated image generation unit 4300 Update plan generation unit 4400 Integrated image update unit 4500 Spatial data information recording unit 4600 Display control unit 4700 Notification unit F Target area

Claims

1. A method for updating a wide-area integrated image generated by integrating a plurality of different partial images obtained by photographing partial areas into which a target area is subdivided from the air, the image updating method comprising: an integrated image generating step of integrating the plurality of partial images to generate the wide-area integrated image; and an updating step of, when a newly acquired image of the partial area is newly acquired after the generation of the wide-area integrated image, updating a corresponding image of the plurality of partial images constituting the wide-area integrated image that corresponds to the partial area included in the newly acquired image to the newly acquired image.

2. An image updating method as described in claim 1, wherein the updating step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and in the updating step, if the acquisition time of the newly acquired image is a predetermined time or more later than the acquisition time of the corresponding image, the newly acquired image is integrated into the wide-area integrated image.

3. An image updating method as described in claim 1, wherein the updating step includes a corresponding image determination step of determining whether the corresponding image corresponding to the newly acquired image is integrated into the wide-area integrated image, and in the updating step, if the corresponding image is not integrated into the wide-area integrated image, the newly acquired image is integrated into the wide-area integrated image.

4. An image updating method as described in claim 1, wherein the updating step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image, and in the updating step, when the acquisition time of the newly acquired image is a predetermined time later than the acquisition time of the corresponding image and the resolution of the newly acquired image is equal to or greater than the resolution of the corresponding image, the newly acquired image is integrated into the wide-area integrated image.

5. An image updating method as described in claim 1, wherein the updating step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image, and in the updating step, if the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount or more, the newly acquired image is integrated into the wide-area integrated image.

6. The image updating method according to claim 1, wherein in said updating step, said wide-area integrated image is partially updated with said newly acquired image acquired by an imaging means at an altitude different from that of said corresponding image.

7. The image updating method of claim 1, wherein the integrated image generating step includes an image suitability determination step for determining whether a plurality of partial images acquired at a first shooting altitude are suitable for inclusion in the wide-area integrated image; a complementation processing step for extracting an area of ​​the target area for which an usable partial image has not been acquired, and for that area using a partial image captured at a second shooting altitude different from the first shooting altitude as a complementation image; and an image integration step for integrating the complementation image into the wide-area integrated image.

8. The image updating method of claim 1, wherein the integrated image generation step includes: an image suitability determination step for determining whether the acquired partial images are suitable for inclusion in the wide-area integrated image; a complementation processing step for extracting areas of the target area for which an adoptable partial image has not been acquired, and for those areas using partial images or geographic data acquired earlier than the partial images as complement images; and an image integration step for integrating the complement images into the wide-area integrated image.

9. An image updating method as described in claim 1, wherein the multiple partial images integrated in the integrated image generation step have overlapping areas in which adjacent partial images capture the same area, and in the integrated image generation step, the qualities of the adjacent partial images are compared, and the partial image with the better quality is adopted as the image in the overlapping area.

10. The image updating method according to claim 1, further comprising: an update request receiving step of receiving an input of a priority update area in said wide-area integrated image for which said image is to be updated preferentially.

11. The image updating method according to claim 10, wherein said update request receiving step receives input of at least one of the acquisition means, the shooting altitude, and the resolution of a newly acquired image to be used for updating said priority update area.

12. The image updating method according to claim 10, further comprising: a measurement request step for generating a measurement command for the priority update area, or a measurement plan generating step for generating a measurement plan; and a display control step for displaying the measurement command or the measurement plan on a display device.

13. The image updating method according to claim 10, further comprising the step of: notifying the user that the updating of the priority update area has been completed when the updating of the priority update area has been completed.

14. The image update method described in claim 1, further comprising a display control step of displaying on a display device, for each partial area, based on a future image acquisition plan for the target area, at least one of a scheduled update time at which the newly acquired image at which the partial area was photographed will be reflected in the wide-area integrated image, the acquisition means for the newly acquired image, or the shooting altitude of the newly acquired image.

15. The image updating method according to claim 1, further comprising a display control step of displaying, on a display device, at least one of the update time of the displayed partial image, the acquisition means of the partial image, and the shooting altitude of the partial image for each partial area.

16. An image updating method as described in claim 1, further comprising: a display condition receiving step of receiving input of display conditions of the partial image to be displayed from a user, the display conditions including at least one of the acquisition time, acquisition means, acquisition altitude or resolution of the partial image to be displayed; and a display control step of displaying on a display device the partial image that matches the display conditions received in the display condition receiving step.

17. An image updating system that updates a wide-area integrated image generated by integrating a plurality of different partial images taken from the air of partial areas into which a target area is divided, comprising: an integrated image generating unit that integrates the plurality of partial images to generate the wide-area integrated image; and an image updating unit that, when a newly acquired image of the partial area is acquired after the generation of the wide-area integrated image, extracts a partial image captured of the partial area included in the newly acquired image from the plurality of partial images constituting the wide-area integrated image as a corresponding image, and updates the corresponding image in the wide-area integrated image to the newly acquired image.

18. An image updating program for a wide-area integrated image generated by integrating a plurality of different partial images captured from the air of partial areas into which a target area is divided, the image updating program causing a computer to execute the following: an integrated image generating command for integrating the plurality of partial images to generate the wide-area integrated image; a corresponding image extraction command for, when a newly acquired image of the partial area is acquired after the generation of the wide-area integrated image, extracting, as a corresponding image, a partial image captured of the partial area included in the newly acquired image from the plurality of partial images constituting the wide-area integrated image; and an update command for updating, in the wide-area integrated image, the corresponding image to the newly acquired image.

Images