Method, system and program for updating image

Abstract

To swiftly update a photographic image of an area concerned.SOLUTION: A method for updating a wide-area integrated image generated by integrating together a plurality of mutually-different partial images taken from high in the skies of partial areas into which an area concerned F is segmentalized, includes allowing a computer to execute: an integrated-image generating step S102 of integrating together a plurality of partial images to generate a wide-area integrated image; and an updating step S106 of, if a newly-obtained image of a partial area is newly obtained after generating the wide-area integrated image, updating a corresponding image corresponding to a partial area included in the newly obtained image out of the plurality of partial images constituting the wide-area integrated image into the newly obtained image.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to an image update method, an image update system, and an image update program.

Background Art

[0002] Patent Document 1 discloses a method for generating ground surface image data, which determines a defective area in which clouds are reflected and a partial area in which the ground surface is reflected from a geostationary satellite image, acquires geostationary satellite images of the defective area at different times and stores them as complementary images, and synthesizes the satellite image of the partial area and the complementary image.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When trying to utilize a wide-area integrated image obtained by integrating a plurality of captured images taken from above for the purpose of situation confirmation when a disaster or other emergency occurs, it is required to generate and update the wide-area integrated image at an early stage.

[0005] In this regard, in the method described in Patent Document 1, it is necessary to acquire a ground surface image by a geostationary satellite again when re-photographing the defective area, and it was not possible to generate and update the ground surface image data at an early stage.

[0006] The present invention has been made in consideration of the above problems, and an object thereof is to quickly generate or update a captured image of a target area.

Means for Solving the Problems

[0007] To achieve the above object, an image update method according to one aspect of the present invention is an update method for a wide-area integrated image generated by integrating a plurality of different partial images taken from above the sky of partial areas obtained by subdividing a target area, wherein a computer executes an integrated image generation step of integrating the plurality of partial images to generate the wide-area integrated image, and an update step of, when a newly acquired image of the partial area is newly acquired after the generation of the wide-area integrated image, updating a corresponding image corresponding to the partial area included in the newly acquired image to the newly acquired image among the plurality of partial images constituting the wide-area integrated image.

[0008] The update step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and in the update step, when the acquisition time of the newly acquired image is after the acquisition time of the corresponding image by a predetermined time or more, the newly acquired image may be integrated into the wide-area integrated image.

[0009] The update step includes a corresponding image determination step of determining whether the corresponding image corresponding to the newly acquired image is integrated into the wide-area integrated image, and in the update step, when the corresponding image is not integrated into the wide-area integrated image, the newly acquired image may be integrated into the wide-area integrated image.

[0010] The update step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image, and in the update step, when the acquisition time of the newly acquired image is after the acquisition time of the corresponding image by a predetermined time or more and the resolution of the newly acquired image is equal to or higher than the resolution of the corresponding image, the newly acquired image may be integrated into the wide-area integrated image.

[0011] The update step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image. In the update step, when the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount or more, the newly acquired image may be integrated into the wide-area integrated image.

[0012] In the update step, the wide-area integrated image may be partially updated with the newly acquired image obtained by a photographing means highly different from the corresponding image.

[0013] In the integrated image generation step, in a plurality of partial images acquired at a first photographing altitude, an image suitability determination step of determining the suitability for adoption into the wide-area integrated image, a complementation processing step of extracting an area in the target area where an adoptable partial image has not been acquired and using, as a complementary image, a partial image photographed at a second photographing altitude different from the first photographing altitude in the area, and an image integration step of integrating the complementary image into the wide-area integrated image may be included.

[0014] The integrated image generation step may include an image suitability determination step of determining the suitability for adoption into the wide-area integrated image in a plurality of acquired partial images, a complementation processing step of extracting an area in the target area where an adoptable partial image has not been acquired and using, as a complementary image, a partial image acquired earlier than the partial image or geographical data in the area, and an image integration step of integrating the complementary image into the wide-area integrated image.

[0015] In the integrated image generation step, the plurality of partial images to be integrated have an overlap region that images the same region in the partial images adjacent to each other. In the integrated image generation step, the quality of the partial images adjacent to each other may be compared, and the partial image with better quality may be adopted as the image in the overlap region.

[0016] It may further execute an update request reception step of receiving an input of a priority update area that preferentially updates the image in the wide-area integrated image.

[0017] In the update request reception step, it may receive an input of at least one of an acquisition means, a shooting altitude, or a resolution of a newly acquired image to be used for updating the priority update area.

[0018] It may further execute a measurement request step of generating a measurement command for the priority update area or a measurement plan generation step of generating a measurement plan, and a display control step of displaying the measurement command or the measurement plan on a display device.

[0019] When the update of the priority update area is completed, it may further execute a notification step of notifying that the update has been completed.

[0020] Based on the future image acquisition plan of the target area, for each partial area, a display control step of displaying on a display device at least one of a scheduled update time for reflecting the newly acquired image in which the partial area was photographed in the wide-area integrated image, an acquisition means of the newly acquired image, or a shooting altitude of the newly acquired image may be further executed.

[0021] For each partial area, a display control step of displaying on a display device at least one of an update time of the displayed partial image, an acquisition means of the partial image, or a shooting altitude of the partial image may be further executed.

[0022] Further execute a display condition reception step of receiving an input of a display condition of the partial image to be displayed from a user, where the display condition includes at least any one of an acquisition time, an acquisition means, an acquisition altitude, or a resolution of the partial image to be displayed, and a display control step of displaying on a display device the partial image that matches the display condition received in the display condition reception step.

[0023] To achieve the above object, an image update system according to another aspect of the present invention is an image update system for updating a wide-area integrated image generated by integrating a plurality of different partial images taken from above the sky of partial areas obtained by subdividing a target area, including an integrated image generation unit that integrates the plurality of partial images to generate the wide-area integrated image, and after generating the wide-area integrated image, when newly acquiring a new acquired image of the partial area, extracting, as a corresponding image, a partial image that has photographed the partial area included in the new acquired image from among the plurality of partial images constituting the wide-area integrated image, and an image update unit that updates the corresponding image to the new acquired image in the wide-area integrated image.

[0024] To achieve the above object, an image update program according to still another aspect of the present invention is an image update program for a wide-area integrated image generated by integrating a plurality of different partial images taken from above the sky of partial areas obtained by subdividing a target area, causing a computer to execute an integrated image generation instruction for integrating the plurality of partial images to generate the wide-area integrated image, a corresponding image extraction instruction for extracting, as a corresponding image, a partial image that has photographed the partial area included in the new acquired image from among the plurality of partial images constituting the wide-area integrated image when newly acquiring a new acquired image of the partial area after generating the wide-area integrated image, and an update instruction for updating the corresponding image to the new acquired image in the wide-area integrated image.

[0025] Note that the computer program can be stored and provided in various data-readable recording media, or provided so as to be downloadable via a network such as the Internet.

Advantages of the Invention

[0026] According to the present invention, a captured image of a target area can be updated quickly.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Further, the embodiments shown below are merely examples, and other known elements and alternative means can be adopted according to the use, purpose, scale, etc.

[0029] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overview)

[0030] The spatial data processing system 4000 integrates a plurality of measurement data partially acquired by the measuring instrument 110 for the target area F (FIG. 2) and generates a wide-area integrated image. The target area F is a two-dimensional area to be the target of data acquisition by the measuring instrument 110, and may be, for example, on water or on the ground. The measuring instrument 110 may exist on the target area F or may be photographed from outside the target area F. The measurement data is appropriate data representing the situation of the ground surface, and is, for example, an image, but may be point cloud data.

[0031] As shown in FIG. 1, an image update system 4000 (hereinafter also referred to as "spatial data processing system 4000") according to an embodiment of the present invention is connected to an airborne data sensing system 1000, a facility equipment management system 5000, and a reservation system 6000 through a network NW. Further, the airborne data sensing system 1000 is connected to the measuring instrument 110. The measuring instrument 110 and the network NW may be wirelessly connected to an appropriate artificial satellite.

[0032] The aviation data sensing system 1000 is a system that controls the measuring instrument 110. The aviation data sensing system 1000 controls each part of the measuring instrument 110, makes the measuring instrument 110 run along a defined route, and causes the measuring instrument 110 to photograph the target area.

[0033] The spatial data processing system 4000 is a system that accumulates measurement data such as images or point cloud data acquired by, for example, the measuring instrument 110. The spatial data processing system 4000 has a function of performing integrated processing with map data and other information and publishing a wide-area integrated image. In addition, the spatial data processing system 4000 may be mounted on a moving body such as a vehicle in addition to being mounted on a facility fixed to the ground surface.

[0034] The equipment and material management system 5000 is a system that manages the equipment and materials at the base where the aircraft 112 takes off and lands.

[0035] The reservation system 6000 is a system that manages the reservation of the base, and manages, for example, the reservation of resources for various operations at the base. The resources include all resources necessary for data collection, such as receiving equipment used for communication and maintenance with the aircraft 112, the landing area, or the number of operators. In addition, the reservation system 6000 may hold information on the time period during which the resources are occupied or the time when the resources are released according to the reservation. The operation management unit 1100 described later creates a measurement plan in consideration of the reservation status of the base and the like. In addition, the reservation system 6000 updates the reservation information of the base based on the measurement plan generated by the operation management unit 1100.

[0036] Note that the communication mode of each component shown in FIG. 1 is arbitrary, and they may be communicably connected to each other via a communication network such as an Internet line or a communication method such as LTE. In addition, each component may be communicatively connected to the communication network by satellite communication via a satellite. Each wireless communication may form a dedicated wireless communication network or may utilize an existing wireless infrastructure.

[0037] (A-1-2. Measuring Device 110)

[0038] As shown in FIG. 2, the measuring device 110 is a device that acquires measurement data of a partial area F110 obtained by subdividing the target area F. The measuring device 110 is an example of imaging means. The measuring device 110 particularly moves with respect to the ground surface, and performs imaging of the target area F while changing the partial area F110 to be imaged as it moves. Further, the measuring device 110 may perform imaging of the target area F by changing the partial area F110 by changing the orientation of the sensor.

[0039] A plurality of measuring devices 110 are included in one aerial data sensing system 1000. The aerial data sensing system 1000 can simultaneously image a plurality of partial areas F110 by flying a plurality of measuring devices 110 simultaneously over one target area F to acquire measurement data.

[0040] The measuring device 110 includes a plurality of aircraft that perform measurements at different measurement altitudes. The measuring device 110 is, for example, an artificial satellite 111 orbiting around the Earth or an aircraft 112. Further, the aircraft 112 may include a rotary-wing aircraft 112a and a fixed-wing aircraft 112b. In this case, the artificial satellite 111 performs measurements at a higher measurement altitude than the aircraft 112. Also, the rotary-wing aircraft 112a performs measurements at a relatively low altitude. The fixed-wing aircraft 112b performs measurements at an altitude equal to or higher than that of the rotary-wing aircraft 112a and lower than that of the artificial satellite 111. The altitude range of each measuring device 110 is determined in advance according to the aircraft type. Also, the altitude of each measuring device 110 may be variable based on a command from the aerial data sensing system 1000. (A-1-2-1. Artificial Satellite 111) The artificial satellite 111 performs measurements of the surface of the target area from an orbit around the Earth. The measurements may be, in addition to image capture, the acquisition of point cloud data by a laser sensor (LiDAR). The artificial satellite 111 may be a satellite exclusively used by this system or a general-purpose artificial satellite 111. The artificial satellite 111 orbits around the Earth along a predetermined path and is, for example, a low-earth orbit satellite, but may also be a geostationary satellite or a medium-earth orbit satellite. Also, in FIGS. 1 and 2, only one artificial satellite 111 is depicted, but a plurality of artificial satellites 111 may be connected to the spatial data processing system 4000. The spatial data processing system 4000 may have a plurality of artificial satellites 111 at different altitudes connected thereto.

[0041] (A-1-2-2. Aircraft 112) The aircraft 112 is, for example, an unmanned aerial vehicle. In this specification, the term "aircraft" refers to any flying object having the function of autonomously controlling its attitude regardless of the power means (electric power, prime mover, etc.) and the control method (wireless or wired, fully autonomous flight type or partially manually controlled type, etc.), and regardless of whether it is manned or unmanned. Also, the aircraft may be referred to as an unmanned aerial vehicle (UAV), a flying object, a multi-copter, an RPAS (Remote Piloted Aircraft Systems), or a UAS (Unmanned Aircraft Systems), etc.

[0042] The aircraft 112 is not limited to the illustrated embodiment, and an appropriate configuration can be adopted. For example, the aircraft 112 may be a vertical take-off and landing aircraft (VTOL) having both fixed wings and rotary wings in addition to a rotary-wing aircraft 112a or a fixed-wing aircraft 112b. Also, when equipped with rotary wings, a propeller guard (not shown) may be provided to prevent interference of the rotary wings with obstacles. The measuring device 110 may have an alarm device, such as a warning light and a speaker, for warning people around the measuring device 110.

[0043] FIG. 3 is a schematic diagram showing the paths of a plurality of types of measuring devices 110 and the states of images acquired by the measuring devices 110. The arrows on each of the paths R111a, R112a, and R112b indicate the moving paths of the measuring devices 110. In the example of this figure, when the artificial satellite 111 moves along the path R111a, it acquires an image P111a having a width H111a along the path R111a. Also, another artificial satellite 111 moves along the path R111b and acquires an image P111b along the path R111b. Since the artificial satellite 111 moves along the satellite orbit to acquire images, there are partial regions where measurement data cannot be acquired.

[0044] In that regard, the aircraft 112 can fly to an arbitrary position and acquire images by generating a flight path by the airborne data sensing system 1000. In the example of this figure, the rotary-wing aircraft 112a flies along the path R112a and acquires an image P112a having a width H112a along the path R112a. The fixed-wing aircraft 112b flying at a higher altitude than the rotary-wing aircraft 112a flies along the path R112b and acquires an image P112b having a width H112b along the path R112b.

[0045] The width of the acquired image is larger as the measurement altitude is higher. That is, the imaging width H111a of the artificial satellite 111 is the largest, followed by the imaging width H112b of the fixed-wing aircraft 112b, and the imaging width H112a of the rotary-wing aircraft 112a is the smallest. Since the artificial satellite 111 flies at an extremely high altitude, it only passes over the target area F, and a wide-width image can be acquired. On the other hand, the fixed-wing aircraft 112b flying at a high altitude flies back and forth over the target area F with a round-trip width H112a without measurement omission to acquire an image of the target area F. Since the rotary-wing aircraft 112a flying at a low altitude has a low altitude, it flies back and forth with a smaller imaging width H112a than the fixed-wing aircraft 112b flying at a high altitude to acquire an image of the target area F without omission. Also, because of the low altitude, a high-resolution image can be acquired. The resolution of the image is the highest for the rotary-wing aircraft 112a, followed by the fixed-wing aircraft 112b, and the image by the artificial satellite 111 has the lowest resolution. Thus, the spatial data processing system 4000 can efficiently acquire a wide-area integrated image representing the target area F by integrating the acquired images after measuring the target area by mutually different measurement methods.

[0046] In addition, since the aircraft 112 can acquire an image of a desired flight path, it can acquire an image of a desired area more quickly than the artificial satellite 111. According to such a configuration, even in a case where the situation of the target area changes moment by moment, such as during a disaster, the situation of the area can be quickly visualized by the user.

[0047] (A-1-2-3. Functional Blocks of Satellite Measurement System 1110) As shown in FIG. 4, the satellite measurement system 1110 includes an arithmetic device such as a CPU for executing information processing, and storage devices such as a RAM and a ROM. Thus, as a software configuration, it mainly has a sensor unit 1111, a self-position and attitude determination unit 1112, a measurement target position determination unit 1113, a data recording unit 1114, and a communication unit 1115 functionally.

[0048] (A-1-2-3-1. Sensor Unit 1111) The sensor unit 1111 is configured to measure the target area F. The sensor unit 1111 mainly includes a laser sensor 1111a and an optical camera 1111b. The laser sensor 1111a acquires point cloud data of the ground surface by, for example, LiDAR. The optical camera 1111b is configured to capture an image of the ground surface. Note that the sensor unit 1111 may include only either the laser sensor 1111a or the optical camera 1111b. Further, the sensor unit 1111 may acquire the acquisition time of the measurement data together with the measurement data.

[0049] (A-1-2-3-2. Self-position and attitude determination unit 1112) The self-position and attitude determination unit 1112 is a functional unit that determines the position and attitude of the artificial satellite 111. The self-position and attitude determination unit 1112 may record the measured altitude of the artificial satellite 111.

[0050] (A-1-2-3-3. Measurement target position determination unit 1113) The measurement target position determination unit 1113 is a functional unit that determines a partial area (hereinafter, also referred to as "measurement target area") for which the acquired measurement data is a measurement target. The measurement target position determination unit 1113 calculates the measurement target area from the position and attitude of the artificial satellite 111. The measurement target position determination unit 1113 calculates, for example, the absolute position coordinates of the area. Further, in a mode in which partial areas obtained by previously subdividing the target area F are stored, the measurement target position determination unit 1113 may perform a process of specifying a partial area corresponding to the acquired measurement data.

[0051] (A-1-2-3-4. Data recording unit 1114) The data recording unit 1114 is a functional unit that records the measurement data obtained by the measuring machine 110 on a recording medium inside the measuring machine 110. The data recording unit 1114 acquires the measurement data obtained by the sensor unit 1111 and the related information of the measurement data. The related information is the acquisition time of the measurement data, the position coordinates obtained by the self-position and attitude determination unit 1112, the measured altitude, or the position information of the measurement target area obtained by the measurement target position determination unit 1113, etc. The recording medium may include a separable medium such as an SD card or a RAM, for example.

[0052] (A-1-2-3-5. Communication Unit 1115) Communication unit 1115 can perform radio communication via communication network NW. For example, it includes a radio communication module. By means of communication network NW, communication unit 1115 can communicate with measuring instrument 110 etc. Communication unit 1115 has a communication function for performing wireless communication with measuring instrument 110 by wireless communication using, for example, frequency bands of Wi-Fi, 2.4 GHz, and 5.6 - 5.8 GHz. Also, communication unit 1115 has a wireless communication function that can communicate with spatial data processing system 4000 via communication network NW using communication standards such as LTE (Long Term Evolution). Communication unit 1115 transmits measurement data, related information, etc. to spatial data processing system 4000 etc. Communication unit 1115 may perform real-time transmission or transmit data in a manner of post hoc transmission.

[0053] (A-1-2-4. Functional Blocks of Aircraft Measurement System 1120) Aircraft measurement system 1120 is a system for controlling aircraft 112. As shown in FIG. 5, aircraft measurement system 1120 includes an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM. Thus, as a software configuration, it mainly has, functionally, sensor unit 1121, self-position and attitude determination unit 1122, measurement target position determination unit 1123, flight control unit 1124, data recording unit 1125, and communication unit 1126.

[0054] (A-1-2-4-1. Sensor Unit 1121) The sensor unit 1121 is a sensor that measures the target area F and acquires measurement data. The sensor unit 1121 is held, for example, below or in front of the fuselage of the aircraft 112. The fuselage may be provided with a plurality of sensor units 1121. The sensor unit 1121 is, for example, a camera that acquires an image of the target area F, but may also be a sensor such as LiDAR (Light Detection And Ranging), or a microphone. Further, the sensor unit 1121 may be an IR camera in addition to a visible light camera. That is, the sensing data acquired by the sensor unit 1121 may include, for example, an image, an IR image, point cloud data, or position data.

[0055] The orientation of the sensor unit 1121 can be adjusted by a sensor actuator (not shown). The sensor unit 1121 may have an automatic control function for parameters such as exposure, contrast, or ISO. The holding unit of the sensor unit 1121 may have a so-called gimbal control mechanism that suppresses the transmission of the shaking or vibration of the fuselage to the sensor unit 1121. The sensor unit 1121 controls the sensor unit 1121 and the holding unit to adjust the shooting range of the camera, etc.

[0056] The measurement data acquired by the sensor unit 1121 is transmitted to the spatial data processing system 4000. Further, the measurement data may be stored in the data recording unit 1125 of the measuring instrument 110 itself.

[0057] (A-1-2-4-2. Self-position and attitude determination unit 1122) The own position and attitude determination unit 1122 receives signals from artificial satellites and measures the position (absolute position) of the aircraft 112 based on the signals. The own position and attitude determination unit 1122 is not particularly limited, and for example, uses GNSS (Global Navigation Satellite System), GPS (Global Positioning System), etc. to measure its own position at the current time. As a method for measuring the own position, for example, RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System) can also be used. The position information includes at least two-dimensional coordinate information (for example, latitude and longitude) in a plan view, and preferably includes three-dimensional coordinate information including altitude information.

[0058] (A-1-2-4-3. Measurement target position determination unit 1123) The measurement target position determination unit 1123 is a functional unit that determines the measurement target area of the acquired measurement data. The measurement target position determination unit 1123 calculates the measurement target area from the position and attitude of the aircraft 112. The measurement target position determination unit 1123 calculates, for example, the absolute position coordinates of the area. Further, in a mode in which partial areas obtained by previously subdividing the target area F are stored, the measurement target position determination unit 1123 may perform a process of specifying the partial area corresponding to the acquired measurement data.

[0059] (A-1-2-4-4. Flight control unit 1124) The flight control unit 1124 is a mechanism and functional unit that operates and flies the aircraft 112, and generates thrust for the aircraft to lift and move in a desired direction. The flight control unit 1124 mainly includes a motor control unit 1124a and a motor 1124b.

[0060] The flight control unit 1124 has a processing unit, also called a flight controller. The processing unit can have one or more processors such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit is accessible to a memory (storage unit). The memory stores logic, code, and / or program instructions executable by the processing unit to perform one or more steps.

[0061] The processing unit includes a control module configured to control the state of the airframe of the aircraft 112. For example, the control module adjusts the spatial arrangement, attitude angle, angular velocity, angular acceleration, angular jerk velocity, and / or acceleration of the aircraft 112 having six degrees of freedom (translational motions x, y, and z, and rotational motions θx, θy, and θz). That is, the flight control unit 1124 causes the aircraft 112 to perform various operations such as takeoff, forward movement, turning, and landing, and controls the attitude angle control and flight operations of the aircraft 112 from takeoff to in-flight and landing.

[0062] The flight control unit 1124 can control the flight of the aircraft 112 based on, for example, an autonomous flight program acquired from the airframe operation unit 1300. The flight control unit 1124 can also control the flight of the aircraft 112 by controlling the motor 1124b based on various information such as the target area F, the flight permission / forbidden area, information on the corresponding flight defense, map information including two-dimensional or three-dimensional map data, the current position information of the aircraft 112, attitude information (nose azimuth information), speed information, and acceleration information, and any combination thereof.

[0063] The flight control unit 1124 may control the orientation and zoom amount of the sensor unit 1121. Regarding the orientation of the sensor unit 1121, the flight control unit 1124 may control either or both of the pitch angle with respect to the horizontal and the yaw angle with respect to a predetermined reference direction.

[0064] (A-1-2-4-5. Data recording unit 1125) The data recording unit 1125 stores, for example, the measurement data and related information measured by the aircraft 112.

[0065] (A-1-2-4-6. Communication unit 1126) The communication unit 1126 has a modem or the like (not shown) and is capable of communicating with the aviation data sensing system 1000 or the space data processing system 4000 or the like via the communication network NW.

[0066] (A-1-3. Aviation data sensing system 1000) (A-1-3-1. Functional blocks of the aviation data sensing system 1000) FIG. 6 is a functional configuration diagram of the aviation data sensing system 1000 of the present embodiment. The aviation data sensing system 1000 is connected to the measuring machine 110 and manages or controls the flight and shooting of the measuring machine 110.

[0067] The aviation data sensing system 1000 includes an arithmetic device such as a CPU for executing information processing, and storage devices such as a RAM and a ROM. As a software configuration, it mainly constitutes functional blocks of an operation management unit 1100, an airspace monitoring and control unit 1200, an aircraft operation operate unit 1300, an acquired data management unit 1400, and a communication infrastructure management unit 1500. (A-1-3-2. Operation management unit 1100) The operation management unit 1100 is a system that makes decisions and gives instructions for the operation of the measuring machine 110. The operation management unit 1100 formulates, for example, a plan regarding the work of the aircraft 112 including measurement and flight, and transmits it to the aircraft 112 via the communication infrastructure management unit 1500. The operation management unit 1100 may formulate plans for a plurality of aircraft 112 and transmit the information of each plan thereto. Further, the operation management unit 1100 may determine the priority order of the plans of the plurality of aircraft 112 and determine a work plan according to the priority order. In addition, the operation management unit 1100 receives a control requirement for the work airspace required for the measurement of the target area F, and performs mediation with an external system by referring to the environment of the work airspace and information of other aircraft.

[0068] When the operation management unit 1100 receives an update request at a predetermined resolution from the update request reception unit 4310 of the spatial data processing system 4000 described later, it may select an aircraft capable of shooting at that resolution. In this case, the operation management unit 1100 stores in association with each other the aircraft that can be controlled, the resolution at which shooting is possible, the measurable altitude, etc., and may select an aircraft with reference to this.

[0069] Note that although the operation management unit 1100 has been described as being included in the aviation data sensing system 1000 in the present embodiment, it may be a configuration included in the measuring device 110 or the spatial data processing system 4000.

[0070] (A-1-3-3. Airspace Surveillance and Control Unit 1200) The airspace surveillance and control unit 1200 communicates with other flying objects existing in the working airspace and grasps the positions of the flying objects. The flying objects may include manned aircraft and unmanned aircraft. According to this configuration, it is possible to avoid contact with other flying objects and ensure the safety of the aircraft 112 during flight.

[0071] (A-1-3-4. Aircraft Operation and Control Unit 1300) The aircraft operation and control unit 1300 is a functional unit that generates a mission for the aircraft 112 and controls the movement of the aircraft 112. The mission is, for example, a movement plan including the movement route and movement speed of the aircraft 112. The aircraft operation and control unit 1300 transmits a control signal to the aircraft 112 via the communication infrastructure management unit 1500 in order to automatically operate the aircraft 112.

[0072] In addition, the aircraft operation operator unit 1300 may determine the measured altitude. Since the resolution of the measurement data acquired by each measuring instrument 110 varies depending on the measured altitude, the aircraft operation operator unit 1300 determines the measured altitude based on, for example, the resolution received by the update request reception unit 4310. Also, the resolution varies depending on the settings of the mounted sensor unit 1121 or the configuration of the lens. Therefore, the aircraft operation operator unit 1300 may change the settings of the sensor unit 1121 of the measuring instrument 110 so that an image at the resolution can be acquired.

[0073] (A-1-3-5. Acquisition Data Management Unit 1400) The acquisition data management unit 1400 is a functional unit that manages acquisition data including video or images. The acquisition data management unit 1400 determines the processing for a huge amount of acquisition data for each piece of acquisition data, and transmits a processing instruction to the aviation data sensing system 1000, the spatial data processing system 4000, the facility equipment management system 5000, the reservation system 6000, etc. via the communication infrastructure management unit 1500. Also, the acquisition data management unit 1400 records, in association with the measurement data, related information such as the measured partial area, the measurement time, the resolution, the measurement method, and the measured altitude for each acquired measurement data.

[0074] (A-1-3-6. Communication Infrastructure Management Unit 1500) The communication infrastructure management unit 1500 is a system that manages communication means for transmitting and receiving data between the aviation data sensing system 1000 and the spatial data processing system 4000. The communication infrastructure management unit 1500 transmits, for example, the information generated by the measurement request unit 4320 to the aviation data sensing system 1000. Also, the communication infrastructure management unit 1500 transmits the measurement plan generated by the operation management unit 1100 to the spatial data processing system 4000, and the measurement plan acquisition unit 4330 acquires this.

[0075] In addition, the communication infrastructure management unit 1500 manages a plurality of different types of communication means. The communication infrastructure management unit 1500 monitors the communication availability of the existing infrastructure that is set to be used in principle and the communication speed of communication in communication means different from the existing infrastructure, and makes a determination such as the speed or urgency of data transmission and reception, and then may determine the data transmission and reception means. The options for the data transmission and reception means may include, for example, communication methods such as parallel transmission and switching transmission, and also the mode in which the operator physically transports the recording memory.

[0076] The configuration of the aviation data sensing system 1000 may be realized as a single device, or may be realized by a plurality of devices (such as the measuring instrument 110 and the operation management unit 1100) partially or entirely connected by a communication network NW. Furthermore, each functional unit of the aviation data sensing system 1000 may be logically realized by cloud computing.

[0077] A plurality of aviation data sensing systems 1000 may be connected to one measuring instrument 110 via a plurality of communication networks NW, that is, the system may be redundant. In this case, even when an abnormality occurs in the operation management unit 1100 or the communication network, the measurement data acquisition process by the measuring instrument 110 can be continued by the redundant other operation management unit 1100 or the communication network NW, so that the reliability of the spatial data processing system 4000 can be improved.

[0078] (A-1-4. Spatial Data Processing System 4000) The spatial data processing system 4000 shown in FIG. 7 is a system that generates a wide-area integrated image of a target area by integrating partial images as measurement data, appropriately updates the wide-area integrated image with the acquired measurement data, and enables the user to visually recognize the situation of the target area. In addition, the spatial data processing system 4000 may perform output processing that is compatible with an external system and output the information included in the wide-area integrated image to the external system. The external system is optional and may be, for example, a business support GIS (Geographic Information System) such as PasCAL (registered trademark), OPEN GIS, Arc GIS, etc.

[0079] The spatial data processing system 4000 includes an arithmetic device such as a CPU for executing information processing, and storage devices such as a RAM and a ROM. As a software configuration, it mainly has a spatial data acquisition unit 4100, an integrated image generation unit 4200, an update plan generation unit 4300, an integrated image update unit 4400, a spatial data information recording unit 4500, a display control unit 4600, and a notification unit 4700 functionally.

[0080] (A-1-4-1. Spatial data acquisition unit 4100) The spatial data acquisition unit 4100 is a functional unit that acquires data of a target area from the measuring instrument 110 or an appropriate external device. The spatial data acquisition unit 4100 mainly has a satellite measurement data acquisition unit 4110, an aircraft measurement data acquisition unit 4120, and a geographic data acquisition unit 4130 functionally.

[0081] The satellite measurement data acquisition unit 4110 acquires the measurement data obtained by the artificial satellite 111. The aircraft measurement data acquisition unit 4120 is a functional unit that acquires the measurement data obtained by the aircraft 112. These measurement data are used when the integrated image generation unit 4200 generates an initial wide-area integrated image and when the integrated image update unit 440 updates a part of the wide-area integrated image. Also, the satellite measurement data acquisition unit 4110 and the aircraft measurement data acquisition unit 4120 may acquire the related information of the measurement data together. The related information is, for example, measurement position coordinates, measurement time, resolution, measurement method, measurement altitude, and the like. The acquired measurement data and related information are respectively recorded in the measurement area recording unit 4510, measurement time recording unit 4520, measurement data resolution recording unit 4530, measurement method recording unit 4540, and measurement altitude recording unit 4550 of the spatial data information recording unit 4500.

[0082] The geographical data acquisition unit 4130 is a functional unit that acquires the geographical data of the target area F. Different from the measurement data, the geographical data is data that is generally associated with the target area F, and includes, for example, map information or polygon data including three-dimensional land and building data. Note that the geographical data may be changed according to changes in the land shape and the construction and demolition of buildings, and may be appropriately updated at a timing independent of the acquisition of the measurement data. The acquired geographical data is stored in the geographical data recording unit 4560 of the spatial data information recording unit 4500.

[0083] (A-1-4-2. Integrated Image Generation Unit 4200) The integrated image generation unit 4200 is a functional unit that integrates a plurality of partial images to generate a wide-area integrated image. The integrated image generation unit 4200 mainly functionally includes an integrated image selection unit 4210, an image suitability determination unit 4220, a complementary image selection unit 4230, and an image integration unit 4240.

[0084] The integrated image selection unit 4210 is a functional unit that selects an image to be integrated according to the measurement area, measurement time, or resolution of each acquired data.

[0085] The plurality of partial images to be integrated have overlapping regions that image the same region in adjacent partial images. Here, the integrated image selection unit 4210 compares the quality of adjacent partial images and adopts the partial image with better quality as the image in the overlapping region. According to such a configuration, a high-quality wide-area integrated image can be generated. The quality of the image may be determined, for example, by the state of sunlight reflection (such as the presence or absence of whiteout), brightness, luminance, or radar intensity. Also, an image with less defect and showing a larger area of the ground surface may be determined to be of high quality.

[0086] The image suitability determination unit 4220 is a functional unit that determines the suitability for adoption into the wide-area integrated image for each of the plurality of partial images. For example, the image suitability determination unit 4220 determines whether there are any defects or measurement omissions in each selected image. Images with defects or measurement omissions are, for example, images in which surface reflection, occlusion, low resolution, out-of-focus, blur, fog, defocus, etc. have occurred. Also, an image in which the area of the ground surface cannot be sufficiently photographed due to clouds or foreign objects may be regarded as a defective image. The image suitability determination unit 4220 determines the area of the ground surface reflected and the blur of the image, etc., by an appropriate configuration. The determination of clouds may be made by the color tone or the difference between normal images. Also, the determination of clouds may be made by an appropriate learning algorithm. The image suitability determination unit 4220 may determine defects or measurement omissions by analyzing the histogram of the measured image. Also, the image suitability determination unit 4220 may identify partial regions where adoptable partial images cannot be obtained.

[0087] The complementary image selection unit 4230 is a functional unit that selects a complementary image for a partial area where the image suitability determination unit 4220 fails to extract a suitable partial image. For example, in an area of the target area F where no adoptable partial image has been obtained, the complementary image selection unit 4230 uses, as the complementary image, a partial image taken at a shooting altitude different from the measurement data whose suitability has been determined by the integrated image selection unit 4210. Images taken at different altitudes may be taken by the same aircraft 112 or by different types of measuring instruments (such as the artificial satellite 111 and the aircraft 112). Further, when the measurement data whose suitability has been determined by the integrated image selection unit 4210 is obtained by the artificial satellite 111, the complementary image selection unit 4230 may use, as the complementary image, measurement data obtained by an artificial satellite 111 orbiting a different orbit from that of the artificial satellite 111. This configuration is particularly suitable for removing an image in which clouds are reflected.

[0088] In particular, the integrated image selection unit 4210 may use, as the complementary image, measurement data measured by a measuring instrument 110 of a different type from the measurement data whose suitability has been determined. For example, the integrated image selection unit 4210 and the image suitability determination unit 4220 use a partial image obtained by the artificial satellite 111, and the complementary image selection unit 4230 adopts an image obtained by the aircraft 112 for an area where no adoptable partial image has been obtained by the artificial satellite 111. According to such a configuration, by adopting data measured by different types of measuring instruments 110, a wide-area integrated image can be efficiently generated.

[0089] In particular, due to its high altitude, the artificial satellite 111 can perform measurements over a wide area at one time. However, since it can only perform measurements in its orbiting path, it is difficult to measure a desired area. In contrast, the aircraft 112 can measure a desired area under control, but due to its lower altitude than the artificial satellite 111, the range that can be measured at one time is narrower, and it takes time for wide-area photography. In this regard, according to the configuration of the present application, since the images acquired by both the artificial satellite 111 and the aircraft 112 can be utilized and integrated, a wide-area integrated image with a wide range in which images taken by multiple types of imaging means are mixed can be quickly generated. Also, since the artificial satellite 111 takes pictures from a high altitude, it obtains relatively low-resolution images, while the aircraft 112 takes pictures from a low altitude, so it can obtain relatively high-resolution images. Therefore, by integrating the images taken by multiple types of imaging means, a wide-area integrated image with an arbitrary area having high resolution can be quickly generated.

[0090] In addition, the complementary image selection unit 4230 may adopt, as a complementary image, a past partial image acquired in the past from the partial image or geographical data in an area of the target area F where an adoptable partial image cannot be acquired. According to the configuration using the past partial image, even when there is no recently acquired image, a wide-area integrated image in which a certain degree of the situation can be grasped can be generated. Also, according to the configuration adopting geographical data, a wide-area integrated image can be generated even if there is an area where no partial image exists.

[0091] The image integration unit 4240 is a functional unit that generates a wide-area integrated image. The image integration unit 4240 performs orthographic transformation on the partial image selected by the integrated image selection unit 4210 or the complementary image selected by the complementary image selection unit 4230 by orthorectification. Note that the orthographic transformation process may be performed on all pixels of the image or on some pixels. The image integration unit 4240 may specify the pixels on which the orthographic transformation process is to be performed according to the attitude, particularly the accuracy, of the measuring instrument 110 or the sensor units 1111 and 1121 when the image is measured.

[0092] The image integration unit 4240 inserts the partial image selected by the integrated image selection unit 4210 or the image selected by the complementary image selection unit 4230 into the target partial area and integrates it into the wide-area integrated image. Also, in the overlap area, the image integration unit 4240 inserts the image selected by the integrated image selection unit 4210. By the image integration unit 4240 performing the above-described image insertion process for each of the plurality of partial areas constituting the target area F, a wide-area integrated image of the target area F is generated. Note that, in addition to the mode of directly inserting the partial image or the complementary image, the image integration unit 4240 may be configured to make the wide-area integrated image visible by associating a link indicating the storage location of the image with the partial area.

[0093] According to such a configuration, since images captured by different imaging means can be mixed and integrated, a wide-area integrated image over a wide range can be generated quickly. Such a spatial data processing system 4000 is also useful, for example, for grasping the situation of an area where a disaster or the like has occurred.

[0094] (A-1-4-3. Update Plan Generation Unit 4300) The update plan generation unit 4300 is a functional unit that generates an update plan for performing an update of a so-called wide-area integrated image, in which a part of the partial images constituting the wide-area integrated image is changed to a newly acquired image. The update plan generation unit 4300 mainly includes, functionally, an update request reception unit 4310, a measurement request unit 4320, a measurement plan acquisition unit 4330, and an update plan creation unit 4340.

[0095] The update request reception unit 4310 is a functional unit that receives an input of a priority update area for preferentially updating an image in the wide-area integrated image. The update request reception unit 4310 may receive a request for the resolution of the image to be acquired in the priority update area. The input of the resolution is not limited to the input of an absolute value, and may be received as a relative value from the current partial image. This may be, for example, the ratio or difference between the desired resolution and the resolution of the current partial image, or a qualitative request such as "a higher-resolution image" may be received. For qualitative requests, the update request reception unit 4310 calculates the value of the desired resolution by applying the resolution of the current image to a preset mathematical formula or the like.

[0096] The update request reception unit 4310 may receive inputs from a plurality of users. The users include, for example, both a countermeasure headquarters that designates a relatively wide area and an individual countermeasure team that designates a relatively local area. Also, when the wide-area integrated image is published on a website or the like, the update request reception unit 4310 may receive an update request from a general citizen who views the website. The priority update area may be a partial area that can be constituted by one partial image, or may be an area having an area constituted by a plurality of partial images. The update request reception unit 4310 receives, for example, an input of at least any one of an acquisition means, a shooting altitude, or a resolution of a newly acquired image to be used for updating the priority update area. The input of the shooting altitude is not limited to the input of an absolute value, and may be received as a relative value from the shooting altitude of the current partial image. This may be, for example, "100 m lower altitude than the current partial image", "half the altitude of the current partial image", etc., or may be received as a qualitative request such as "lower altitude".

[0097] Further, the update request reception unit 4310 may be configured to receive a confirmation to issue a measurement request for the received update request. For example, when there are multiple users, a measurement request command may be generated by having another user confirm an update request input by one user. Such a configuration may be in a form where, when an individual countermeasure team or a member of the general public inputs an update request, the countermeasure headquarters confirms it. According to this configuration, it helps in the cooperation of multiple users.

[0098] The measurement request unit 4320 is a functional unit that generates a measurement request command for the priority update area. The measurement request command includes information such as the position coordinates of the partial area that requires measurement, the resolution of the update image to be acquired, the measurement method, or the measurement altitude. The measurement request unit 4320 transmits the measurement request command to the aerial data sensing system 1000. Based on this measurement request command, the operation management unit 1100 of the aerial data sensing system 1000 selects the type and individual of the measuring aircraft 110 for measurement, and makes an operation decision and gives instructions. Also, the aircraft operation operate unit 1300 determines the mission of the measuring aircraft 110 for measurement. Through a series of processes of the operation management unit 1100 and the aircraft operation operate unit 1300, a measurement plan for the priority update area is determined.

[0099] The measurement plan acquisition unit 4330 is a functional unit that acquires a measurement plan (an example of an image acquisition plan) from the aerial data sensing system 1000. The measurement plan may include the scheduled acquisition time when the measuring aircraft 110 acquires a partial image of the priority update area.

[0100] The update plan creation unit 4340 is a functional unit that creates an update plan according to the acquired measurement plan. The update plan creation unit 4340 refers to the scheduled time for each area where the measurement device 110 periodically patrols to acquire a newly acquired image, and calculates the scheduled update time when a partial area of the wide-area integrated image is updated for each partial area. Also, the update plan creation unit 4340 calculates the scheduled update time when the wide-area integrated image is updated based on the scheduled acquisition time when the measurement device 110 acquires a newly acquired image of the priority update area. The update plan creation unit 4340 may calculate the scheduled update time by taking into account the time when the spatial data processing system 4000 acquires the newly acquired image, the time for determining the suitability of the newly acquired image, and the time for updating the wide-area integrated image using the newly acquired image at the scheduled acquisition time of the newly acquired image. The update plan creation unit 4340 calculates the scheduled update time for each partial area.

[0101] (A-1-4-4. Integrated Image Update Unit 4400) The integrated image update unit 4400 is a functional unit that updates the image to be displayed in a predetermined partial area of the wide-area integrated image with the newly acquired image. The integrated image update unit 4400 mainly has a function of an update image suitability determination unit 4410 and an image update unit 4420.

[0102] The update image suitability determination unit 4410 is a functional unit that determines whether a newly acquired image is suitable as a partial image to be used for update, that is, as an update image. The update image suitability determination unit 4410 determines the suitability of the update image according to whether the measurement target area of the update image is within the target area F, the measurement time is the latest, for example, after a predetermined time, the resolution is higher than a predetermined value, etc., based on the information of the spatial data information recording unit 4500.

[0103] When the image update unit 4420 newly acquires a new acquired image of a partial area after generating the wide-area integrated image, among the plurality of partial images constituting the wide-area integrated image, the image corresponding to the partial area included in the newly acquired image (hereinafter, also referred to as "corresponding image") is changed to the newly acquired image. According to such a configuration, even when the image of the target area F is partially acquired, the wide-area integrated image can be partially updated, so that the situation of the target area F that changes moment by moment can be quickly reflected in the wide-area integrated image.

[0104] The image update unit 4420 partially updates the wide-area integrated image with a newly acquired image acquired by the measuring instrument 110 that is highly different from the corresponding image. According to such a configuration, for example, a partial area composed of a partial image acquired by the artificial satellite 111 can be updated with a partial image acquired by the aircraft 112.

[0105] The priority of the measurement conditions for updating the wide-area integrated image is set for the image update unit 4420. That is, for example, the image update unit 4420 may update the image with the acquisition time being newer than the corresponding image as the first priority, or may update the image with the resolution being higher than the corresponding image as the first priority. The measurement conditions to be prioritized may be set in advance or may accept the user's selection.

[0106] The image update unit 4420 compares the acquisition time of the newly acquired image with the acquisition time of the corresponding image. When the acquisition time of the newly acquired image is after the acquisition time of the corresponding image by a predetermined time or more, or when the image is not integrated into the partial area, the image update unit 4420 integrates the newly acquired image into the wide-area integrated image. According to such a configuration, a new partial image can be reflected in the wide-area integrated image. Also, for a partial area where there is no partial image and geographical data is displayed, it can be updated according to the acquisition of the partial image, and a sequential, detailed, and up-to-date wide-area integrated image can be generated.

[0107] The image update unit 4420 may compare the resolution of the newly acquired image with the resolution of the corresponding image. When the acquisition time of the newly acquired image is after a predetermined time or more from the acquisition time of the corresponding image and the resolution of the newly acquired image is equal to or higher than the resolution of the corresponding image, the image update unit 4420 may integrate the newly acquired image into the wide-area integrated image. According to such a configuration, the wide-area integrated image can be made to have a high resolution according to the acquired measurement data.

[0108] Also, when the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount or more, the image update unit 4420 may integrate the newly acquired image into the wide-area integrated image.

[0109] (A-1-4-5. Spatial Data Information Recording Unit 4500) The spatial data information recording unit 4500 is a functional unit that records the acquired spatial data. The spatial data includes, in addition to the measurement data acquired by the measuring instrument 110, geographical data acquired from an appropriate database. The spatial data information recording unit 4500 accumulates the measurement data that has not been adopted in the wide-area integrated image, as well as the measurement data that has been adopted in the past, together with the measurement data adopted in the wide-area integrated image. The spatial data information recording unit 4500 functionally has mainly a measurement area recording unit 4510, a measurement time recording unit 4520, a measurement data resolution recording unit 4530, a measurement method recording unit 4540, a measurement altitude recording unit 4550, and a geographical data recording unit 4560.

[0110] The measurement area recording unit 4510 records the position coordinates where the measurement data was measured. The measurement time recording unit 4520 records the measurement time when the measurement data was measured. The measurement data resolution recording unit 4530 records the resolution of the measurement data. The measurement method recording unit 4540 records the type of the measuring instrument 110 that measured the measurement data. Also, the measurement method recording unit 4540 may record the identification information of the measuring instrument 110. The measurement altitude recording unit 4550 records the measurement altitude when the measurement data was measured. The geographical data recording unit 4560 records the geographical data acquired from the database together with the coordinates of the corresponding partial area.

[0111] (A-1-4-6. Display control unit 4600) The display control unit 4600 is a functional unit that controls the display on an appropriate display device for the user to view. The display device is, for example, a display connected by the network NW, and may be connected to a personal computer, or may be a mobile terminal such as a smartphone or a tablet terminal. The display control unit 4600 mainly functionally includes a display condition reception unit 4610, an integrated image display unit 4620, an update plan display unit 4630, and an update image display unit 4640.

[0112] The display condition reception unit 4610 receives from the user the input of the display conditions for the partial image to be displayed. The display conditions include, for example, at least any one of the acquisition time, acquisition means, acquisition altitude, or resolution of the partial image to be displayed.

[0113] The integrated image display unit 4620 is a functional unit that displays a wide-area integrated image on the display device. Also, when the display condition reception unit 4610 receives the display conditions, the integrated image display unit 4620 displays on the display device the partial image that matches the received display conditions. According to such a configuration, a wide-area integrated image conforming to the desired display conditions of the user can be displayed on the display device.

[0114] The update plan display unit 4630 displays the update plan of the wide-area integrated image generated by the update plan creation unit 4340 on the display device. Further, the update plan display unit 4630 may display a measurement request command or a measurement plan. According to such a configuration, the update schedule of the wide-area integrated image is clear to the user, and the user can make a prediction for grasping the area of interest.

[0115] The updated image display unit 4640 displays the updated wide-area integrated image on the display device. Further, the updated image display unit 4640 may display the measurement conditions of the displayed partial images for each partial area on the display device. The measurement conditions include, for example, at least any one of the update time, the acquisition means of the partial image, or the shooting altitude of the partial image. For example, when the user selects a partial area of interest by tapping or rolling over it, the measurement conditions of the area are displayed.

[0116] (A-1-4-7. Notification unit 4700) The notification unit 4700 is a functional unit that gives a predetermined notification to the user via a display device or the like. This notification may be, in addition to the display on the screen, sound, vibration, or the like. For example, when the update of the priority update area is completed, the notification unit 4700 gives a notification indicating that the update has been completed. This notification may be, for example, pop-up displayed on the wide-area integrated image, or an icon may be displayed separately from the wide-area integrated image. The update completion notification for the priority update area may be given only to the terminal in which the priority update area is input by the user. Further, the notification unit 4700 may display, on the display device, the update scheduled time and the scheduled measurement conditions for reflecting the newly acquired images of the partial areas in the wide-area integrated image for each partial area based on the measurement plan of the target area. The scheduled measurement conditions are, for example, at least any one of the acquisition means of the newly acquired image or the shooting altitude of the newly acquired image.

[0117] In addition, when the work of the image update process involves multiple responsible persons, the notification unit 4700 may estimate the work status of the worker or obtain the work status input by the worker, and notify the user's display device of the status of the progress of the work for the work sharing of each preset worker. The status may be, for example, "not shared", "before starting work", "working", "work completed", etc. Further, according to the work status or status, the estimated image update scheduled time may be estimated and displayed on the display device.

[0118] Note that part or all of each component of the spatial data processing system 4000 may be implemented in the aerial data sensing system 1000 or may be implemented in the measuring instrument 110. When each component is implemented in the aircraft 112, the creation or update of the wide-area integrated image may be performed in parallel with the acquisition of the measurement data.

[0119] (A-1-5. Flowchart) (A-1-5-1. Outline of the process) Using FIG. 8, an example of the flow of a series of processes executed when acquiring spatial data and updating the wide-area integrated image will be described. First, the spatial data acquisition unit 4100 of the spatial data processing system 4000 acquires the measurement data measured by the measuring instrument 110 (step S101). Next, the integrated image generation unit 4200 generates a wide-area integrated image (step S102). The integrated image display unit 4620 of the display control unit 4600 displays the generated wide-area integrated image on the display device (step S103). Next, the update plan generation unit 4300 generates an update plan (step S104). Next, the spatial data acquisition unit 4100 acquires additional spatial data based on the update plan (step S105).

[0120] Next, the integrated image update unit 4400 updates the wide-area integrated image (step S106). Next, the updated wide-area integrated image is displayed on the display device by the updated image display unit 4640 of the display control unit 4600. Further, the notification unit 4650 notifies the display device that the update has been made (step S107).

[0121] (A-1-5-2. Creation Process of Wide-Area Integrated Image) Next, with reference to FIG. 9, the process of creating a wide-area integrated image will be described. This process is an example of a subroutine in step S102 of FIG. 8.

[0122] First, the spatial data processing system 4000 sets a target area F for creating an integrated image (step S201). Next, the integrated image selection unit 4210 selects an image corresponding to the target area F (step S202). Next, the image suitability determination unit 4220 excludes an image with a defect from the selected images (step S203).

[0123] Next, it is determined whether there are multiple images corresponding to the same partial area (step S204). In this case, for example, when the partial areas overlap by a predetermined ratio or more, they may be presumed to correspond to the same partial area. If there are multiple such images (Y in step S204), the process proceeds to step S205. If there is one or no such image (N in step S204), the process proceeds to step S208.

[0124] In step S205, the acquisition times of multiple partial images corresponding to the same partial area are compared. If the acquisition times of the multiple partial images are the same (N in step S205), the partial image with the higher resolution is selected (step S206). If the acquisition times of the multiple partial images are different (Y in step S205), the image with the latest measurement time is selected (step S207). Then the process proceeds to step S210.

[0125] In step S208, it is determined whether there is a partial area without an image, that is, a missing area, within the target area F. If there is a missing area (Y in step S208), a previously acquired partial image corresponding to the missing area or geographical data corresponding to the area is selected (step S209). If there is no missing area in step S208 (N in step S208), the process proceeds to step S210. Perform the processes of step S204 to step S209 in all sub - regions constituting the target area F, and determine one by one the partial images or geographical data to be adopted.

[0126] In step S210, orthorectify the partial image to be adopted. Then, generate a wide - area integrated image by integrating the partial image or geographical data to be adopted (step S211), and end the process.

[0127] (A - 1 - 5 - 3. Generation process of update plan) Using FIG. 10, the generation process of the update plan will be described. Note that this process is an example of a sub - routine of step S104 in FIG. 8.

[0128] First, the update request reception unit 4310 obtains an update request such as an area that the user desires to update preferentially from the user (step S301). In this process, the update request reception unit 4310 may receive a specification of the resolution, measurement method, or measurement altitude of the image obtained by the update. Next, the measurement request unit 4320 sends a measurement request to the aerial data sensing system 1000 (step S302). Next, the measurement plan acquisition unit 4330 acquires the measurement plan generated by the aerial data sensing system 1000 (step S303). Next, the update plan creation unit 4340 creates an update plan according to the measurement plan acquired in step S303 (step S304). The update plan display unit 4630 of the display control unit 4600 displays the update plan on the display device (step S305).

[0129] (A - 1 - 5 - 4. Update process of wide - area integrated image) Using FIG. 11, an example of the processing flow for updating the wide - area integrated image will be described. Note that this process is an example of a sub - routine of step S106 in FIG. 8.

[0130] First, the update image suitability determination unit 4410 selects an image that satisfies the conditions of the update request (step S401). Next, the update image suitability determination unit 4410 excludes an image with defects from the selected images (step S402).

[0131] Next, the updated image suitability determination unit 4410 determines whether the acquisition time of the additionally acquired additional image is newer than the acquisition time of the pre-update image in the same area (step S403). If the update time of the additional image is newer than the update time of the pre-update image in the same area (Y in step S403), the additional image is selected as the updated image (step S404). If the acquisition time of the additional image is not newer than the acquisition time of the pre-update image in the same area, that is, if it is an image acquired at the same time or before the pre-update image (N in step S403), the image with the higher resolution is determined as the image to be used for the integrated image (step S405).

[0132] Note that in the example of this figure, the process of selecting an image with priority given to the acquisition time has been described. Instead of this, a process of selecting an image with priority given to the resolution may be used. That is, first, the resolutions of the additional image and the pre-update image are compared, and the image with the higher resolution is adopted for the integrated image. When the resolutions of the additional image and the pre-update image are the same, the image with the newer acquisition time may be adopted. Also, it may be possible to switch based on the user's designation which of the acquisition time and the resolution is given priority in determining the image.

[0133] Next, it is determined whether there is a missing area where no image exists within the target area F (step S406). If there is a missing area (Y in step S406), the complementary image selection unit 4230 selects an image corresponding to the missing area from the images measured in the past, or adopts the geographical data at the corresponding position (step S407), and proceeds to step S408. If there is no missing area in step S406 (N in step S406), it proceeds to step S408 without going through step S407.

[0134] Next, orthorectification of the image adopted for image update is performed (step S408). Next, the image update unit 4420 updates the image using the orthorectified image (step S409).

[0135] (A-1-6. Screen Example) FIG. 12 is a diagram showing an example of a screen G10 for inputting information on a priority update area received by an update request reception unit 4310. A wide-area integrated image G11 is displayed on this screen G10. Also, on the wide-area integrated image G11, boundaries of a plurality of partial areas constituting the wide-area integrated image G11 are displayed as lines. In the present embodiment, since the partial areas are rectangular, the lines representing the boundaries of the partial areas are formed in a grid pattern, but the shape of the partial areas is not limited to a rectangle. Also, partial images of different sizes may be mixed. The lines representing the boundaries are appropriately represented according to the shape of the partial areas. Also, when geographical data is adopted as a complementary image and the geographical data is displayed in a part of the wide-area integrated image, the display control unit 4600 may display the partial area where the geographical data is displayed in a display mode different from that of the partial area where the image is displayed. Also, depending on the credibility of the data, it may be configured to be displayed in a plurality of types of display modes. For example, the display mode may be varied according to the resolution, or may be varied according to the acquisition time.

[0136] The update request reception unit 4310 receives an input of a priority update area for each partial area. More specifically, the user can input a request for updating the relevant partial area by selecting one or a plurality of partial areas on the screen G10 by tapping, clicking, or the like. Also, the user may input a request for updating the surrounded area by inputting an operation of partially surrounding the wide-area integrated image G11. In the figure, the priority update area is displayed with a semi-transparent shading so that the current partial image and the fact that it is the priority update area can be visually recognized together.

[0137] When a predetermined operation is performed on the input priority update area, it may be possible to input measurement conditions for each priority update area. The predetermined operation may be, for example, a long press, a right click, or a predetermined key input. In the figure, a screen after the input of the predetermined operation is shown, and a rectangular pop-up screen G13 is displayed in the vicinity of the priority update area, and it is possible to input an acquisition method (measurement method), an acquisition altitude (measurement altitude), and a resolution.

[0138] FIG. 13 is a diagram showing an example of a display screen G20 of the updated wide-area integrated image. On the screen G20, a wide-area integrated image G21 is displayed. Also, on the screen G20, a semi-transparent shading is displayed on one or more updated partial regions G22. With such a configuration, the updated partial regions are clear. Note that this shading may disappear according to a predetermined operation or the passage of a certain period of time. Also, when a predetermined operation is performed on the updated partial region G22, a pop-up screen G23 for displaying measurement conditions and the like of the partial region G22 is displayed. For example, on the pop-up screen G23, as measurement conditions, the update date and time, the acquisition method (measurement method), and the acquisition altitude (measurement altitude) are displayed. Also, the next scheduled update date and time may be displayed on the pop-up screen G23. Also, as a status indicating the progress of the update, displays such as "not measured", "measured", and "image processing in progress" may be made.

[0139] FIG. 14 is a diagram showing an example of a screen G30 that is displayed when the display condition reception unit 4610 receives the display conditions of a partial region from the user. As the display conditions, when the display condition reception unit 4610 receives at least any one of the conditions of the image acquisition time, the image acquisition means, the image acquisition altitude, and the image resolution, an image G32 that matches the condition is selectively displayed. The set display conditions may be displayed in a predetermined region G33. Also, when there is no image that matches the display conditions, a blank region G34 may be displayed. Note that geographical data may be displayed in the blank region G34.

[0140] In the figure, an example is shown in which only partial images whose update date and time are after 03 / 09 / 12:30 are extracted and displayed. In this example, a situation for quickly grasping the river flooding situation is taken as an example. As a result of the aircraft 112 frequently flying over the area around the river and acquiring images, the images of the area around the river are frequently updated. Therefore, in the figure, the area around the river is the image that matches the "update date and time: after 03 / 09 / 12:30" specified as the display condition.

[0141] [A-2. Effects of the Present Embodiment] According to this embodiment, the captured image of the target area can be updated quickly.

[0142] Note that the present invention is not limited to the above embodiment, and various configurations can of course be adopted based on the description in this specification.

[0143] A series of processes described in relation to the above embodiment may be realized using any of software, hardware, and a combination of software and hardware. It is possible to create a computer program for realizing each function of the image update system 1 according to this embodiment and install it on a PC or the like. Further, a computer-readable recording medium storing such a computer program can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed without using a recording medium, for example, via a communication network NW.

[0144] Regarding the flowchart used in the above embodiment, it is not necessarily executed in the order shown in the figure. Some processing steps may be executed in parallel. Further, additional processing steps may be adopted, and some processing steps may be omitted.

Explanation of Reference Numerals

[0145] 1 Aviation Data Sensing System 1000 Aircraft Control System 1100 Operation Management Unit 1200 Airspace Monitoring and Control Unit 1300 Aircraft Operation Control Unit 1400 Acquired Data Management Unit 1500 Communication Infrastructure Management Unit 110 Measuring Instrument 111 Artificial Satellite 112 Aircraft 4000 Space Data Processing System (Image Update System) 4100 Space Data Acquisition Unit 4200 Integrated Image Generation Unit 4300 Update Plan Generation Unit 4400 Integrated Image Update Unit 4500 Spatial Data Information Recording Unit 4600 Display Control Unit 4700 Notification Unit F Target Area

Claims

1. A method for updating a wide-area integrated image generated by integrating a plurality of different partial images obtained by photographing a partial area obtained by subdividing a target area from above, comprising: a computer an integrated image generation step of integrating the plurality of partial images to generate the wide-area integrated image; an update step of, when a new acquired image of the partial area is newly acquired after the generation of the wide-area integrated image, updating a corresponding image corresponding to the partial area included in the new acquired image with the new acquired image among the plurality of partial images constituting the wide-area integrated image; An image update method that executes the above.

2. The update step includes an acquisition time comparison step of comparing the acquisition time of the new acquired image with the acquisition time of the corresponding image, In the update step, when the acquisition time of the new acquired image is after the acquisition time of the corresponding image by a predetermined time or more, the new acquired image is integrated into the wide-area integrated image. The image update method according to claim 1.

3. The update step includes a corresponding image determination step of determining whether the corresponding image corresponding to the new acquired image is integrated into the wide-area integrated image, In the update step, when the corresponding image is not integrated into the wide-area integrated image, the new acquired image is integrated into the wide-area integrated image. The image update method according to claim 1.

4. The update step includes an acquisition time comparison step of comparing the acquisition time of the new acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the new acquired image with the resolution of the corresponding image, In the update step, when the acquisition time of the new acquired image is after the acquisition time of the corresponding image by a predetermined time or more and the resolution of the new acquired image is equal to or higher than the resolution of the corresponding image, the new acquired image is integrated into the wide-area integrated image. The image update method according to claim 1.

5. The update step includes an acquisition time comparison step of comparing the acquisition time of the newly acquired image with the acquisition time of the corresponding image, and a resolution comparison step of comparing the resolution of the newly acquired image with the resolution of the corresponding image. In the update step, when the acquisition time of the newly acquired image is within a predetermined time range of the acquisition time of the corresponding image and the resolution of the newly acquired image is higher than the resolution of the corresponding image by a predetermined amount or more, the newly acquired image is integrated into the wide-area integrated image. The image update method according to claim 1.

6. In the update step, the wide-area integrated image is partially updated with the newly acquired image acquired by a photographing means having a significantly different altitude from the corresponding image. The image update method according to claim 1.

7. In the integrated image generation step, an image suitability determination step of determining the suitability for adoption into the wide-area integrated image among a plurality of partial images acquired at a first photographing altitude; a complementary processing step of extracting an area in the target area where the adoptable partial image has not been acquired, and using, as a complementary image, a partial image photographed at a second photographing altitude different from the first photographing altitude in the area; an image integration step of integrating the complementary image into the wide-area integrated image; and The image update method according to claim 1.

8. The integrated image generation step includes an image suitability determination step of determining the suitability for adoption into the wide-area integrated image among the plurality of acquired partial images; a complementary processing step of extracting an area in the target area where the adoptable partial image has not been acquired, and using, as a complementary image, a partial image acquired earlier than the partial image or geographical data in the area; An image integration step of integrating the complementary image into the wide-area integrated image, including The image update method according to claim 1.

9. In the integrated image generation step, the plurality of partial images to be integrated have an overlapping area that captures the same area in the adjacent partial images, In the integrated image generation step, the quality of the adjacent partial images is compared, and the partial image with better quality is adopted as the image in the overlapping area. The image update method according to claim 1.

10. Further executing an update request receiving step of receiving an input of a priority update area in which the image is preferentially updated in the wide-area integrated image. The image update method according to claim 1.

11. In the update request receiving step, an input of at least one of an acquisition means, a shooting altitude, or a resolution of a newly acquired image used for updating the priority update area is received. The image update method according to claim 10.

12. A measurement request step of generating a measurement command for the priority update area, or a measurement plan generation step of generating a measurement plan, A display control step of displaying the measurement command or the measurement plan on a display device, Further executing The image update method according to claim 10.

13. When the update of the priority update area is completed, further executing a notification step of notifying that the update is completed. The image update method according to claim 10.

14. Based on the future image acquisition plan of the target area, for each of the partial areas, a display control step of displaying on a display device at least one of the update scheduled time for reflecting the newly acquired image of the partial area in the wide-area integrated image, the acquisition means of the newly acquired image, or the shooting altitude of the newly acquired image, is further executed. The image update method according to claim 1.

15. For each of the partial areas, a display control step of displaying on a display device at least one of the update time of the displayed partial image, the acquisition means of the partial image, or the shooting altitude of the partial image, is further executed. The image update method according to claim 1.

16. A display condition reception step of receiving an input of the display condition of the partial image to be displayed from a user is further executed. The display condition includes at least one of the acquisition time, acquisition means, acquisition altitude, or resolution of the partial image to be displayed. A display control step of displaying on a display device the partial image that matches the display condition received in the display condition reception step is further executed. The image update method according to claim 1.

17. An image update system for updating a wide-area integrated image generated by integrating a plurality of different partial images taken from above of partial areas obtained by subdividing a target area, An integrated image generation unit that integrates the plurality of partial images to generate the wide-area integrated image; After the generation of the wide-area integrated image, when a new acquired image of the partial area is newly acquired, among the plurality of partial images constituting the wide-area integrated image, the partial image taken of the partial area included in the new acquired image is extracted as a corresponding image, and in the wide-area integrated image, an image update unit that updates the corresponding image with the new acquired image. An image update system comprising the above.

18. An image update program for a wide-area integrated image generated by integrating a plurality of different partial images obtained by photographing a partial area obtained by subdividing a target area from above, to cause a computer to, execute an integrated image generation instruction for generating the wide-area integrated image by integrating the plurality of partial images, when a newly acquired image of the partial area is newly acquired after the generation of the wide-area integrated image, extract, as a corresponding image, a partial image that photographed the partial area included in the newly acquired image from among the plurality of partial images constituting the wide-area integrated image; a corresponding image extraction instruction; in the wide-area integrated image, execute an update instruction to update the corresponding image with the newly acquired image; An image update program that causes the above to be executed.

Images