Control system, control method, and control program

Abstract

To provide a system that can further efficiently utilize acquisition data acquired by an aircraft.SOLUTION: A control system of the present invention comprises: a data acquisition unit that acquires determination target data such as sensing data of a target area acquired using a sensor mounted on a flying object; a data propriety determination unit that performs a propriety determination on the determination target data acquired by the data acquisition unit; and an additional information generation unit that generates additional information, including information about a content of the propriety determination when the data is determined inappropriate by the data propriety determination unit. A ground position and the like of the inappropriate data determined to be inappropriate, or the target area related to the inappropriate determination, is recorded in a recording unit or displayed on a display unit, in association with the additional information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a control system, a control method, and a control program.

Background Art

[0002] In recent years, the practical application of an airborne sensing system that uses an aircraft to acquire images and spatial data of a target area on the ground from the air has been under consideration. When performing airborne sensing, it is necessary to perform sensing according to preset flight conditions and measurement conditions so that the airborne sensing data can be used for a desired purpose. However, due to the influence of weather conditions, equipment conditions, etc., it may be impossible to temporarily acquire airborne sensing data that satisfies the above conditions during airborne sensing. Airborne sensing data generally has a huge amount of data, and it has taken a great deal of time to find out the airborne sensing data that does not satisfy the above conditions from the huge amount of airborne sensing data. Therefore, there has been a demand for a function to determine data that does not satisfy predetermined conditions from airborne sensing data.

[0003] Patent Document 1 discloses a technique in which when an unmanned aircraft performing aerial photography is at a predetermined distance from the planned flight route, a flag is incorporated into the data of the aerial photography video, and it is notified that there is a need to edit the aerial photography video at the time when the flag is incorporated.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 discloses determining whether or not there is a photographing failure according to the actual flight position of an aircraft and incorporating information on the photographing failure into aerial photography video data. However, only a binary determination was made, i.e., not using the photographing data if there was a photographing failure and using the photographing data if there was no photographing failure. Therefore, even if the data can be used without problems depending on the use purpose of the data and the content of data processing, the data determined to be a photographing failure is not used, so it was not possible to efficiently use the aviation sensing data and quickly provide the aviation sensing data or the processed data to the user.

[0006] Therefore, an object of the present invention is to provide a system that can more efficiently use the acquisition data obtained by an aircraft.

Means for Solving the Problems

[0007] According to the present invention, there is provided a control system including: a data acquisition unit that acquires at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on an aircraft, in-process data generated during the process of data processing of the sensing data, and post-processed data generated by data processing of the sensing data; a data suitability determination unit that performs a suitability determination on the determination target data acquired by the data acquisition unit; and an accompanying information generation unit that generates accompanying information including information regarding the content of the suitability determination when it is determined to be inappropriate by the data suitability determination unit, and associating the inappropriate data determined to be inappropriate, or the ground position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target path of the aircraft with the accompanying information, and recording it in a recording unit or displaying it on a display unit.

Effects of the Invention

[0008] According to the present invention, by more efficiently using the acquisition data obtained by an aircraft, data can be quickly provided to the user.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Embodiments for Carrying Out the Invention

[0010] The content of the embodiments of the present invention will be listed and described. The present invention has the following configuration. [Item 1] Sensing data including an image or point cloud information of a target area acquired using sensors mounted on an aircraft, processing data generated during the data processing of the sensing data, and a data acquisition unit that acquires at least any of the determination target data of the processed data generated by the data processing of the sensing data, A data suitability determination unit that determines the suitability of the determination target data acquired by the data acquisition unit, An accompanying information generation unit that generates accompanying information including information regarding the content of the suitability determination when it is determined to be inappropriate by the data suitability determination unit, and A control system that associates inappropriate data determined to be inappropriate, or the ground position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target path of the aircraft with the accompanying information, and records it in a recording unit or displays it on a display unit. [Item 2] In the control system according to Item 1, The accompanying information includes information regarding the reason determined to be inappropriate by the suitability determination, a control system. [Item 3] In the control system according to Item 1 or 2, On at least any of a map format, a three-dimensional model, and a three-dimensional space screen, the accompanying information is displayed in association with the inappropriate data, or the ground position of the target position related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target path of the aircraft. [Item 4] In the control system according to any one of Items 1 to 3, The accompanying information includes information on the degree to which the inappropriate data exceeds the reference value of the suitability determination when it is determined to be inappropriate by the suitability determination, a control system. [Item 5] In the control system according to any one of Items 1 to 4, The accompanying information includes at least any one of the sensing execution date and time related to the acquisition of the sensing data, the aircraft identification information, the sensor identification information, the flight control information, the sensor control information, and the state information of the sensing data, and is a control system. [Item 6] In the control system according to any one of Items 1 to 5, The data suitability determination unit performs a suitability determination on the determination target data using a preset determination condition or artificial intelligence, or displays candidates for inappropriate data, and is a control system. [Item 7] In the control system according to any one of Items 1 to 6, The data suitability determination unit performs a suitability determination on the determination target data based on the suitability determination input information for the determination target data received from the user, and is a control system. [Item 8] In the control system according to any one of Items 1 to 7, When the data suitability determination unit performs a suitability determination on the sensing data, the determination condition includes at least any one of the state of the sensing data, the state of the aircraft, and the state of the sensor. Or, when the data suitability determination unit performs a suitability determination on the processed data, the determination condition includes the state of the processed data, and is a control system. [Item 9] In the control system according to any one of Items 1 to 8, When the data suitability determination unit performs a suitability determination on the sensing data and the determination condition includes the state of the aircraft, the determination condition is set by comparing with a reference value of a state quantity related to the aircraft, or comparing with a past performance value, or comparing with a control target value of a control target state quantity related to the aircraft, and is a control system. [Item 10] In the control system according to any one of Items 1 to 9, The control system is such that when the data suitability determination unit determines the suitability of the sensing data, and when the state of the sensing data is included in the determination condition, the determination condition is set by comparing a state quantity related to the sensing data with a reference value, or comparing it with a state quantity of past sensing data, or comparing it with a target value, or the acquisition rate of the sensing data with respect to a target area to be sensed. [Item 11] In the control system according to any one of Items 1 to 10, The determination criterion when the data suitability determination unit determines the suitability of the in - process data or the post - process data is set by comparing a state quantity related to the in - process data or the post - process data with a reference value, or comparing it with a state quantity of past post - process data, or comparing it with a target value. The control system. [Item 12] In the control system according to any one of Items 1 to 11, It includes a corresponding action determination unit that determines a corresponding action when the data suitability determination unit determines that the data to be determined is inappropriate, The control system that includes information on the corresponding action in the accompanying information or executes the corresponding action. [Item 13] In the control system according to any one of Items 1 to 12, The corresponding action determination unit determines the corresponding action according to at least any one of the content of the data to be determined, the state of the flying object, the state of the sensor, and the result of the suitability determination. The control system. [Item 14] In the control system according to any one of Items 1 to 13, The corresponding action determination unit determines the corresponding action according to the designated input information of the corresponding action received from the user. The control system. [Item 15] In the control system according to any one of Items 1 to 14, The corresponding action includes at least one of the following: not using the inappropriate data, substituting other data for the inappropriate data, notifying of restrictions on the use of the processed data, notifying of the inclusion of the inappropriate data, specifying a data processing method, re-executing sensing by a flying object, specifying a data processing method according to a user-specified use, and specifying a display method of the processed data according to a user-specified use. It is a control system. [Item 16] In the control system according to any one of Items 1 to 15, The accompanying information includes information on the corresponding action. It is a control system. [Item 17] In the control system according to any one of Items 1 to 16, When the information on the corresponding action included in the accompanying information is not using the inappropriate data, the information on the target area related to the inappropriate data is included in the accompanying information. It is a control system. [Item 18] In the control system according to any one of Items 1 to 17, When the information on the corresponding action included in the accompanying information is substituting other data for the inappropriate data, the information on the other data to be substituted is included in the accompanying information. It is a control system. [Item 19] In the control system according to any one of Items 1 to 18, When the information on the corresponding action included in the accompanying information is notifying of restrictions on the use of the processed data, the information on the restrictions on the use is included in the accompanying information. It is a control system. [Item 20] In the control system according to any one of Items 1 to 19, When the information on the corresponding action included in the accompanying information is specifying a data processing method, the information on the specified data processing method is included in the accompanying information. It is a control system. [Item 21] In the control system according to any one of Items 1 to 20, When the information on the corresponding action included in the accompanying information is re - execution of sensing by the aircraft, sensing re - execution schedule information including at least any one of the scheduled position, scheduled area, and scheduled time of sensing re - execution is included in the accompanying information, control system. [Item 22] In the control system according to any one of Items 1 to 21, Receive from the user the availability determination conditions for determining that the inappropriate data is available by at least any one of the corresponding actions of notification of the usage restriction of the processed data, notification of including the inappropriate data, and designation of the data processing method, and when an inappropriate determination is made by the appropriateness determination, determine the available inappropriate data according to the availability determination conditions, or display the available inappropriate data, control system. [Item 23] In the control system according to any one of Items 1 to 22, Receive from the user the usage determination conditions for determining that the inappropriate data is available by at least any one of the corresponding actions of notification of the usage restriction of the processed data, notification of including the inappropriate data, and designation of the data processing method, and when an inappropriate determination is made by the appropriateness determination, determine the corresponding action for using the inappropriate data according to the usage determination conditions, or display the candidate corresponding actions for using the inappropriate data, control system. [Item 24] In the control system according to any one of Items 1 to 23, When an inappropriate determination is made by the appropriateness determination, determine the corresponding action or display the corresponding action according to the determination conditions of the corresponding action received from the user, or using artificial intelligence, control system. [Item 25] The computer A data acquisition step of acquiring at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on a flying object, in-process data generated during the data processing of the sensing data, and post-processed data generated by the data processing of the sensing data; A data suitability determination step of performing a suitability determination on the determination target data acquired by the data acquisition unit; An accompanying information generation step of generating accompanying information including information regarding the content of the suitability determination for the inappropriate data determined to be inappropriate; A step of associating the inappropriate data, or the surface position of the target area corresponding to the inappropriate data, or the position of the flying object, or the control target position of the flying object, or the control target path of the flying object with the accompanying information, and recording it in a recording unit or displaying it on a display unit; A control method for executing the above. [Item 26] A computer is caused to A data acquisition step of acquiring at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on a flying object, in-process data generated during the data processing of the sensing data, and post-processed data generated by the data processing of the sensing data; A data suitability determination step of performing a suitability determination on the determination target data acquired by the data acquisition unit; An accompanying information generation step of generating accompanying information including information regarding the content of the suitability determination for the inappropriate data determined to be inappropriate; A step of associating the inappropriate data, or the surface position of the target area corresponding to the inappropriate data, or the position of the flying object, or the control target position of the flying object, or the control target path of the flying object with the accompanying information, and recording it in a recording unit or displaying it on a display unit; A control program for causing the above to be executed.

[0011] <A. First Embodiment> Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted. Further, the following embodiments are merely examples, and other known elements or alternative means can be adopted according to the use, purpose, scale, etc.

[0012] [A-1. Configuration] (A-1-1. Overview) FIG. 1 is an overall configuration diagram of an aviation data processing system 1 (hereinafter also referred to as "this system 1") according to an embodiment of the present invention. As shown in FIG. 1, this system includes an aircraft 1000, a data acquisition base system 2000, a data processing base system 3000, a distribution system 4000, and a satellite system 6000.

[0013] The aircraft 1000 acquires acquired data obtained by sensing information of a target area from above using sensors including optical cameras, infrared cameras, and laser sensors such as LiDAR. The data acquisition base system 2000 is a base equipped with a remote control device that communicates control information with the aircraft when the aircraft is an unmanned aircraft, and is configured as a movable vehicle type or an immovable fixed type base. The data processing base system 3000 is a system that acquires the acquired data acquired by the aircraft, performs data processing, and generates processed data. The distribution system 4000 acquires the processed data and distributes the processed data to the user terminal device 5000 via the Internet line.

[0014] A crisis management headquarters terminal, which is an example of a user terminal device, is a terminal device used by users such as the person in charge of the crisis management headquarters. By accessing the distribution system from the crisis management headquarters terminal, the processed data can be viewed. In addition, the data acquisition base system 2000 and the data processing base system 3000 are provided with a display device and a user input device for users who manage or operate these systems.

[0015] (A-1-2. Aircraft 1000) FIG. 2 is a functional configuration diagram of an aircraft according to an embodiment of the present invention. The aircraft 1000 is a system that constitutes an aircraft, and includes a flight unit 1100 having a flight function, a sensing unit 1200 that performs sensing by sensors, a communication unit 1300 that communicates with a data acquisition base system via a satellite system 6000 or a mobile phone communication network in the aircraft data processing system 1, a state determination unit 1400 that determines each state of flight, sensing, and positioning, a sensing data recording unit 1500 that records acquired data and the like, and a state determination result recording unit 1630 that records information and state determination results related to flight control and measurement control.

[0016] In this specification, the term "aircraft" refers to an entire flying object having a function of autonomously controlling its attitude, regardless of the power means (electric power, prime mover, etc.) and the control method (wireless or wired, fully autonomous flight type or partially manual control type, etc.), and regardless of whether it is manned or unmanned. An aircraft may also be referred to as an unmanned aerial vehicle (UAV), a flying object, a multi-copter, a remote piloted aircraft systems (RPAS), or an unmanned aircraft systems (UAS). Further, the aircraft may be any of a fixed-wing type having a fixed wing, a multi-copter type having a plurality of propellers, and a VTOL type having both a fixed wing and a plurality of propellers.

[0017] (A-1-2―1. Flight Unit 1100) The flight unit 1100 includes a self-position / velocity determination unit 1110, an attitude determination unit 1120, and a flight control unit 1130.

[0018] The own - position and speed determination unit 1110 has a function of determining the position and speed of the aircraft 1000. For example, based on satellite signals received from artificial satellites by a satellite signal antenna mounted on the aircraft using GPS (Global Positioning System), GNSS (Global Navigation Satellite System), etc., the position (absolute position) of the aircraft body is measured. Also, for the determination of the own - position and speed, for example, RTK - GNSS (Real Time Kinematic - Global Navigation Satellite System) can also be used. The position information includes at least two - dimensional coordinate information (e.g., latitude, longitude) in a plan view, and preferably includes three - dimensional coordinate information including altitude information.

[0019] Next, the attitude determination unit 1120 has a function of determining the attitude (azimuth) of the aircraft 1000. Specifically, the attitude of the aircraft body is determined by a GPS compass composed of a pair of satellite antennas mounted on the aircraft, or a geomagnetic sensor, etc.

[0020] Next, the flight control unit 1130 has a processing unit also called a flight controller. The processing unit can have one or more processors such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit can access a memory (storage unit). The memory stores logic, code, and / or program instructions executable by the processing unit to perform one or more steps.

[0021] The processing unit has a function of controlling the flight state of the aircraft by controlling the output of the thrust generating device. Specifically, the propeller, the motor that drives the propeller, and the motor control unit that controls the motor output control the thrust generated from the propeller to adjust the spatial arrangement, attitude angle, angular velocity, angular acceleration, angular jerk velocity, and / or acceleration of the aircraft having six degrees of freedom (translational motions x, y, and z, and rotational motions θx, θy, and θz). That is, the flight control unit 1130 causes the aircraft to perform operations such as takeoff, forward movement, turning, and landing, and controls the attitude angle control and flight operations of the aircraft from takeoff to in-flight and landing to control the flight state of the aircraft.

[0022] (A-1-2―2. Sensing Unit 1200) The sensing unit 1200 includes a sensor 1210, a sensor attitude control unit 1220, and a sensor control unit 1230.

[0023] The sensor 1210 is composed of, for example, an optical camera that acquires an optical image, an infrared camera that acquires an infrared image, a laser sensor such as LiDAR that acquires point cloud data, etc. The sensor acquires optical images, infrared images, point cloud data, etc. of the ground surface of the sensing target area from above the sensing target area such as a disaster area as acquired data.

[0024] The sensor attitude control unit 1220 controls the relative angle of the sensor with respect to the airframe of the aircraft. For example, at least one of the angles around the three axes of the gimbal provided between the sensor and the airframe and supporting the sensor is controlled to control the relative angle of the sensor with respect to the airframe.

[0025] The sensor control unit 1230 has a function of changing the timing of performing sensing by the sensor 1210 and measurement parameters. For example, when the sensor is an optical camera, it controls the image acquisition timing, exposure, shutter speed, aperture (F value), zoom amount, ISO, resolution, etc.

[0026] (A-1-2―3. Communication Unit 1300) The communication unit 1300 includes a control communication unit 1310, a data communication unit 1320, and a status data communication unit 1330. Also, the communication unit 1300 is capable of direct communication, radio communication via a satellite system, or radio communication via a communication network NW, and includes, for example, a radio communication module. The communication unit 1300 can communicate with a data acquisition base system or the like. The communication unit 1300 has a communication function for performing wireless communication with a data acquisition base system or the like by using, for example, Wi-Fi, a frequency band of 2.4 GHz, or 5.6 - 5.8 GHz. Further, the communication unit 1300 has a wireless communication function capable of communicating with a data acquisition base system or the like via a communication network NW by using a communication standard such as LTE (Long Term Evolution).

[0027] The control communication unit 1310 conducts communication regarding the flight control of the aircraft that is exchanged between the aircraft 1000 and the data acquisition base system 2000. For example, when the aircraft is an unmanned aircraft and is remotely controlled from the data acquisition base system, the control communication unit 1310 receives a flight control command transmitted from the data acquisition base system to the aircraft. Also, the control communication unit 1310 transmits information regarding the flight state of the aircraft, including the position, speed, and attitude of the aircraft determined by the flight unit 1100, from the aircraft to the data acquisition base system.

[0028] The data communication unit 1320 transmits the acquired data obtained by the sensing unit 1200 to a data acquisition base system or the like. The status data communication unit 1330 transmits the determination results of the flight state, sensing state, and communication state determined by the status determination unit 1400, which will be described later, to the data acquisition base system.

[0029] (A-1-2-4. Status Determination Unit 1400) The status determination unit 1400 includes a flight state determination unit 1410, a sensing state determination unit 1420, and a communication state determination unit 1430.

[0030] The flight state determination unit 1410 determines a flight anomaly when the position, speed, or attitude of the aircraft determined by the self-position / speed determination unit 1110 or the attitude of the aircraft determined by the attitude determination unit 1120 deviates from a preset normal range. Also, even when the position, speed, and attitude do not deviate from the normal range, if at least any one of the position, speed, and attitude does not follow the control command value from the flight control unit 1130 normally, it is also determined as a flight anomaly.

[0031] The sensing state determination unit 1420 determines a sensing anomaly when the acquisition data is not acquired by the sensor 1210 of the sensing unit 1200 or when an anomaly is found in the acquisition data. Also, it may be determined as a sensing anomaly when the sensor attitude control unit 1220 does not operate correctly with respect to the control command value.

[0032] The communication state determination unit 1430 determines the communication state between the aircraft and the external (data acquisition base system, satellite system, etc.) by the communication unit 1300. The communication state to be determined is specifically the communication speed (such as bps), and may include the communication strength (such as dB) in addition to the communication speed. The communication state determination unit 1430 particularly determines the data communication speed of the acquisition data by the data communication unit 1320. Also, when the self-position / speed determination unit 1110 uses GPS or GNSS, or when the attitude determination unit 1120 uses a GPS compass, the communication state determination unit 1430 may determine the reception strength, positioning accuracy, and measurement accuracy of the satellite signals received from artificial satellites in GPS or GNSS. Here, the positioning accuracy can be determined based on the strength of the satellite signals, the number of satellites from which satellite signals can be received, and the elevation angle of the satellites.

[0033] (A-1-2-5. Sensing Data Recording Unit 1500) The sensing data recording unit 1500 records the acquisition data acquired by the sensing unit 1200. The recorded acquisition data is transmitted by the data communication unit 1320 to the data acquisition base system or the like. The sensing data recording unit 1500 is composed of a storage device such as a RAM or a ROM.

[0034] (A-1-2 - 6. Status Data Recording Unit 1600) The status data recording unit 1600 includes a flight control information recording unit 1610, a measurement control information recording unit 1620, and a status determination result recording unit 1630. The status data recording unit 1600 is composed of a storage device such as a RAM or a ROM.

[0035] The flight control information recording unit 1610 records the information on the position, speed, and attitude of the aircraft determined by the self-position / speed determination unit 1110 and the attitude determination unit 1120. Also, not limited to the position, speed, and attitude of the aircraft, status quantities such as acceleration, angular velocity, and angular acceleration, which are other status quantities measured by the flight unit 1100, may be recorded.

[0036] The measurement control information recording unit 1620 records the status information on the relative angle of the sensor with respect to the aircraft body obtained by the sensor attitude control unit 1220. Also, it records the timing of the sensing execution by the sensor 1210 controlled by the sensor control unit 1230 and the information on the measurement parameters (image acquisition timing, exposure, shutter speed, aperture (F value), zoom amount, ISO, resolution). Also, it acquires the information on whether or not the sensing is executed by the sensor 1210 from the sensor control unit 1230, or acquires the information on whether or not the sensing data is recorded in the sensing data recording unit 1500, and records it as the sensing data presence / absence information.

[0037] The status determination result recording unit 1630 records the determination results determined by the flight status determination unit 1410, the sensing status determination unit 1420, and the communication status determination unit 1430 of the status determination unit 1400.

[0038] (A-1-3. Data Acquisition Base System 2000) FIG. 3 is a configuration diagram of a data acquisition base system according to an embodiment of the present invention. The data acquisition base system 2000 includes a communication infrastructure management system 2100, an aircraft operation operate system 2200, a data management system 2300, an operation management system 2500, and an airspace monitoring system 2600.

[0039] When the aircraft is an unmanned aircraft, the data acquisition base system 2000 is a system that exchanges control information regarding the aircraft, flight unit, and sensing unit, enabling remote control or data acquisition operations by an operator (user) who uses the data acquisition base system. On the other hand, when the aircraft is a manned aircraft, it is a system that exchanges control information and airframe status information regarding the aircraft, flight unit, and sensing unit to support data acquisition operations by the aircraft. Further, the data acquisition base system may be composed of movable vehicles, ships, flying bodies, etc., or may be composed of immovable buildings (fixed type) that do not move.

[0040] The communication infrastructure management system 2100 is a system that manages the communication means for data (acquired data obtained by the sensing operation of the aircraft, attached information generated by the data management system, etc.) and control communication information transmitted and received between the data acquisition base system 2000, the aircraft 1000, and the data processing base system 3000. The communication infrastructure management system 2100 transmits information generated by, for example, the airframe operation control system 2200 and the data management system 2300 described later to the aircraft 1000.

[0041] In addition, as communication means with the aircraft 1000, there are multiple communication means such as communication means for directly communicating with the aircraft 1000 using antenna facilities within the data acquisition base system 2000, communication means via a mobile phone communication network (such as LTE), and communication means for communication by satellite relay via the satellite system 6000. Therefore, the communication infrastructure management system 2100 monitors the communication speed and communication strength of these communication means and performs switching of communication means. The transmission process may be, for example, parallel transmission or switched transmission. Further, when the communication infrastructure management system 2100 determines, based on the data management system 2300 described later, that it is appropriate to transmit data by physically transporting a memory by a person, it may display that fact on an appropriate display unit and request the administrator to physically transport the memory.

[0042] The airframe operation control system 2200 is a system that generates the mission of the aircraft to be controlled and controls the movement of the aircraft. The mission is a movement plan including, for example, the movement route and movement speed of the aircraft 1000. The movement route is generated in an airspace below an altitude of 150 m, for example, to avoid interference with other aircraft such as passenger aircraft in advance. The airframe operation control system 2200 transmits a control signal to the aircraft 1000 via the communication infrastructure management system 2100 in order to automatically operate the aircraft.

[0043] The data management system 2300 acquires the acquired data and status data of the target area sensed by the aircraft via the communication infrastructure management, determines the content of the acquired data and the appropriateness of the status data at the time of sensing, and generates associated information. The detailed functions of the data management system 2300 will be described later.

[0044] The operation management system 2500 is a system that makes decisions and gives instructions for the operation of the aircraft 1000. The operation management system 2500 formulates a plan for the operations of the aircraft 1000 including, for example, sensing and flight, and transmits it to the aircraft 1000 via the communication infrastructure management system 2100. In addition, the operation management system 2500 may determine the priority of the operations that require processing and transmit it to the aircraft 1000. The operation management system 2500 may formulate plans for a plurality of aircraft 1000 and transmit the information of the respective plans to each aircraft 1000.

[0045] The airspace monitoring system 2600 is a system that monitors the airspace in which the aircraft 1000 to be controlled is flying. The airspace monitoring system 2600 acquires information from an existing air traffic control system. The air traffic control system may be, for example, a drone operation management system (UTM), an operation management subsystem (UASSP), or an air traffic management system (ATM). That is, the airspace monitoring system 2600 measures or acquires information on the environment and other aircraft in the airspace in which the aircraft 1000 is flying, and transmits the information to the operation management system 2500. The operation management system 2500 changes the work plan of the aircraft 1000 if there is a problem with the work plan based on the information from the airspace monitoring system 2600.

[0046] (A-1-4. Data Processing Base System 3000) FIG. 4 is a configuration diagram of a data processing base system according to an embodiment of the present invention. The data processing base system includes a communication unit 3100, a data processing unit 3200, and a data processing management unit 3300.

[0047] The communication unit 3100 transmits and receives data (acquired data and accompanying information) between the data processing base system 3000 and external systems (the data acquisition base system 2000 and the distribution system 4000). The communication unit 3100 receives, for example, acquired data and accompanying information from the data acquisition base system 2000. Also, the processed data generated in the data processing base system 3000 is transmitted to the distribution system 4000.

[0048] The data processing unit 3200 executes data processing based on the acquired data acquired by the aircraft 1000 and the accompanying information generated by the data management system.

[0049] The data processing management unit 3300 determines the suitability and corresponding actions of the processed data based on the content of the processed data or the data being processed generated by the data processing unit 3200, generates accompanying information, and provides the processed data and the accompanying information to the distribution system. Details of the data processing management unit 3300 will be described later.

[0050] (A-1-5. Distribution System 4000) FIG. 5 is a configuration diagram of a distribution system according to an embodiment of the present invention. The distribution system 4000 includes a communication unit 4100 and a distribution data management unit 4200.

[0051] The communication unit 4100 receives processed data and accompanying information from the data processing base system 3000. The received processed data and accompanying information may be data after web optimization processing is performed in the data processing unit 3200 of the data processing base system 3000. Further, the communication unit 4100 distributes the processed data to user terminal devices 5000 including a crisis management headquarters terminal via the Internet line.

[0052] The distribution data management unit 4200 records the processed data received via the communication unit 4100, and distributes the latest processed data recorded in the distribution data management unit 4200 to user terminal devices including a crisis management headquarters terminal that accesses via the Internet line.

[0053] (A-1-6. Data Management System 2300) FIG. 6 is a functional configuration diagram of a data management system according to an embodiment of the present invention. The data management system 2300 includes a user request acquisition unit 2310, a sensing data acquisition unit 2320, a state information acquisition unit 2330, a data appropriateness determination unit 2340, a corresponding action determination unit 2350, an accompanying information addition unit 2360, and a user input reception unit 2370.

[0054] (A-1-6-1. User Request Acquisition Unit 2310) The user requirement acquisition unit 2310 acquires in advance the requirement information from the user. The requirement information acquired by the user requirement acquisition unit 2310 includes the appropriateness judgment criteria in the appropriateness judgment for the sensing data executed by the data appropriateness judgment unit 2340 described later. In this case, for example, the appropriateness judgment criteria include judgment criteria regarding the state of the sensing data by the aircraft, the state of the aircraft, the state of the sensor, etc. For example, it includes various inappropriate judgment criteria as described in the user requirement information shown in Table T101 of FIG. 8 described later.

[0055] Here, when the sensing data is an image captured by a camera, as a criterion for inappropriate judgment regarding the state of the sensor, for example, the resolution r of the camera image is less than or equal to the reference value P1, or the luminance Y of the camera image is less than or equal to the reference value L1, or the focus of the camera image is off, etc., the judgment criteria resulting from the setting parameters of the sensor can be acquired as user requirement information.

[0056] Also, as a criterion for inappropriate judgment regarding the state of the aircraft, for example, the absolute value of the difference between the position X of the aircraft and the target position X1 is greater than or equal to the reference value, the absolute value of the difference between the flight speed V of the aircraft and the target speed V1 is greater than or equal to the reference value, the absolute value of the difference between the flight altitude h of the aircraft and the target altitude h1 is greater than or equal to the reference value, the angular difference between the attitude angle (Yaw·Roll·Pitch) of the aircraft and the target angle is greater than or equal to the reference value (for example, 15 degrees), etc., the judgment criteria regarding the flight state of the aircraft can be acquired as user requirement information.

[0057] Also, as a criterion for inappropriate judgment regarding the state of the sensing data, for example, the sensing operation is not properly executed and there is data loss in the camera image, there are water droplets, condensation, or dirt on the lens when the camera image is acquired, or there are defects in the camera image due to clouds, fog, etc., the target area image cannot be acquired because the lens cover of the camera image is closed, etc., the judgment criteria regarding the content of the sensing data can be acquired as user requirement information.

[0058] The desired information obtained by the user requirement acquisition unit 2310 may include the appropriateness determination criteria in the appropriateness determination for the processed data executed by the processed data appropriateness determination unit 3330 described later. In this case, for example, the appropriateness determination criteria include inappropriate determination criteria related to the state of the processed data.

[0059] The desired information obtained by the user requirement acquisition unit 2310 includes the determination criteria for the corresponding action determination executed by the corresponding action determination unit 2350 described later. In this case, for example, the determination criteria for the corresponding action include a plurality of determination criteria related to the state of the sensing data by the aircraft, the state of the aircraft, the state of the sensor, etc., and the corresponding actions corresponding to each determination criteria. For example, it includes each determination criteria as described in the corresponding action determination criteria shown in Table T102 of FIG. 9 described later.

[0060] Here, when the sensing data is an image captured by a camera, as the criteria for the corresponding action determination according to the state of the sensor, for example, the resolution r of the camera image, the luminance Y of the camera image, the defocus of the camera image, etc., the corresponding action according to the sensor state caused by the setting parameters of the sensor can be obtained as the user requirement information.

[0061] Also, as the criteria for the corresponding action determination according to the state of the aircraft, for example, the difference between the position X of the aircraft and the target position, the difference between the flight speed V of the aircraft and the target speed, the difference between the flight altitude h of the aircraft and the target altitude, the angular difference between the attitude angle (Yaw·Roll·Pitch) of the aircraft and the target angle, etc., the corresponding action according to the flight state of the aircraft can be obtained as the user requirement information.

[0062] In addition, as criteria for determining corresponding actions according to the state of sensing data, for example, due to data loss in a camera image caused by the influence of clouds, fog, etc., defects in the camera image due to water droplets, condensation, or dirt on the lens when acquiring the camera image, or the image of the target area not being acquired because the lens cover of the camera image is closed, etc., corresponding actions according to defects in the content of the sensing data can be acquired as user demand information.

[0063] Note that the above-mentioned corresponding actions include corresponding actions that allow the use of inappropriate data determined to be inappropriate (for example, "notification of restrictions on the use purpose of processed data", "notification of including inappropriate data", "specification of data processing methods", "specification of the display method of processed data", etc.). On the other hand, corresponding actions that prohibit the use of inappropriate data determined to be inappropriate (for example, "non-use of inappropriate data", "substitution with other data instead of inappropriate data", "re-execution of sensing by an aircraft") may also be included. That is, the demand information acquired by the user demand acquisition unit 2310 may include criteria for determining whether to allow the use of inappropriate data determined to be inappropriate.

[0064] (A-1-6―2. Sensing Data Acquisition Unit 2320) The sensing data acquisition unit 2320 acquires sensing data sensed by the sensing unit 1200 mounted on the aircraft via the communication infrastructure management system 2100. The sensing data acquired here is, for example, optical image data or point cloud data acquired by a laser sensor such as LiDAR.

[0065] (A-1-6―3. Status Information Acquisition Unit 2330) The status information acquisition unit 2330 acquires each piece of information recorded in the flight control information recording unit 1610, measurement control information recording unit 1620, and status determination result recording unit 1630 of the aircraft via the communication infrastructure management system 2100.

[0066] FIG. 10 is a table showing an example of information acquired by the state information acquisition unit according to an embodiment of the present invention. The information acquired from the flight control information recording unit 1610 is, for example, in addition to the position, flight speed, flight altitude, and attitude angles (Yaw·Roll·Pitch) of the aircraft as shown in Table T103 of FIG. 10, state quantities such as acceleration, angular velocity, and angular acceleration. The information acquired from the measurement control information recording unit 1620 is, for example, the image acquisition timing, exposure, shutter speed, aperture (F value), zoom amount (affecting focus), ISO, resolution, luminance of the sensor as shown in Table T103 of FIG. 10, as well as other measurement parameters (contrast, histogram, etc.), and the timing of sensing execution and the state information of the relative angles (Yaw·Roll·Pitch) of the sensor with respect to the aircraft body. Further, sensing data presence / absence information regarding the presence or absence of sensing data may be acquired from the measurement control information recording unit 1620.

[0067] Also, the information acquired from the state determination result recording unit 1630 is the information of the determination results determined by the flight state determination unit 1410, the sensing state determination unit 1420, and the communication state determination unit 1430 of the state determination unit 1400. In particular, the determination results such as the reception intensity of satellite signals, the positioning accuracy, and the measurement accuracy are acquired.

[0068] (A-1-6―4. Data suitability determination unit 2340) The data suitability determination unit 2340 determines the suitability of the sensing data based on the content of the sensing data acquired by the sensing data acquisition unit 2320, or various state information (including the state of the aircraft and the state of the sensor) acquired by the state information acquisition unit 2330, or a combination thereof, and displays the determination result. Further, the determination may be made based on the suitability determination criteria as shown in FIG. 8, or the suitability determination using artificial intelligence may be performed. Or, it may be a determination based on the input information of the determination result determined to be inappropriate by the user. The data suitability determination unit 2340 includes a data analysis unit 2341, a state information analysis unit 2342, and a suitability determination result display unit 2343.

[0069] FIG. 8 is a table showing an example of the criteria for determining the suitability of sensing data according to an embodiment of the present invention. Table T101 in FIG. 8 shows an example of the criteria for inappropriate determination when the sensing data is an image captured by a camera. The inappropriate determination criteria include inappropriate criteria related to the sensor state, inappropriate criteria related to the state of the aircraft, and inappropriate criteria related to the state of the sensing data.

[0070] This suitability determination criterion can be set based on the input information from the user acquired by the user requirement acquisition unit 2310, but is not limited thereto, and may be set in advance by the system. Alternatively, the criterion to be applied may be selected by the user inputting in the selection column on the left side of T101 with respect to the default setting criterion for each target state quantity preset by the system, or arbitrary conditions may be set by the user inputting in the criterion column for inappropriate determination on the right side of T101.

[0071] Also, as an example of the criteria for determining the suitability of sensing data different from FIG. 8, the user inputs the usage of the sensing data or the processed data obtained by processing it (such as land feature inspection use, forest survey use, etc.), and according to the input usage, the system may automatically generate the criteria for inappropriate determination.

[0072] First, the data analysis unit 2341 analyzes the content of the sensing data acquired by the sensing data acquisition unit 2320, and determines whether the sensing data is inappropriate in light of the criteria for inappropriate determination related to the state of the sensing data as shown in Table T101 of FIG. 8. In Table T101, as the criteria for inappropriate determination related to the state of the sensing data, for example, there is data loss in a part of the camera image due to the influence of clouds, fog, etc., water droplets, condensation, dirt on the lens at the time of camera image acquisition, or there are defects in a part of the camera image due to local luminance deficiency or whiteout, the image of the target area cannot be acquired because the lens cap of the camera image is closed, the luminance Y of the camera image is less than or equal to the reference value L1, or the reference value L2 ≧, the focus of the camera image is out of focus, etc., and shows the determination criteria that can be determined based on the analysis of the content of the sensing data.

[0073] In addition to the above, the criteria may also include that the image is distorted, the image is blurred, the image is from an unnecessary position outside the sensing target area (such as the turning route of the aircraft), the resolution, contrast, or histogram is below a predetermined value, no sensing data information is received, the relative angle (Yaw·Roll·Pitch) of the sensor with respect to the aircraft body exceeds the reference value, etc.

[0074] Also, when performing sensing by a laser sensor, as criteria for inappropriate determination, it may also be included in the criteria for inappropriate determination that there are no predetermined feature points in the point cloud data, the reflectivity of the laser is below a predetermined value, there is an area with missing point cloud acquisition within the sensing target area, the point cloud is from an unnecessary position outside the sensing target area (such as the turning route of the aircraft), etc.

[0075] When the data analysis unit 2341 makes an inappropriate determination based on the state of the sensing data, as described above, in addition to the determination method by comparing the state quantity related to the sensing data with the reference value shown in T101, it can also be determined by comparing with the state quantity of past sensing data, or comparing with the target value of the state quantity of the sensing data, or the acquisition rate of the sensing data for the target area of the sensing target, etc.

[0076] Next, the state information analysis unit 2342 analyzes the content of various state information acquired by the state information acquisition unit 2330, and determines whether the sensing data is inappropriate in light of the criteria for inappropriate determination regarding the state of the sensor or the state of the aircraft as shown in the table T101 of FIG. 8. In the table T101, as criteria for inappropriate determination regarding the state of the sensor, for example, the resolution r of the camera image is below the reference value P1, or the luminance Y of the camera image is below the reference value L1, or is greater than or equal to the reference value L2, or the focus of the camera image is off, etc., determination criteria resulting from the setting parameters of the sensor are shown.

[0077] In addition to the above, receiving information without sensing data, the relative angles (Yaw·Roll·Pitch) of the sensor with respect to the aircraft body deviating from the reference range, the absolute angles (Yaw·Roll·Pitch) of the sensor deviating from the reference range, and the data of the point to be sensed not being acquirable may also be included in the determination criteria.

[0078] Also, in Table T101, as criteria for inappropriate determination regarding the state of the aircraft, for example, the difference X between the position of the aircraft and the target position (or target route) being outside the reference value range, the difference V between the flight speed of the aircraft and the target speed being outside the reference value range, the difference h between the flight altitude of the aircraft and the target altitude being outside the reference value range, the angular difference between the attitude angle (Yaw·Roll·Pitch) of the aircraft and the target angle being outside the reference range (for example, 15 degrees or more), etc., determination criteria regarding the flight state of the aircraft are shown.

[0079] In addition to the above, the reception intensity of the satellite signal falling below the reference value, the measurement accuracy such as the positioning accuracy and speed of the position of the aircraft falling below the reference value, the state quantities such as the acceleration, angular velocity, and angular acceleration of the aircraft deviating from the reference range, etc. may also be included in the determination criteria. Also, particularly when performing sensing by a laser sensor, since laser reflected light cannot be obtained if the distance to the sensing object is too far, the flight altitude being higher than a predetermined value may also be included in the determination criteria for inappropriate determination.

[0080] When the state information analysis unit 2342 makes an inappropriate determination based on the state of the aircraft, as described above, in addition to the determination method by comparing the state quantity of the aircraft with the reference value shown in T101, an inappropriate determination may also be made by comparing with the past performance value regarding the aircraft or by comparing with the control target value of the control target state quantity regarding the aircraft.

[0081] In addition, as the method for determining the above-mentioned inappropriate determination, it is also possible to use a method of determining that it is inappropriate if the determination criteria are deviated even once. However, if the number of times of deviating from the determination criteria exceeds a predetermined number, or when the area deviating from the semi-hand standard exists in a part of the image and the area deviating from the determination criteria exceeds a predetermined area, or the ratio occupying the entire image exceeds a predetermined ratio, it may also be a determination method of determining that it is inappropriate.

[0082] Next, the appropriateness determination result display unit 2343 displays the result of the above-mentioned inappropriate determination. FIG. 12 is a diagram showing a display example when the appropriateness determination result display unit of the present invention displays the result of the appropriateness determination. In the example shown in FIG. 12, inappropriate data determined to be inappropriate for the sensing data subject to the appropriateness determination is displayed in an identifiable manner. The user of the data management system 2300 can select any inappropriate data by the user input reception unit 2370 described later. When any inappropriate data is selected, the data content of the selected inappropriate data (image data if the inappropriate data is an image) is displayed, and the reason for the determination as inappropriate (such as out of focus), the degree of exceeding the reference value (such as large, medium, or small), the time when the inappropriate data was sensed, and the coordinate information of the ground area corresponding to the inappropriate data are displayed.

[0083] In addition, when any inappropriate data is selected, the appropriateness determination result display unit 2343 displays an input button on the screen that can be used to select whether to confirm the inappropriate data together with the above-mentioned information, so that the user can operate it by the user input reception unit 2370 described later.

[0084] (A-1-6-5. Corresponding action determination unit 2350) The corresponding action determination unit 2350 determines a corresponding action or displays candidates for the corresponding action based on the content of the sensing data acquired by the sensing data acquisition unit 2320, or various state information (including the state of the aircraft and the state of the sensor) acquired by the state information acquisition unit 2330, or a combination thereof, or the result of an inappropriate determination by the aforementioned data validity determination unit 2340. The corresponding action determination unit 2350 includes a determination processing unit 2351 and a corresponding action determination result display unit 2352. The determination processing unit 2351 determines the corresponding action to be taken particularly when the data is determined to be inappropriate by the data validity determination unit 2340.

[0085] Here, FIG. 9 is a diagram showing an example of the corresponding action determination criteria according to an embodiment of the present invention. The determination processing unit 2351 may determine the corresponding action based on the corresponding action determination criteria as shown in Table T102 of FIG. 9, or may determine the corresponding action using artificial intelligence or the like. Alternatively, the corresponding action may be determined based on the command input information of the corresponding action received from the user.

[0086] Table T102 in FIG. 9 shows an example of the determination criteria for corresponding action determination when it is assumed that the sensing data is an image captured by a camera. The corresponding action determination criteria include inappropriate criteria related to the sensor state, the state of the aircraft, and the determination criteria for the corresponding action according to the content of the sensing data. However, the determination criteria for corresponding action determination are not limited to this, and the determination criteria for the corresponding action corresponding to the determination result of the inappropriate determination may be set.

[0087] This determination criteria can be set based on the input information from the user acquired by the user requirement acquisition unit 2310, but is not limited to this, and may be set in advance by the system. Alternatively, the user may select the criteria to be applied by inputting in the selection column on the left side of T102 with respect to the default setting criteria for each target state quantity preset by the system, or may set arbitrary conditions by inputting in the corresponding action determination column on the right side of T102.

[0088] Also, as an example of the correspondence action determination criteria for sensing data different from that in FIG. 9, the user may input the usage of the sensing data or the post-processed data obtained by processing the sensing data (such as ground feature inspection usage, forest survey usage, etc.), and according to the input usage, the system may automatically generate the criteria and determination results for the correspondence action determination.

[0089] Examples of the correspondence actions determined by the determination processing unit 2351 include, for example, not using the inappropriate data determined to be inappropriate, substituting other data for the inappropriate data, notifying the restriction on the usage of the post-processed data, notifying the inclusion of inappropriate data, specifying the data processing method, re-performing sensing by an aircraft, specifying the data processing method according to the user-specified usage, specifying the display method of the post-processed data according to the user-specified usage, and the like.

[0090] The above-mentioned "specifying the data processing method according to the user-specified usage" and "specifying the display method of the post-processed data according to the user-specified usage" mean, for example, specifying the data processing method and the display method of the post-processed data according to the specified input of the usage obtained by the user requirement acquisition unit 2310.

[0091] Here, the determined correspondence actions include the correspondence actions that do not use the inappropriate data determined to be inappropriate and the correspondence actions that allow the use of the inappropriate data. The correspondence actions that do not use the inappropriate data are, for example, "not using the inappropriate data determined to be inappropriate", "substituting other data for the inappropriate data", "re-performing sensing by an aircraft", and the correspondence actions that allow the use of the inappropriate data are, for example, "notifying the restriction on the usage of the post-processed data", "notifying the inclusion of inappropriate data", "specifying the data processing method", "specifying the data processing method according to the user-specified usage", "specifying the display method of the post-processed data according to the user-specified usage".

[0092] As a criterion for determining the corresponding action, for example, for inappropriate data where the degree of excess from the criterion for inappropriate determination is less than a predetermined value, a corresponding action that allows the use of the inappropriate data may be taken, and for inappropriate data where the degree of excess from the criterion for inappropriate determination is greater than the predetermined value, a corresponding action of not using the inappropriate data may be taken.

[0093] First, in Table T102, as a criterion for determining the corresponding action regarding the state of the sensing data, for example, when there is data loss in a part of the camera image due to the influence of disturbances such as clouds or fog, as the corresponding action, "substitute with other data" and "re-measurement" are determined. Here, "substitute with other data" means substituting other data (such as geographical and map information or past sensing data) for inappropriate data. Also, "re-measurement" means re-performing sensing by an aircraft.

[0094] Also, in Table T102, when there are defects in a part of the camera image due to water droplets, condensation, dirt on the lens, or local luminance deficiency or whiteout during camera image acquisition, or when the image of the target area cannot be acquired because the lens cover of the camera image is closed, "substitute with other data" and "re-measurement" are determined as corresponding actions that do not allow the use of inappropriate data.

[0095] Also, when the luminance Y of the camera image is less than or equal to the reference value L1 or is greater than or equal to the reference value L2, as the corresponding action, "image adjustment for luminance adjustment" is determined. That is, as a corresponding action that allows the use of inappropriate data determined to be inappropriate by the data appropriateness determination unit 2340, "specification of data processing method" is determined. In other words, on the condition that image adjustment for luminance adjustment is performed as the specified data processing method, it is determined to allow the use of inappropriate data for data processing.

[0096] Also, when the focus of the camera image is out of focus due to, for example, the focus setting of the camera, as a countermeasure action for allowing the use of inappropriate data, "notification of including inappropriate data" and "not available for use in human determination use purposes" are determined. Here, "notification of including inappropriate data" means allowing the use of inappropriate data for data processing on the condition that the data processing person or the user who views the processed data is notified that inappropriate data is included. Also, "not available for use in human determination use purposes" means allowing the use of inappropriate data for data processing on the condition that the data processing person or the user who views the processed data is notified that it is not to be used for purposes of determining the presence of a person, etc. from the image data as the content of restricting the use purpose of the processed data.

[0097] Next, in Table T102, as a criterion for determining the countermeasure action regarding the state of the aircraft, for example, when the resolution r of the camera image is less than or equal to the reference value P1 and greater than or equal to the reference value P2, "notification of including inappropriate data" and "not available for use in human determination use purposes" are determined as countermeasure actions for allowing the use of inappropriate data. Also, when the resolution r of the camera image is less than the reference value P2, "substitute with other data" and "re-measurement" are determined as countermeasure actions for not allowing the use of inappropriate data.

[0098] Next, in Table T101, as a criterion for determining the countermeasure action regarding the state of the aircraft, for example, when the absolute value X of the difference between the position of the aircraft and the target position (or target route) is greater than or equal to the reference value X1 and less than or equal to the reference value X2, "position correction processing of the ground surface photographing area according to the position error" is determined as a countermeasure action for allowing the use of inappropriate data. That is, it is determined that the use of inappropriate data for data processing is allowed on the condition that "position correction processing of the ground surface photographing area according to the position error" is performed as the specified data processing method. On the other hand, when the absolute value X of the difference between the position of the aircraft and the target position (or target route) is greater than X2, "substitute with other data" and "re-measurement" are determined as countermeasure actions for not allowing the use of inappropriate data.

[0099] Also, when the absolute value V of the difference between the flight speed of the aircraft and the target flight speed is equal to or greater than the reference value V1 and equal to or less than the reference value V2, it is determined as the corresponding action of allowing the use of inappropriate data, "notification of including inappropriate data". On the other hand, when the absolute value V of the difference between the flight speed of the aircraft and the target flight speed is greater than V2, it is determined as the corresponding actions of not allowing the use of inappropriate data, "substitute with other data" and "re-measurement".

[0100] Also, when the absolute value h of the difference between the flight altitude of the aircraft and the target flight altitude is equal to or greater than the reference value h1 and equal to or less than the reference value h2, it is determined as the corresponding action of allowing the use of inappropriate data, "notification of including inappropriate data". On the other hand, when the absolute value h of the difference between the flight altitude of the aircraft and the target flight altitude is greater than h2, it is determined as the corresponding actions of not allowing the use of inappropriate data, "substitute with other data" and "re-measurement".

[0101] Also, when the angular difference between the attitude angle (Yaw·Roll·Pitch) of the aircraft and the target angle is within a predetermined range (for example, 15 degrees or more and 30 degrees or less), it is determined as the corresponding action of allowing the use of inappropriate data, "position correction processing of the ground surface imaging area according to the angular error". That is, it is determined to allow the use of inappropriate data for data processing on the condition that "position correction processing of the ground surface imaging area according to the angular error" is performed as the specified data processing method. On the other hand, when the angular difference between the attitude angle (Yaw·Roll·Pitch) of the aircraft and the target angle is greater than 30 degrees, it is determined as the corresponding actions of not allowing the use of inappropriate data, "substitute with other data" and "re-measurement".

[0102] As described above, in the example shown in Table T102 of FIG. 9, the available inappropriate data is determined by determining either the corresponding action of not allowing the use of inappropriate data or the corresponding action of allowing the use of inappropriate data by the determination processing unit 2351.

[0103] Next, the corresponding action determination result display unit 2352 displays the determination result of the corresponding action determined by the determination processing unit 2351. FIG. 13 is a diagram showing a display example when the corresponding action determination result display unit of the present invention displays the determination result of the corresponding action. In the example shown in FIG. 13, "image processing for brightness adjustment" is displayed as the determination result of the corresponding action. In addition, other candidates for corresponding actions (such as notification to the user, inapplicable for human judgment, data utilization not allowed, substitution with other data, etc.) are also displayed.

[0104] A user of the data management system 2300 can select any inappropriate data by the user input reception unit 2370 described later, and the determination result of the corresponding action corresponding to the selected inappropriate data is displayed. Also, when any inappropriate data is selected, the above-described determination result of the corresponding action and other candidates are displayed in a selectable manner, and further, a confirmation button for the corresponding action is displayed. The user can select any corresponding action by the user input reception unit 2370, and can confirm the corresponding action by operating the confirmation button for the corresponding action in a state where a specific corresponding action is selected.

[0105] (A-1-6-6. Associated Information Adding Unit 2360) The associated information adding unit 2360 generates associated information including information regarding the content of the appropriateness determination determined by the data appropriateness determination unit 2340, and records the associated information in association with inappropriate data or the like, or displays it on the display unit. The associated information adding unit 2360 includes an associated information generation unit 2361, an information adding unit 2362, and an associated information display unit 2363. The associated information generation unit 2361 generates associated information, and records and displays the associated information in association with inappropriate data or the like.

[0106] FIG. 11 is a table showing an example of the additional information generated by the additional information providing unit according to an embodiment of the present invention. In the table T104 of FIG. 11, as an example of the additional information generated by the additional information generation unit 2361, information on the degree to which inappropriate data exceeds the reference value as a result of inappropriate determination, the reason for inappropriate determination, the measurement conditions of inappropriate determination data, the content of the corresponding action, and the corresponding area of inappropriate data are included.

[0107] The information on "result of inappropriate determination" included in the additional information is information that can identify whether the sensing data is determined to be inappropriate by the data appropriateness determination unit 2340. For example, it can be information indicating the presence of an inappropriate determination flag. Next, the "information on the degree to which inappropriate data exceeds the reference value for appropriateness determination" included in the additional information is information indicating how much it exceeds the reference value of the inappropriate determination criterion when the data appropriateness determination unit 2340 determines that the data is inappropriate. For example, it can be information such as three levels of large, medium, and small inappropriate excess levels.

[0108] Next, the "reason for inappropriate determination" included in the additional information is information regarding the reason for being determined inappropriate when the data appropriateness determination unit 2340 determines that the data is inappropriate. For example, it is identification information of the determination criterion that corresponds to the inappropriate determination criterion among the inappropriate determination criteria described in the table T101 of FIG. 8. As an example, it is "the difference between the target position of the aircraft position and the reference value is greater than or equal to the reference value".

[0109] Next, the "measurement conditions of inappropriate data" included in the additional information is information regarding the measurement conditions of the sensing data determined to be inappropriate by the data appropriateness determination unit 2340. For example, it is information including the sensing date and time, aircraft identification information, sensor identification information, aircraft control information, sensor control information, and state information of the sensing data. Specifically, it is information acquired by the state information acquisition unit 2330.

[0110] Next, the "content of the corresponding action" included in the accompanying information is the information on the corresponding action determined by the corresponding action determination unit 2350, and is, for example, the information on the corresponding action shown in Table T102 of FIG. 9.

[0111] Next, the "related position / area of inappropriate data" included in the accompanying information is the information on the position or area related to the inappropriate determination determined to be inappropriate. For example, it can be the ground position sensed by the inappropriate data, or the position of the aircraft when sensed by the inappropriate data. Considering the case where the deviation of the position and attitude of the aircraft from the reference range is the cause of the inappropriate determination, it is desirable to use the ground position that the inappropriate data should have originally sensed, or the control target position of the aircraft when sensed by the inappropriate data, or the control target path. Alternatively, when the data is determined to be inappropriate by the data suitability determination due to the lack of sensing data (not sensed) at the ground position or the position of the aircraft to be sensed, it can be the ground position that should have been originally sensed, or the control target position of the aircraft, or the control target path.

[0112] In Table T104 of FIG. 11, an example is shown in which the information on the three-dimensional coordinates of the four corners of the ground position sensed by the inappropriate data is shown as the "related position / area of inappropriate data".

[0113] Examples of combinations of the corresponding actions and other information included in the accompanying information will be given and described below.

[0114] First, when the information on the corresponding action included in the accompanying information is the non-use of inappropriate data, the information on the target area related to the inappropriate data is included in the accompanying information. The information on the target area related to the inappropriate data is, for example, the ground position sensed by the inappropriate data, or the position of the aircraft when sensed by the inappropriate data, the ground position that the inappropriate data should have originally sensed, or the control target position of the aircraft when sensed by the inappropriate data, or the control target path, etc.

[0115] Next, when the information on the corresponding action included in the accompanying information is a substitute for other data instead of inappropriate data, information on the other data to be substituted is included in the accompanying information. This other data to be substituted is, for example, geographical information (map information), image data acquired in the past, or spatial data including three-dimensional shape information that has already been created.

[0116] Next, when the information on the corresponding action included in the accompanying information is a notification of restrictions on the usage of the processed data, information on the restrictions on the usage is included in the accompanying information. Here, the information on the restrictions on the usage is, for example, not available for use in applications such as image analysis for human determination. Although inappropriate data determined to be inappropriate because of out-of-focus or resolution below the reference value of the image can be provided to the user as an ortho-image, it is considered that there is a high possibility of misdetection or non-detection of the detection target for which image analysis is to be performed because the situation of the target area is not clearly captured. Therefore, it is desirable not to make it available for use in applications such as image analysis for determination.

[0117] Next, when the information on the corresponding action included in the accompanying information is a specification of a data processing method, information on the specified data processing method is included in the accompanying information. Here, the information on the specified data processing method is, for example, image processing for brightness adjustment in the case of inappropriate determination due to abnormal brightness, or position correction processing of the ground surface imaging area according to the position error in the case of inappropriate determination due to the position error of the aircraft.

[0118] Next, when the information on the corresponding action included in the accompanying information is re-execution of sensing by an aircraft, sensing re-execution schedule information including at least any one of the scheduled position, scheduled area, and scheduled time of the sensing re-execution is included in the accompanying information. Here, the scheduled position of the sensing re-execution includes the ground position of the sensing schedule or the target position of the aircraft at the time of sensing, the scheduled area includes the ground area of the sensing schedule or the target flight area of the aircraft at the time of sensing, and the scheduled time includes the date and time information of the sensing schedule.

[0119] Next, the function of the information adding unit 2362 to record the associated information in association with inappropriate data or the like will be described. The information adding unit 2362 associates inappropriate data determined to be inappropriate, or the surface position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target route of the aircraft with the associated information and records it in the storage device.

[0120] Here, the ground position of the target area related to the inappropriate determination includes, for example, the sensed ground position sensed by the inappropriate data, and the position coordinates (latitude, longitude, height (L.L.H.)) of the ground position that should have been sensed originally, or the position coordinates (latitude, longitude, height (L.L.H.)) above it. Also, the position of the aircraft related to the inappropriate determination includes, for example, the position coordinates (latitude, longitude, height (L.L.H.)) of the aircraft when the inappropriate data is acquired by sensing. Further, the control target position of the aircraft related to the inappropriate determination includes, for example, the control target position of the aircraft that should have been sensed originally when the inappropriate data is acquired by sensing. Also, the control target route of the aircraft related to the inappropriate determination includes the control target route of the aircraft that should have been sensed originally. Here, the control target route may be a flight mission expressed by a Waypoint including a target position and a target speed. Also, the control target route and the control target position are generally created in advance before the aircraft starts flying.

[0121] Next, the associated information display unit 2363 can inform the user of the data management system 2300 of the content of the associated information by displaying the associated information generated by the associated information generation unit 2361. As an example, the associated information is displayed on the screen of at least one of the display devices in the map format, three-dimensional model, and three-dimensional space, together with the inappropriate data, or the position coordinates (latitude, longitude, height (L.L.H.)) of the ground position of the target position related to the inappropriate determination, or the position coordinates (latitude, longitude, height (L.L.H.)) of the position of the aircraft, or the position coordinates (latitude, longitude, height (L.L.H.)) of the control target position of the aircraft, or in association with the control target path of the aircraft. Here, the map is two-dimensional or three-dimensional map format information, and the three-dimensional model or three-dimensional space is point cloud data or three-dimensional information generated by SfM processing. Display examples of the associated information displayed by the associated information display unit 2363 are shown in FIGS. 14, 15, and 16. FIG. 14 is a diagram showing a display example when the associated information display unit of the present invention displays the associated information in association with inappropriate data. FIG. 15 is a diagram showing a display example when the associated information display unit of the present invention displays the associated information in association with the ground sensing target area. FIG. 16 is a diagram showing a display example when the associated information display unit of the present invention displays the associated information in association with the position of the aircraft.

[0122] In the example shown in FIG. 14, the associated information is displayed in association with inappropriate data (sensing data determined to be inappropriate). Here, the inappropriate data is image data, and the user can operate the cursor to select any inappropriate data. On the screen of the display device, the above-described associated information is displayed for the selected inappropriate data. In FIG. 14, the inappropriate determination result, the reason for the inappropriate determination, the corresponding action, the sensing time, the coordinates of the corresponding ground area, and substitute data are displayed as the associated information.

[0123] In the example shown in FIG. 15, the accompanying information is displayed on the screen of the display device in map form, associated with the ground sensing target area. In FIG. 15, the ground sensing range in each sensing data is indicated by a dotted line, and the accompanying information is displayed in association with two ground sensing target areas corresponding to inappropriate data. Also, here, as the accompanying information, the reason for the inappropriate determination and the corresponding action are displayed.

[0124] In the example shown in FIG. 16, the accompanying information is displayed on the screen of the display device in map form, associated with the position of the aircraft. In FIG. 16, the position of the aircraft (for example, the coordinates of the measured position) at the time of executing the sensing of each sensing data is indicated by a ○ mark, and the accompanying information is displayed in association with two positions of the aircraft corresponding to inappropriate data. Also, here, as the accompanying information, the reason for the inappropriate determination and the corresponding action are displayed. Note that the position indicated by the ○ mark in FIG. 16 may be the control target position on the control target path of the aircraft.

[0125] (A-1-6―7. User Input Reception Unit 2370) The user input reception unit 2370 receives user input with respect to the appropriateness determination result displayed by the appropriateness determination result display unit 2343 and the corresponding action determination result displayed by the corresponding action determination result display unit 2352. As described above, the user input reception unit 2370 can receive user input with respect to the displayed appropriateness determination result and determine inappropriate data. Also, the user input reception unit 2370 can receive user input with respect to the displayed corresponding action determination result and determine the corresponding action.

[0126] The user input reception unit 2370 is not limited to the above-described content, and regardless of the appropriateness determination result, can receive user input that allows the user to select sensing data determined to be inappropriate and determine the inappropriate data by the user. Similarly, it can receive user input that allows the user to input the corresponding action determined by the user and determine the corresponding action by the user.

[0127] (A-1-7. Data Processing Management Unit 3300) Based on the processed data generated by the data processing unit 3200 or the content of the data being processed, the data processing management unit 3300 determines the suitability of the processed data and the corresponding actions, generates accompanying information, and provides the processed data and the accompanying information to the distribution system. FIG. 7 is a functional configuration diagram of the data acquisition management unit according to an embodiment of the present invention. As shown in FIG. 7, the data processing management unit 3300 includes an accompanying information display unit 3310, a data processing execution command unit 3320, a data suitability determination unit 3330 for the processed data, a corresponding action determination unit 3340, an accompanying information adding unit 3350, a user input reception unit 3360, and a distribution information generation unit 3370.

[0128] (A-1-7-1. Accompanying Information Display Unit 3310) Similar to the accompanying information display unit 2363 in the data management system 2300 described above, the accompanying information display unit 3310 displays the accompanying information generated by the accompanying information generation unit 2361. However, the accompanying information display unit 3310 is different from the accompanying information display unit 2363 in that it displays the accompanying information to the user of the data processing management unit 3300 (i.e., the user in charge of data processing), but the content of the displayed accompanying information may be the same.

[0129] Here, when the corresponding action included in the accompanying information is a specification of a data processing method, such as "image processing for brightness adjustment", "position correction processing of the ground surface imaging area according to the position error", "position correction processing of the ground surface imaging area according to the angle error", etc., the command of the data processing method is notified to the data processing person by the accompanying information display unit 3310. Therefore, the data processor can control the data processing unit 3200 according to the corresponding action.

[0130] In addition, the accompanying information display unit 3310 displays accompanying information including the appropriateness determination results and corresponding actions for the in - process data and post - process data generated by the accompanying information adding unit 3350 described later. With this configuration, the data processing personnel can take actions such as not using inappropriate data and performing re - sensing according to the corresponding actions.

[0131] (A - 1 - 7 - 2. Data Processing Execution Command Unit 3320) The data processing execution command unit 3320 determines the content of the data processing to be executed by the data processing unit 3200 and sends an execution command for the data processing to the data processing unit 3200. FIG. 17 is a table showing an example of the data processing content in which the data processing execution command unit of the present invention commands the data processing unit. The data processing execution command unit 3320 can generate execution commands for various data processings as shown in the table T105 of FIG. 17. Also, the content of the data processing for which the execution command is issued may be set in advance, or may be set by selecting a specific data processing from the data processing list shown in the table T105.

[0132] The content of each data processing shown in the table T105 is as follows. · Identification No. 001: A process of acquiring a plurality of optical images as acquisition data, performing ortho - image conversion on the acquired plurality of optical images, and generating an ortho - image by integrating the plurality of ortho - images. · Identification No. 002: A process of acquiring a plurality of optical images as acquisition data and generating three - dimensional space data from the plurality of optical images using SfM (Structure from Motion) processing. · Identification No. 003: A process of acquiring point - cloud data as acquisition data and generating three - dimensional space data based on DSM (Digital Surface Model) or DEM (Digital Elevation Model) from the point - cloud data. · Identification No. 004: A process of performing the ortho - image generation process shown in No. 001, acquiring geographical data as processing data, and generating map integration data by integrating the ortho - image and the geographical data. · Identification No. 005: A process of performing the map integration image generation process shown in No. 004, obtaining a past map integration image as processing data, and performing a difference analysis between the current and past map integration images. · Identification No. 006: A process of performing the ortho image generation process shown in No. 001, obtaining disaster information as processing data, and generating a disaster information integration image by integrating the ortho image and the disaster information. · Identification No. 007: A process of performing the ortho image generation process shown in No. 001, obtaining on-site team information as processing data, and generating an on-site team information integration image by integrating the ortho image and the on-site team information.

[0133] (A-1-7-3. Data Appropriateness Judgment Unit 3330) The data appropriateness judgment unit 3330 performs a data appropriateness judgment or displays a judgment result based on the content of the in-process data generated during the data processing by the data processing unit 3200 or the post-process data generated when the data processing is completed. The data appropriateness judgment unit 3330 includes a data analysis unit 3331 and an appropriateness judgment result display unit 3332.

[0134] The data analysis unit 3331 analyzes the content of the in-process data and post-process data obtained from the data processing unit 3200, and determines inappropriate data based on the judgment criteria regarding the content of the in-process data and post-process data. The judgment criteria determined to be inappropriate in this data analysis unit 3331 can be set as follows, for example, for each of the above-described data processing contents.

[0135] The judgment criteria for inappropriate judgment in data processing identification Nos. 001, 004, 005, and 006 for performing ortho image generation include, for example, when lens distortion of a predetermined value or more is detected from the image data, when image blur or focus deviation of the image data is detected, when partial acquisition omission is detected in the captured image, when an image at an unnecessary position (such as the turning path of an aircraft) is detected, when insufficient light intensity or white dropout of the image data is detected, and the like.

[0136] The criteria for inappropriate determination in data processing identification No. 002 that performs SfM processing using multiple images can be, for example, when the object for which three-dimensional information is to be generated has not moved into the image data, or when the feature points of the object cannot be obtained. Or, when lens distortion of a predetermined value or more is detected, when ground reference points (such as GCP (Ground control point), CP (Check point), etc.) cannot be photographed from the image data, when doming of a predetermined value or more is detected, when a partial acquisition omission is detected in the photographed image, when an image at an unnecessary position (such as the turning path of an aircraft) is detected, when insufficient light intensity or white blooming of the image data is detected, etc. can be included. Also, the criteria for inappropriate determination are not limited to those described above, and criteria for each processing step of SfM processing can be provided.

[0137] The criteria for inappropriate determination in data processing identification No. 003 that generates three-dimensional space data from the measurement point cloud can be, for example, when the laser does not reach the ground due to terrain such as cliffs, or when an area where point cloud data cannot be acquired is detected due to a water surface, fog, or other objects that absorb the laser. Also, it can include cases where a partial acquisition omission is detected in the point cloud data, or when feature points cannot be detected from the point cloud data.

[0138] The criteria for determination as inappropriate in this data analysis unit 3331 may include the following criteria in addition to the above. · When the data processing content cannot detect a pair of images that can be used as stereo images for measuring the ground distance included in the image data. · When processing both image data and point cloud data, when the point cloud data is not obtained and the presence of clouds or fog is detected by the image.

[0139] The inappropriate determination of the in-process data and post-process data in the above-described data analysis unit 3331 may be determined by comparing the state quantity and the reference value of the in-process data and post-process data as described above. However, in addition, it may also be determined by comparing with the state quantity of past post-process data or comparing with a target value.

[0140] Next, the appropriateness determination result display unit 3332 displays the result of the above-described inappropriate determination. Specifically, it can be the same display as the result display of the inappropriate determination displayed on the appropriateness determination result display unit 2343 of the data management system 2300 shown in FIG. 12. That is, for the in-process data or post-process data subject to the appropriateness determination, the inappropriate data determined to be inappropriate is displayed in an identifiable manner. The user of the data processing management unit 3300 can select any inappropriate data displayed by the user input reception unit 3360 described later.

[0141] (A-1-7-4. Corresponding Action Determination Unit 3340) The corresponding action determination unit 3340 determines a corresponding action or displays candidates for corresponding actions based on the content of the in-process data or post-process data acquired from the data processing unit 3200, or the result of a combination thereof. The corresponding action determination unit 3340 includes a determination processing unit 3341 and a corresponding action determination result display unit 3342. The determination processing unit 3341 determines the corresponding action to be taken particularly when the data is determined to be inappropriate by the data appropriateness determination unit 3330.

[0142] The corresponding action determination unit 3340 determines a corresponding action based on the state quantities regarding the data during processing and the data after processing, and the determination result of the inappropriate determination by the data appropriateness determination unit 3330. As the corresponding action determined by the determination processing unit 3341, similar to the determination processing unit 2351 of the data management system 2300, for example, non-use of inappropriate data determined to be inappropriate, substitution of other data instead of inappropriate data, notification of restrictions on the usage purpose of the data after processing, notification of including inappropriate data, specification of the data processing method, re-execution of sensing by an aircraft, specification of the data processing method according to the usage purpose designated by the user, specification of the display method of the data after processing according to the usage purpose designated by the user, and the like are included.

[0143] Here, in the case of performing data processing identification No. 002 by SfM processing using a plurality of images, when doming equal to or greater than a predetermined value is detected by inappropriate determination, the corresponding action determination unit 3340 determines, as corresponding actions, photographing of the oblique image of the target area by "re-execution of sensing by an aircraft" and execution of doming countermeasure processing by "specification of the data processing method".

[0144] The determination criteria of the corresponding action determination unit 3340 can be set based on the input information from the user acquired by the user requirement acquisition unit 2310, but are not limited thereto, and may be preset by the system. Alternatively, for the default setting criteria for each target state quantity preset by the system, the criteria to be applied may be selected by the user inputting in the selection column on the left side of T102, or arbitrary conditions may be set by the user inputting in the corresponding action determination column on the right side of T102.

[0145] Also, as an example of the corresponding action determination criteria, the user may input the usage purpose of the data after processing (such as ground feature inspection purpose, forest survey purpose, etc.), and the system may automatically generate the criteria and determination results for corresponding action determination according to the input usage purpose.

[0146] Next, the corresponding action determination result display unit 3342 displays the determination result of the corresponding action determined by the determination processing unit 3341. The content displayed by the corresponding action determination result display unit 3342 can be the same as the content of the corresponding action determination result display unit 2352 of the data management system 2300 as shown in FIG. 13. The user of the data processing management unit 3300 can select any inappropriate data by the user input reception unit 3360 described later, and the determination result of the corresponding action corresponding to the selected inappropriate data is displayed. Further, when any inappropriate data is selected, the determination result of the corresponding action described above and other candidates are displayed so as to be selectable, and further, a confirmation button for the corresponding action is displayed. The user can select any corresponding action by the user input reception unit 3360, and can confirm the corresponding action by operating the confirmation button for the corresponding action in a state where a specific corresponding action is selected.

[0147] (A-1-7-5. Associated Information Adding Unit 3350) The associated information adding unit 3350 generates associated information including information regarding the content of the suitability determination determined by the data suitability determination unit 3330, and records the associated information in association with inappropriate data or the like, or displays it on the display unit. The associated information adding unit 3350 includes an associated information generating unit 3351 and an information adding unit 3352. The associated information generating unit 3351 generates associated information, and records and displays the associated information in association with inappropriate data or the like.

[0148] The associated information generating unit 3351 generates associated information obtained by adding or replacing the determination results by the data suitability determination unit 3330 and the corresponding action determination unit 3340 of the data processing management unit 3300 to the associated information generated in the data management system 2300 as shown in FIG. 11. Therefore, the associated information generated by the associated information generating unit 3351 can be associated information including at least any one or all of the suitability determination result and the corresponding action determination result generated in the data management system 2300 before the execution of the data processing and the suitability determination result and the corresponding action determination result generated during or after the execution of the data processing.

[0149] The information providing unit 3352 has the same function as the above-described information providing unit 2362, and records the accompanying information generated by the accompanying information generation unit 3351 in association with inappropriate data or the like. The information providing unit 3352 associates the inappropriate data determined to be inappropriate, or the surface position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target route of the aircraft with the accompanying information, and records it in the storage device.

[0150] The accompanying information generated by the accompanying information generation unit 3351 is displayed by the above-described accompanying information display unit 3310 in association with inappropriate data or the like to the user (that is, the data processing person in charge) of the data processing management unit 3300. The accompanying information display unit 3310 associates the inappropriate data determined to be inappropriate, or the surface position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target route of the aircraft with the accompanying information, and displays the accompanying information on the display device.

[0151] In this way, by displaying the accompanying information including the corresponding actions for the data being processed and the data after processing to the data processing person in charge of the data processing management unit 3300, the data processing person in charge can take corresponding actions such as not using inappropriate data and performing resensing according to the corresponding actions.

[0152] (A-1-7-6. User Input Reception Unit 3360) The user input reception unit 3360 receives user input for the appropriateness determination result displayed by the appropriateness determination result display unit 3332 and the corresponding action determination result displayed by the corresponding action determination result display unit 3342. As described above, the user input reception unit 3360 can receive user input for the displayed appropriateness determination result and confirm inappropriate data. In addition, the user input reception unit 3360 can receive user input for the displayed corresponding action determination result and confirm the corresponding action.

[0153] The user input reception unit 3360 is not limited to the above-described content. Regardless of the appropriateness determination result, it can receive a user input that allows the user to select sensing data determined to be inappropriate by the user and confirm the inappropriate data with the user. Similarly, it can receive a user input that allows the user to input a corresponding action determined by the user and confirm the corresponding action with the user.

[0154] (A-1-7-7. Distribution information generation unit 3370) The distribution information generation unit 3370 generates distribution information in which the accompanying information generated by the accompanying information generation unit 3351 is associated with the processed data generated by the data processing unit 3200, so that the content of the accompanying information corresponding to the processed data can be notified to the user of the user terminal device that views the distribution information via the distribution system 4000.

[0155] As an example, the distribution information generation unit 3370 generates distribution information in which the accompanying information is displayed in a map format on the screen of the display device of the user terminal device in association with the inappropriate data determined to be inappropriate, or the ground position of the target position related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target route of the aircraft.

[0156] Examples of the display when the distribution information generated by the distribution information generation unit 3370 is displayed on the display screen of the user terminal device are shown in FIGS. 18 and 19. FIG. 18 is a diagram showing an example of the display when the distribution information generation unit of the present invention displays the accompanying information in association with the ground sensing target area. FIG. 19 is a diagram showing an example of the display when the distribution information generation unit of the present invention displays the accompanying information in association with the position of the aircraft.

[0157] In the display example shown in FIG. 18, an example is shown in which accompanying information is displayed in association with a ground sensing target area corresponding to inappropriate data on the screen of a display device on which post - processed data is displayed in a map format. Further, in FIG. 18, the ground range corresponding to each post - processed data is indicated by a dotted line, and the accompanying information is displayed in association with two ground ranges corresponding to inappropriate data. Also, here, as the accompanying information, the reason for the inappropriate determination, the corresponding action, etc. are displayed.

[0158] In the display example shown in FIG. 19, an example is shown in which accompanying information is displayed in association with the position of an aircraft corresponding to inappropriate data on the screen of a display device on which post - processed data is displayed in a map format. Further, in FIG. 19, the position of the aircraft (for example, the coordinates of the measured position) at the time corresponding to each post - processed data is indicated by a ○ mark, and the accompanying information is displayed in association with two positions of the aircraft corresponding to inappropriate data. Also, here, as the accompanying information, the reason for the inappropriate determination, the corresponding action, etc. are displayed. Note that the position indicated by the ○ mark in FIG. 19 may be the control target position on the control target path of the aircraft.

[0159] In the examples shown in FIGS. 18 and 19, for example, an example is shown in which the reason for the inappropriate determination is "out of focus", and the corresponding actions "notification of inappropriate determination information to the user" and "not available for use in human determination applications" are displayed. Therefore, a user who views this distribution information can grasp that the inappropriate data has been processed based on sensing data with out - of - focus, and can also grasp that it is not available for use in applications for determining the presence or absence of a person due to the influence of out - of - focus.

[0160] In this way, by displaying the post - processed data or the associated information in association with the post - processed data or information such as the corresponding ground position or aircraft position to the browsing user who browses the post - processed data, it is possible to correctly recognize the locations where the data determined to be photographing failures are being used and those where they are not, as well as the reasons why inappropriate data is determined to be inappropriate. Furthermore, since it is possible to grasp the scope of applications for which the post - processed data can be used, etc., it is possible to correctly grasp the situation of the target area based on the post - processed data. Also, since it is possible to grasp future response actions such as whether inappropriate data is scheduled to be updated by re - measurement, if the user who has confirmed the post - processed data is a member of the crisis management headquarters, etc., it is possible to more appropriately judge the plan for future crisis management operations.

[0161] <Hardware Configuration> FIG. 20 is a diagram showing an example of the hardware configuration of the data management system 2300 and the data processing management unit 3300. Here, the data management system 2300 and the data processing management unit 3300 in the present invention are information processing devices such as a server device or a PC. As shown in the figure, the data management system 2300 and the data processing management unit 3300 include an input device 100, an output device 200, a processing device 300, a main storage device 400, an auxiliary storage device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0162] The input device 100 is a device for a user to input information and instructions into the data management system 2300 and the data processing management unit 3300. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or a voice input device such as a microphone.

[0163] The output device 200 is a device that outputs the information generated by the data management system 2300 and the data processing management unit 3300. Specifically, the output device 200 is a display device (including display devices for eyewear, AR, VR, etc.), a printer, or a speaker.

[0164] The processing device 300 is a device that performs arithmetic processing, for example. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing arithmetic operations, etc.

[0165] The main memory device 400 is a memory device such as a RAM that temporarily stores various read information, a ROM that stores programs and application programs executed by the processing device 300, and various other information. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that can store digital information.

[0166] The communication device 600 is a device that performs information communication wirelessly or by wire with an external device.

[0167] In FIG. 20, the hardware configurations of the data management system 2300 and the data processing management unit 3300 were described. However, in other systems within the data acquisition base system 2000 (communication infrastructure management system 2100, airframe operation operating system 2200, operation management system 2500, airspace monitoring system 2600), other systems in the data processing base system 3000 (communication unit 3100, data processing unit 3200), and the distribution system, as well as user terminal devices (including the crisis management headquarters terminal), the same hardware configuration as that in FIG. 20 described above can be realized.

[0168] <Control Flow> (A-1-8. Upper-level Control Flow of the Aviation Data Processing System) Next, the control flow of the aviation data processing system will be described with reference to the drawings. FIG. 21 is a flowchart showing the processing flow by the aviation data processing system 1.

[0169] First, the user requirement acquisition unit 2310 of the data management system 2300 acquires user requirement information (step 101). Note that this step may be acquired in advance before starting the sensing operation by the aircraft, or the user requirements may be acquired during the execution of the sensing operation.

[0170] Next, the sensing data acquisition unit 2320 and the state information acquisition unit 2330 of the data management system 2300 acquire sensing data and state information from the aircraft (step 102). As an example of the state information to be acquired, state information as shown in the table T103 of FIG. 10 is acquired.

[0171] Next, the data appropriateness determination unit 2340 of the data management system 2300 performs an inappropriate determination of the data based on the acquired information such as the sensing data and the state information (step 103). Here, after the aircraft finishes the flight for the sensing operation, within the data acquisition base system 2000, the data appropriateness determination in this step may be performed based on the data received via wireless communication or read from the sensing data recording unit 1500 and the state data recording unit 1600 of the aircraft. However, each functional unit provided in the data acquisition base system 2000 can be implemented on the aircraft, and the aircraft can perform the data appropriateness determination in this step during the flight for the sensing operation.

[0172] Next, it is determined whether there is inappropriate data determined to be inappropriate as a result of the data appropriateness determination by the data appropriateness determination unit 2340, and a step to transition according to the determination result is determined (step 104). If there is inappropriate data, the process transitions to step 105, and if there is no inappropriate data, the process transitions to step 108.

[0173] Next, the corresponding action determination unit 2350 of the data management system 2300 determines a corresponding action when it is determined that the data is inappropriate (step 105). Here, this step may be performed within the data acquisition base system 2000 after the aircraft has completed a flight for sensing operations. However, each functional unit provided in the data acquisition base system 2000 may be implemented on the aircraft, and the corresponding action determination according to this step may also be performed during the flight of the aircraft for sensing operations.

[0174] Next, the attached information adding unit 2360 of the data management system 2300 generates attached information including information regarding the appropriateness determination and information regarding the content of the corresponding action determination, and records the attached information in association with inappropriate data or the like (step 106).

[0175] Next, the attached information adding unit 2360 of the data management system 2300 displays the attached information in association with inappropriate data or the like to the user of the data management system 2300 (that is, the person in charge of data sensing). Further, the attached information display unit 3310 of the data processing management unit 3300 displays the attached information in association with inappropriate data or the like to the user of the data processing management unit 3300 (that is, the person in charge of data processing) (step 107).

[0176] Next, the data processing execution command unit 3320 of the data processing management unit 3300 commands the execution of data processing to the data processing unit 3200 (step 108).

[0177] Next, the data appropriateness determination unit 3330 of the data processing management unit 3300 performs a data appropriateness determination on the data being processed or the processed data (step 109).

[0178] Next, based on the result of data validity determination by the data validity determination unit 3330 of the data processing management unit 3300, it is determined whether there is inappropriate data determined to be data-inappropriate, and a step of transitioning according to the determination result is determined (step 110). If there is inappropriate data, transition to step 111, and if there is no inappropriate data, end the processing of this flowchart.

[0179] Next, the corresponding action determination unit 3340 of the data processing management unit 3300 determines the corresponding action when it is determined that the data is inappropriate (step 111).

[0180] Next, the attached information adding unit 3350 of the data processing management unit 3300 generates attached information including information related to the validity determination and information related to the content of the corresponding action determination, and records the attached information in association with inappropriate data and the like (step 112).

[0181] Next, the attached information display unit 3310 of the data processing management unit 3300 displays the attached information in association with inappropriate data and the like to the user of the data processing management unit 3300 (that is, the person in charge of data processing) (step 113).

[0182] Next, the distribution information generation unit 3370 of the data processing management unit 3300 generates distribution information including the attached information and the processed data (step 114). Further, the distribution information generated in this step is distributed to the user terminal device 5000 by the distribution system 4000.

[0183] (A-1-9. Processing Flow of Data Validity Determination) FIG. 22 is a flowchart showing a processing flow performed when the data validity determination unit 2340 in the data management system of the present invention performs data validity determination.

[0184] First, the data analysis unit 2341 determines whether there is any data loss in the sensing data acquired from the sensing data acquisition unit 2320 (step 201). For example, when the sensing operation is not performed and sensing data cannot be obtained, when the number of sensing data is clearly small, or when the ground image cannot be captured with the lens cover closed as shown in FIG. 8, an inappropriate determination due to data loss can be made.

[0185] Next, the data analysis unit 2341 makes an appropriateness determination based on the content of the sensing data (step 202). For example, as shown in FIG. 8, when there are water droplets, condensation, or dirt on the lens during camera image acquisition, or when there is a defective area in the camera image where the ground cannot be photographed due to clouds, fog, etc., an inappropriate determination based on the sensing data can be made.

[0186] Next, the state information analysis unit 2342 makes an appropriateness determination based on the state of the aircraft (step 203). For example, as shown in FIG. 8, an appropriateness determination can be made based on state quantities such as the position, flight speed, flight altitude, and attitude of the aircraft.

[0187] Next, the state information analysis unit 2342 makes an appropriateness determination based on the state of the sensor (step 204). For example, as shown in FIG. 8, an appropriateness determination can be made based on sensor setting values such as the resolution, focus, and brightness of the camera.

[0188] Next, it is determined whether there is an inappropriate determination as a result of the appropriateness determinations performed in steps 201 to 204, and a process to be transitioned according to the determination result is determined (step 205). If there is an inappropriate determination, the process transitions to step 206, and if there is no inappropriate determination, the process transitions to step 208.

[0189] Next, the appropriateness determination result display unit 2343 displays the inappropriate data for which an inappropriate determination has been made and the content of the inappropriate determination (step 206). For example, as shown in FIG. 12, inappropriate data, the reason for the inappropriate determination for the inappropriate data, the degree of exceeding the reference value, the sensing time, the position of the target ground area, etc. are displayed.

[0190] Next, the user input reception unit 2370 receives a confirmation input from the user for the displayed inappropriate determination result (step 207). That is, by operating the confirmation button for the inappropriate data as shown in FIG. 12, the inappropriate determination result is confirmed for the sensing data determined to be inappropriate data by the inappropriate determination and its determination content.

[0191] Next, the display device displays the sensing data including the sensing data that has not been determined to be inappropriate in the data appropriateness determination to the user regardless of the above appropriateness determination result (step 208).

[0192] Next, the user input reception unit 2370 receives a selection input of the sensing data to be determined to be inappropriate from the user (step 209). By steps 208 and 209, even for the sensing data that has not been determined to be inappropriate by the data appropriateness determination unit 2340, the user can manually make an inappropriate determination and input it to the system.

[0193] Next, based on the input information acquired from the user in steps 207 and 209, the data appropriateness determination unit 2340 determines the inappropriate data and the content of the inappropriate determination, and records the determined inappropriate data and its inappropriate determination content (step 210).

[0194] (A-1-10. Processing Flow of Corresponding Action Determination) FIG. 23 is a flowchart showing the processing flow performed when the corresponding action determination unit 2350 of the present invention determines the corresponding action.

[0195] First, the determination processing unit 2351 performs an action determination based on the data content of the sensing data acquired from the sensing data acquisition unit 2320 (step 301). For example, as shown in FIG. 9, when there is data loss, or there is a defective area in the camera image where the ground cannot be photographed due to water droplets, condensation, dirt on the lens at the time of camera image acquisition, or clouds or fog, etc., corresponding actions according to each state can be determined.

[0196] Next, the determination processing unit 2351 performs a corresponding action determination based on the state of the aircraft (step 302). For example, as shown in FIG. 9, a corresponding action determination can be performed according to state quantities such as the position, flight speed, flight altitude, and attitude of the aircraft.

[0197] Next, the determination processing unit 2351 performs a corresponding action determination based on the state of the sensor (step 303). For example, as shown in FIG. 9, a corresponding action determination can be performed according to sensor setting values such as the resolution, focus, and luminance of the camera.

[0198] Next, the corresponding action determination result display unit 2352 displays the results of the corresponding action determinations performed in steps 301 to 303 (step 304). For example, inappropriate data, reasons for inappropriate determination, corresponding actions for the inappropriate data, etc., as shown in FIG. 13, are displayed.

[0199] Next, the user input reception unit 2370 receives a confirmation input from the user for the displayed corresponding action determination result (step 305). That is, by operating the confirmation button for the corresponding action as shown in FIG. 13, a confirmation input for the corresponding action determined by the corresponding action determination is received. Alternatively, an input of another corresponding action different from the corresponding action determination result is received from the user.

[0200] Next, the corresponding action determination unit 2350 determines the corresponding action based on the input information acquired from the user in step 305, and records the determined corresponding action (step 306).

[0201] In addition, in FIG. 23, the processing flow of the corresponding action determination unit 2350 of the data management system 2300 has been described. However, in the processing flow of the corresponding action determination unit 3340 of the data processing management unit 3300, the steps 302 and 303 in FIG. 23 can be omitted.

[0202] (A-1-11. Processing Flow of Appropriateness Judgment in Data Processing Management Unit 3300) FIG. 24 is a flowchart showing the processing flow performed when the data appropriateness determination unit 3330 in the data processing management unit of the present invention performs data appropriateness determination.

[0203] First, the data analysis unit 3331 performs an appropriateness determination based on the data content of the in-process data obtained from the data processing unit 3200 (step 401).

[0204] Next, the data analysis unit 3331 performs an appropriateness determination based on the data content of the post-process data obtained from the data processing unit 3200 (step 402).

[0205] Next, the data appropriateness determination unit 3330 determines whether there is an inappropriate determination as a result of the appropriateness determinations performed in steps 401 and 402, and determines the process to be transitioned according to the determination result (step 403). If there is an inappropriate determination, it transitions to step 404, and if there is no inappropriate determination, it transitions to step 406.

[0206] Next, the appropriateness determination result display unit 3332 displays the inappropriate data that has been determined to be inappropriate and the content of the inappropriate determination (step 404). For example, as shown in FIG. 12, inappropriate data, the reason for the inappropriate determination for the inappropriate data, the degree of exceeding the reference value, the sensing time, the position of the target ground area, etc. are displayed.

[0207] Next, the user input reception unit 3360 receives a confirmation input from the user for the displayed inappropriate determination result (step 405). That is, by operating the confirmation button for inappropriate data as shown in FIG. 12, the in-process data or post-process data determined to be inappropriate data by the inappropriate determination and the determination content thereof are confirmed for the inappropriate determination result.

[0208] Next, the display device displays data including in-process data and post-process data that have not been determined to be inappropriate in the data appropriateness determination for the user regardless of the above-described appropriateness determination result (step 406).

[0209] Next, the user input reception unit 3360 receives a selection input of in-process data or post-process data to be determined as inappropriate from the user (step 407). Even for data that has not been determined to be inappropriate in steps 401 and 402, the user can manually perform an inappropriate determination and input it into the system.

[0210] Next, based on the input information acquired from the user in step 407, the data appropriateness determination unit 3330 determines the inappropriate data and the content of the inappropriate determination, and records the determined inappropriate data and the content of the inappropriate determination thereof (step 408).

[0211] (A-1-12. Processing Flow of Appropriateness Determination when Executing SfM Processing in Data Processing Unit 3200) FIG. 25 is a flowchart showing a processing flow when performing data processing and data appropriateness determination when the data processing base system of the present invention executes SfM processing. Using this figure, each processing step when executing SfM processing in the data processing unit 3200 and an example in which the data appropriateness determination unit 3330 performs an appropriateness determination based on a plurality of in-process data will be described.

[0212] First, the data processing unit 3200 detects feature points from the image and estimates the three-dimensional distribution of the feature points (step 501).

[0213] Next, the data validity determination unit 3330 determines whether or not feature points could be extracted in step 501, and determines the process to be transitioned based on the determination result (step 502). If the feature points could be extracted, the process transitions to step 503. If the feature points could not be extracted, the process transitions to step 512, and it is determined that the data is inappropriate.

[0214] Next, the data processing unit 3200 determines a typo point, which is a point that can be grasped from both of the plurality of images from the image feature points (step 503).

[0215] Next, the data processing unit 3200 estimates the position and orientation during shooting by an aircraft as external calibration elements (step 504).

[0216] Next, the data processing unit 3200 estimates the distortion of the lens as internal calibration elements (step 505).

[0217] Next, the data validity determination unit 3330 determines whether or not the parameters (aperture (f-number), focus amount, etc.) used for estimating the distortion of the lens in step 505 have changed, and determines the process to be transitioned based on the determination result (step 506). If the parameters have not changed, the process transitions to step 507. If the parameters have changed, the process transitions to step 512, and it is determined that the data is inappropriate.

[0218] Next, the data processing unit 3200 detects ground control points (GCPs) and accuracy verification points (CPs) (step 507).

[0219] Next, the data validity determination unit 3330 determines whether the ground reference points (GCPs) and accuracy verification points (CPs) could be detected in the processing step of step 507, and determines the process to be transitioned based on the determination result (step 508). If the ground reference points (GCPs) and accuracy verification points (CPs) could be detected, the process transitions to step 509. If the ground reference points (GCPs) and accuracy verification points (CPs) could not be detected, the process transitions to step 512, and it is determined that the data is inappropriate.

[0220] Next, the data processing unit 3200 performs dorming countermeasure processing (step 509). The details of the dorming countermeasure processing will be described later.

[0221] Next, the data processing unit 3200 constructs a point cloud model from the image feature points and a plurality of stereo pair images (step 510).

[0222] Next, the data validity determination unit 3330 determines that the data is appropriate (step 510).

[0223] (A-1-13. Explanation of Dorming Countermeasure Processing) The concept of the dorming countermeasure processing executed in step 509 of the flowchart shown in FIG. 25 will be briefly explained with reference to FIG. 26. FIG. 26 is a diagram showing an example of the dorming countermeasure processing executed by the data processing unit of the present invention.

[0224] As shown in the upper diagram of FIG. 26, in this embodiment, dorming refers to the occurrence of an error such that the shape of the ground surface estimated by the SfM process is more curved than the actual ground surface shape when performing SfM processing using a plurality of vertically downward images taken from above the ground. Such dorming occurs when performing SfM processing using only vertically downward images or when there is an error in the internal calibration elements.

[0225] As a dorming countermeasure process that takes into account such dorming, by performing SfM processing using a plurality of images including an image facing diagonally downward in addition to the vertically downward image, as shown in the lower diagram of FIG. 26, it is possible to reduce the estimation error of the ground surface shape due to dorming.

[0226] In each of the above-described embodiments, the form of implementing the data management system 2300 within the data acquisition base system 2000 has been described. However, all or part of the functions of the data management system 2300 can also be implemented in the aircraft 1000, the data processing base system 3000, the distribution system 4000, or other systems.

[0227] The above-described embodiments are merely examples for facilitating the understanding of the present invention and are not for limiting the interpretation of the present invention. It goes without saying that the present invention can be changed and improved without departing from its gist, and equivalents of the present invention are included therein.

[0228] [A-2. Effects of this Embodiment] In the above-described embodiment, particularly when the data suitability determination unit determines that the sensing data is inappropriate, in order to display the accompanying information including the content of the suitability determination, the reason for the suitability determination, the corresponding action, etc. to the user (data processing person) of the data processing management unit, the data processing person does not immediately determine not to use the inappropriate data, but can perform data processing such as data processing that allows the use of even inappropriate data, and it becomes possible to effectively use the inappropriate data.

[0229] In addition, by displaying the post - processing data, or the associated information in association with the information such as the post - processing data, or the corresponding ground position and aircraft position, etc., to the viewing users such as the crisis management headquarters that views the post - processing data, it is possible to correctly recognize the locations where the data determined to be a shooting failure is used and those where it is not, as well as the reasons why the inappropriate data is determined to be inappropriate. Furthermore, since it is possible to grasp the scope of applications for which the post - processing data can be used, etc., it is possible to correctly grasp the situation of the target area based on the post - processing data. Also, since it is possible to grasp the future response actions such as whether the inappropriate data is scheduled to be updated by re - measurement, etc., when the user who has confirmed the post - processing data is a member of the crisis management headquarters, etc., it is possible to more appropriately judge the plan for future crisis management operations.

[0230] In addition, for the users of the data management system 2300, for example, the person in charge of the sensing operation using the aircraft, in order to display the accompanying information including the content of the determination of the suitability of the sensing data, the reason for the determination of the suitability, the response action, etc., the person in charge of the sensing operation can accurately grasp whether it is necessary to re - acquire the data for the data determined to be a shooting failure. When it is necessary to re - perform the sensing, it is possible to quickly re - execute the sensing process, and it is possible to quickly provide the post - processing data based on the re - sensing to the viewing users.

Description of Reference Signs

[0231] 1…Aviation data processing system 100…Input device 200…Output device 300…Processing device 400…Main memory device 500…Auxiliary storage device 600…Communication device 700…Bus 1000…Aircraft 1100…Flight unit 1110…Self - position and speed determination unit 1120…Attitude determination unit 1130…Flight control unit 1200…Sensing unit 1210…Sensor 1220…Sensor attitude control unit 1230…Sensor control unit 1300…Communication unit 1310…Control Communication Unit 1320…Data Communication Unit 1330…Status Data Communication Unit 1400…Status Judgment Unit 1410…Flight Status Judgment Unit 1420…Sensing Status Judgment Unit 1430…Communication Status Judgment Unit 1500…Sensing Data Recording Unit 1600…Status Data Recording Unit 1610…Flight Control Information Recording Unit 1620…Measurement Control Information Recording Unit 1630…Status Judgment Result Recording Unit 2000…Data Acquisition Base System 2100…Communication Infrastructure Management System 2200…Aircraft Operation Control System 2300…Data Management System 2310…User Requirement Acquisition Unit 2320…Sensing Data Acquisition Unit 2330…Status Information Acquisition Unit 2340, 3330…Data Validity Judgment Unit 2341, 3331…Data Analysis Unit 2342…Status Information Analysis Unit 2343, 3332…Validity Judgment Result Display Unit 2350, 3340…Corresponding Action Judgment Unit 2351, 3341…Judgment Processing Unit 2352, 3342…Corresponding Action Judgment Result Display Unit 2360, 3350…Accompanying Information Adding Unit 2361, 3351…Accompanying Information Generation Unit 2362, 3352…Information Adding Unit 2363, 3310…Accompanying Information Display Unit 2370, 3360…User Input Reception Unit 2500…Operation Management System 2600…Airspace Monitoring System 3000…Data Processing Base System 3100…Communication Unit 3200…Data Processing Unit 3300…Data Processing Management Unit 3320…Data Processing Execution Command Unit 3370…Distribution Information Generation Unit 4000…Distribution System 4100…Communication Unit 4200…Distribution Data Management Unit 5000…User Terminal Device

Claims

1. A data acquisition unit that acquires at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on an aircraft, in-process data generated during the data processing of the sensing data, and post-processed data generated by the data processing of the sensing data; A data suitability determination unit that determines the suitability of the determination target data acquired by the data acquisition unit; An accompanying information generation unit that generates accompanying information including information regarding the content of the suitability determination when it is determined to be inappropriate by the data suitability determination unit, and A control system that associates inappropriate data determined to be inappropriate, or the ground position of the target area related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target path of the aircraft with the accompanying information, and records it in a recording unit or displays it on a display unit.

2. The control system according to claim 1, wherein the accompanying information includes information regarding the reason determined to be inappropriate by the suitability determination.

3. The control system according to claim 1, on at least any one of a map format, a three-dimensional model, and a three-dimensional space screen, the accompanying information is displayed in association with the inappropriate data, or the ground position of the target status related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target path of the aircraft.

4. The control system according to claim 1, wherein the accompanying information includes information on the degree to which the inappropriate data exceeds the reference value of the suitability determination when it is determined to be inappropriate by the suitability determination.

5. The control system according to claim 1, The accompanying information includes at least any one of the sensing execution date and time related to the acquisition of the sensing data, the aircraft identification information, the sensor identification information, the aircraft control information, the sensor control information, and the status information of the sensing data, and is a control system.

6. In the control system according to claim 1, The data suitability determination unit performs a suitability determination on the determination target data or displays candidates for inappropriate data by using a preset determination condition or artificial intelligence, and is a control system.

7. In the control system according to claim 1, The data suitability determination unit performs a suitability determination on the determination target data based on the suitability determination input information for the determination target data received from the user, and is a control system.

8. In the control system according to claim 6, When the data suitability determination unit performs a suitability determination on the sensing data, the determination condition includes at least any one of the state of the sensing data, the state of the aircraft, and the state of the sensor, Or, when the data suitability determination unit performs a suitability determination on the processed data, the determination condition includes the state of the processed data, and is a control system.

9. In the control system according to claim 8, When the data suitability determination unit performs a suitability determination on the sensing data and the determination condition includes the state of the aircraft, the determination condition is set by comparing with a reference value of a state quantity related to the aircraft, or comparing with a past performance value, or comparing with a control target value of a control target state quantity related to the aircraft, and is a control system.

10. In the control system according to claim 8, When the data suitability determination unit determines the suitability of the sensing data, and when the determination condition includes the state of the sensing data, the determination condition is set by comparing a state quantity related to the sensing data with a reference value, or comparing with a state quantity of past sensing data, or comparing with a target value, or the acquisition rate of the sensing data with respect to a target area to be sensed, for a control system.

11. In the control system according to claim 8, When the data suitability determination unit determines the suitability of the in-process data or the post-process data, the determination criterion is set by comparing a state quantity related to the in-process data or the post-process data with a reference value, or comparing with a state quantity of past post-process data, or comparing with a target value, for a control system.

12. In the control system according to claim 1, A corresponding action determination unit that determines a corresponding action when the data suitability determination unit determines that the data to be determined is inappropriate is provided, A control system that executes the corresponding action.

13. In the control system according to claim 12, The corresponding action determination unit determines the corresponding action according to at least any one of the content of the data to be determined, the state of the aircraft, the state of the sensor, and the result of the suitability determination, for a control system.

14. In the control system according to claim 12, The corresponding action determination unit determines the corresponding action according to the designated input information of the corresponding action received from the user, for a control system.

15. In the control system according to claim 13 or 14, The corresponding action includes at least one of the following: not using the inappropriate data, substituting other data for the inappropriate data, notifying of restrictions on the use of the processed data, notifying that the inappropriate data is included, specifying a data processing method, re-performing sensing by a flying object, specifying a data processing method according to a user-specified use, and specifying a method for displaying the processed data according to a user-specified use.

16. In the control system according to claim 12, the accompanying information includes information regarding the corresponding action.

17. In the control system according to claim 16, when the information on the corresponding action included in the accompanying information is non-use of the inappropriate data, the information on the target area related to the inappropriate data is included in the accompanying information.

18. In the control system according to claim 16, when the information on the corresponding action included in the accompanying information is substitution of other data for the inappropriate data, the information on the other data to be substituted is included in the accompanying information.

19. In the control system according to claim 16, when the information on the corresponding action included in the accompanying information is notification of restrictions on the use of the processed data, the information on the restrictions on the use is included in the accompanying information.

20. In the control system according to claim 16, when the information on the corresponding action included in the accompanying information is specification of the data processing method, the information on the specified data processing method is included in the accompanying information.

21. In the control system according to claim 16, When the information on the corresponding action included in the accompanying information is re - execution of sensing by the aircraft, sensing re - execution schedule information including at least any one of the scheduled position, scheduled area, and scheduled time of sensing re - execution is included in the accompanying information, control system.

22. In the control system according to claim 12, Receive from the user the use - ability determination conditions for determining that the inappropriate data is available by at least any one of the corresponding actions of notification of the usage restriction of the processed data, notification of including the inappropriate data, and designation of the data processing method, and when an inappropriate determination is made by the appropriateness determination, determine the available inappropriate data according to the use - ability determination conditions, or display the available inappropriate data, control system.

23. In the control system according to claim 12, Receive from the user the use determination conditions for determining that the inappropriate data is available by at least any one of the corresponding actions of notification of the usage restriction of the processed data, notification of including the inappropriate data, and designation of the data processing method, and when an inappropriate determination is made by the appropriateness determination, determine the corresponding action for using the inappropriate data according to the use determination conditions, or display the candidate corresponding actions for using the inappropriate data, control system.

24. In the control system according to claim 12, When an inappropriate determination is made by the appropriateness determination, determine the corresponding action or display the corresponding action according to the determination conditions of the corresponding action received from the user, or using artificial intelligence, control system.

25. A computer, A data acquisition step of acquiring at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on a flying object, in-process data generated during the data processing of the sensing data, and post-processed data generated by the data processing of the sensing data; A data suitability determination step of performing a suitability determination on the determination target data acquired by the data acquisition unit; An accompanying information generation step of generating accompanying information including information regarding the content of the suitability determination for the inappropriate data determined to be inappropriate; A step of associating the inappropriate data, or the surface position of the target area corresponding to the inappropriate data, or the position of the flying object, or the control target position of the flying object, or the control target path of the flying object with the accompanying information, and recording it in a recording unit or displaying it on a display unit; A control method for executing.

26. On a computer, A data acquisition step of acquiring at least any one of determination target data including image or point cloud information of a target area acquired using a sensor mounted on a flying object, in-process data generated during the data processing of the sensing data, and post-processed data generated by the data processing of the sensing data; A data suitability determination step of performing a suitability determination on the determination target data acquired by the data acquisition unit; An accompanying information generation step of generating accompanying information including information regarding the content of the suitability determination for the inappropriate data determined to be inappropriate; A step of associating the inappropriate data, or the surface position of the target area corresponding to the inappropriate data, or the position of the flying object, or the control target position of the flying object, or the control target path of the flying object with the accompanying information, and recording it in a recording unit or displaying it on a display unit; A control program for causing the above to be executed.

Images