Control system, control method and control program

Abstract

To provide a system that is able to quickly provide a user with a measurement result using a flying object.SOLUTION: The present invention pertains to a control system including: a data acquisition unit configured to acquire determination target data such as sensing data including an image or point group information of a target area acquired using a sensor mounted on a flying object; a data suitability determination unit configured to perform suitability determination on the determination target data acquired by the data acquisition unit; and a remeasurement condition determination unit configured to determine a remeasurement condition for sensing data using the flying object, according to a result of the suitability determination by the data suitability determination unit. The control system further includes at least one of a command unit configured to execute remeasurement, based on the remeasurement condition determined by the remeasurement condition determination unit, a transmission unit configured to transmit the remeasurement condition to an external system, and an output unit configured to output the remeasurement condition by means of display or sound.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control system, a control method, and a control program. [Background technology]

[0002] In recent years, there has been progress in the practical application of sensing systems that use aircraft to acquire images and spatial data of target areas on the ground from the sky. When performing measurements using aircraft, it is necessary to acquire sensing data that meets pre-set conditions so that the sensing data can be used for the desired purpose. However, there are cases where it is not possible to acquire sensing data that meets the above-mentioned conditions due to factors such as weather conditions and the state of the equipment.

[0003] Patent Document 1 discloses a technique for providing an unmanned moving body that can easily take a photograph of an object again if the photograph fails to be taken. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-191523 Summary of the Invention [Problem to be solved by the invention]

[0005] Sensing data measured using an aircraft is generally huge in volume, and it takes a lot of time to find measurement failure data that does not satisfy the preset conditions from among this huge amount of sensing data. Furthermore, when remeasurement is performed to compensate for measurement failure data that does not satisfy the above conditions, it is necessary to appropriately set the flight conditions and measurement conditions for the remeasurement so that the remeasurement does not result in a measurement failure.

[0006] Patent Document 1 discloses that, if an image of an object fails to be captured, the flying means and the capturing camera are controlled so that the object is re-captured at the position where the image capture failure was detected, but does not consider resetting flight conditions and measurement conditions so as to prevent measurement failure in the remeasurement. As a result, when remeasurement is performed using the flying object, the image capture fails again, and it is not possible to quickly obtain sensing data that meets the desired conditions.

[0007] Therefore, one object of the present invention is to provide a system that can quickly provide measurement results to users by appropriately setting flight conditions and measurement conditions for remeasurement. [Means for solving the problem]

[0008] According to the present invention, a control system is obtained which comprises: a data acquisition unit that acquires data to be judged, which is at least one of sensing data including images or point cloud information of a target area acquired using a sensor mounted on an aircraft, data in process generated during data processing of the sensing data, and processed data generated by data processing of the sensing data; a data suitability judgment unit that judges the suitability of the data to be judged acquired by the data acquisition unit; and a remeasurement condition judgment unit that judges the remeasurement conditions for sensing data using the aircraft based on the results of the suitability judgment by the data suitability judgment unit, and which comprises at least one of: a command unit that executes remeasurement based on the remeasurement conditions judged by the remeasurement condition judgment unit, or a transmission unit that transmits the remeasurement conditions to an external system, or an output unit that displays or outputs the remeasurement conditions by voice. [Effects of the Invention]

[0009] According to the present invention, a system can be provided that can quickly provide users with measurement results using an aircraft. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a diagram illustrating the overall configuration of an aviation data processing system according to an embodiment of the present invention. [Figure 2] 1 is a functional configuration diagram of an aircraft according to an embodiment of the present invention. FIG. [Figure 3] 1 is a configuration diagram of a data acquisition site system according to an embodiment of the present invention. [Figure 4] 1 is a configuration diagram of a data processing base system according to an embodiment of the present invention. [Figure 5] 1 is a configuration diagram of a distribution system according to an embodiment of the present invention. [Figure 6] 1 is a functional configuration diagram of a data management system according to an embodiment of the present invention; [Figure 7] FIG. 2 is a functional configuration diagram of a data acquisition management unit according to an embodiment of the present invention. [Figure 8] 10 is a table illustrating an example of criteria for determining whether or not sensing data is appropriate according to an embodiment of the present invention. [Figure 9] FIG. 10 is a diagram illustrating an example of a response action determination criterion according to an embodiment of the present invention. [Figure 10] 10 is a table showing an example of information acquired by a state information acquisition unit according to an embodiment of the present invention. [Figure 11] 10A and 10B are diagrams showing examples of display when the suitability judgment result display unit of the present invention displays the suitability judgment result. [Figure 12] 10A and 10B are diagrams showing examples of display when a response action determination result display unit of the present invention displays a response action determination result. [Figure 13] 10 is a diagram showing a criterion for determining a remeasurement condition when a remeasurement condition determination unit of the present invention determines the remeasurement condition. FIG. [Figure 14] 10A and 10B are diagrams showing examples of display when the judgment condition display unit of the present invention displays the judgment result of the remeasurement condition. [Figure 15] 10A and 10B are diagrams showing examples of display when the response action display unit of the present invention displays response actions in association with sensing target areas on the ground. [Figure 16] 10A and 10B are diagrams showing examples of display when the response action display unit of the present invention displays a response action in association with the position of the flying object. [Figure 17] 10 is a table showing an example of data processing content instructed to the data processing unit by the data processing execution command unit of the present invention. [Figure 18] 10 is a diagram showing an example of display information generated by the processed data display information generating unit of the present invention by associating the processed data with a terrestrial sensing target area. FIG. [Figure 19] FIG. 2 is a diagram illustrating an example of the hardware configuration of a data management system and a data processing management unit according to the present invention. [Figure 20] 3 is a flowchart showing a processing flow by the control system 1. [Figure 21] 10 is a flowchart showing a processing flow when the data appropriateness determining unit 2340 determines whether or not data is appropriate for pre-processing data. [Figure 22] 10 is a flowchart showing a processing flow performed by the corresponding action determining unit 2350 when determining a corresponding action to be taken on pre-processing data. [Figure 23] 10 is a flowchart showing a processing flow performed by a remeasurement condition determination unit 2360 when determining the remeasurement condition. [Figure 24] 10 is a flowchart showing a processing flow performed when a data appropriateness determining unit 3320 in the data processing management unit determines whether data is appropriate. [Figure 25] 10 is a flowchart showing a processing flow when the data suitability determination unit 2340 determines whether data is suitable during measurement by an aircraft. [Figure 26] 10 is a flowchart showing a processing flow for performing data processing and determining whether the data is appropriate when the data processing base system executes FM processing. [Figure 27] FIG. 10 is a diagram illustrating an example of a doming countermeasure process executed by a data processing unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] The present invention will be described below by listing the contents of the embodiments. The present invention has the following configuration. [Item 1] a data acquisition unit that acquires data to be determined, which includes sensing data including images or point cloud information of a target area acquired using a sensor mounted on an aircraft, data in process generated in the process of data processing of the sensing data, and processed data generated by data processing of the sensing data; a data suitability determination unit that determines suitability of the judgment target data acquired by the data acquisition unit; a remeasurement condition determination unit that determines a condition for remeasurement of sensing data using the aircraft according to a result of the suitability determination by the data suitability determination unit, A control system comprising at least one of a command unit that executes remeasurement based on the remeasurement condition determined by the remeasurement condition determination unit, a transmission unit that transmits the remeasurement condition to an external system, or an output unit that displays or outputs the remeasurement condition by voice. [Item 2] In the control system according to item 1, A control system, wherein the re-measurement condition determination unit determines the re-measurement condition according to the determination result of the suitability determination unit. [Item 3] In the control system according to item 1 or 2, A control system, wherein the remeasurement conditions determined by the remeasurement condition determination unit include flight conditions relating to at least one of the flight path, flight altitude, flight speed, and flight attitude angle of the aircraft. [Item 4] In the control system according to any one of items 1 to 3, A control system in which the re-measurement conditions determined by the re-measurement condition determination unit include setting conditions regarding at least one of the exposure, exposure, gain, contrast, focus, aperture, and shutter speed of the optical camera when an optical camera is used as the sensor. [Item 5] In the control system according to any one of items 1 to 4, A control system, wherein the remeasurement conditions determined by the remeasurement condition determination unit include at least one of the type of the sensor and the type of the flying object. [Item 6] In the control system according to any one of items 1 to 5, A control system, wherein the re-measurement condition determination unit determines the re-measurement condition in accordance with a preset determination criterion in addition to the result of suitability determination by the data suitability determination unit when determining the re-measurement condition. [Item 7] In the control system according to any one of items 1 to 6, When the remeasurement condition is output from a display device or an audio output device based on the remeasurement condition determined by the remeasurement condition determination unit, a re-measurement condition input receiving unit that receives user input including changes or approvals to the output re-measurement conditions; [Item 8] In the control system according to any one of items 1 to 7, a corresponding action determination unit that determines a corresponding action according to a result of the suitability determination by the data suitability determination unit; A control system in which the command unit executes the response action, the transmission unit transmits information about the response action to the outside, or the output unit displays or outputs the response action by voice. [Item 9] In the control system according to any one of items 1 to 8, The response action determination unit determines a response action depending on the suitability determination made by the data suitability determination unit. [Item 10] In the control system according to any one of items 1 to 9, The response action determination unit determines a response action depending on the result of the suitability determination by the data suitability determination unit and the timing of execution of the suitability determination. [Item 11] In the control system according to any one of items 1 to 10, A control system in which, if the timing of executing the suitability judgment is during measurement by the aircraft, the corresponding action judgment unit judges that the corresponding action is at least one of stopping the measurement, changing the measurement parameters of the sensor during flight, or switching the sensor. [Item 12] In the control system according to any one of items 1 to 11, The command unit executes a corresponding action of changing the measurement parameters of the sensor or switching the sensor during flight, and stops measurement if the sensing data measured after the measurement parameters are changed or the sensor is switched is determined to be inappropriate by the data suitability determination unit. [Item 13] In the control system according to any one of items 1 to 12, A control system in which the response action determined by the response action determination unit includes at least one of remeasurement, denial of permission to use the sensing data, substitution of other data for the sensing data, notification to the user that the sensing data contains inappropriate data, and specification of a data processing method for the sensing data. [Item 14] In the control system according to any one of items 1 to 13, a usability determination criterion receiving unit that receives, from a user, a determination criterion for permitting use of the inappropriate data determined to be inappropriate by the data appropriateness determining unit; A control system in which, when the data suitability judgment unit judges the data to be judged to be inappropriate, the output unit displays or outputs audio of the usable inappropriate data judged based on the information received by the usability judgment criteria reception unit, or the transmission unit transmits the usable inappropriate data to the outside. [Item 15] In the control system according to any one of items 1 to 14, an inappropriate data use condition receiving unit that receives from a user a use condition for the inappropriate data that permits use of the inappropriate data determined to be inappropriate by the data appropriateness determining unit; A control system in which, when the data suitability determination unit determines that the data to be determined is inappropriate, the command unit executes the response action determined based on the information received by the inappropriate data usage conditions reception unit, or the output unit displays or outputs the action as sound, or the transmission unit transmits the action to the outside. [Item 16] In the control system according to any one of items 1 to 15, a response action judgment criterion receiving unit that receives, from a user, a judgment criterion for determining the response action according to the result of the suitability judgment by the data suitability judgment unit; A control system in which the response action determined in accordance with the judgment criteria received by the response action judgment criteria receiving unit is executed by the command unit, or is displayed or output as audio by the output unit, or is transmitted to the outside by the transmission unit. [Item 17] In the control system according to any one of items 1 to 16, A control system in which the suitability determination by the data suitability determination unit is performed automatically in accordance with suitability determination criteria, or based on information on the suitability determination result received from a user. [Item 18] In the control system according to any one of items 1 to 17, A control system in which the suitability judgment by the data suitability judgment unit is automatically made in accordance with suitability judgment criteria regarding the flight control state of the aircraft or the measurement control state by the sensor. [Item 19] In the control system according to any one of items 1 to 18, A control system in which the suitability judgment by the data suitability judgment unit is automatically made based on the results of a comparison with a control target value or past control actual value regarding the flight control state of the aircraft or the measured control state by the sensor. [Item 20] In the control system according to any one of items 1 to 19, A control system in which the suitability determination by the data suitability determination unit is automatically determined in accordance with suitability determination criteria regarding the data content of at least one of the sensing data, the data in process, and the processed data. [Item 21] In the control system according to any one of items 1 to 20, A control system in which the suitability judgment by the data suitability judgment unit is automatically made based on the reference value of a predetermined parameter related to the data content of at least one of the sensing data, the data being processed, and the processed data, or the result of comparison with the predetermined parameter of past data for the same target area. [Item 22] The computer a data acquisition step of acquiring data to be determined, which includes sensing data including an image or point cloud information of a target area acquired using a sensor mounted on an aircraft, data in process generated in the process of data processing of the sensing data, and / or post-processing data generated by data processing of the sensing data; a data suitability determination step of determining suitability of the judgment target data acquired by the data acquisition unit; a remeasurement condition determination step of determining a condition for remeasurement of sensing data using the aircraft according to the result of the suitability determination; a step of executing a remeasurement based on the determined remeasurement condition, or outputting the remeasurement condition from a display device or an audio output device; A control method for performing [Item 23] On the computer, a data acquisition command to acquire at least one of the following data to be determined: sensing data including an image or point cloud information of a target area acquired using a sensor mounted on an aircraft; data in process generated in the process of data processing of the sensing data; and post-processing data generated by data processing of the sensing data; a data suitability determination command for determining suitability of the judgment target data acquired by the data acquisition unit; A remeasurement condition determination command for determining the remeasurement conditions of the sensing data using the flying object according to the result of the suitability determination; A command to execute remeasurement based on the determined remeasurement conditions, or a command to output the remeasurement conditions from a display device or an audio output device; A control program for executing the above.

[0012] <A. First Embodiment> Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Further, the embodiments shown below are merely examples, and other known elements and alternative means can be adopted according to the use, purpose, scale, etc.

[0013] [A-1. Configuration] (A-1-1. Overview) FIG. 1 is an overall configuration diagram of a control system 1 (hereinafter also referred to as "this system 1") according to an embodiment of the present invention. As shown in FIG. 1, this system includes a flying object 1000, a data acquisition base system 2000, a data processing base system 3000, a distribution system 4000, and a satellite system 6000.

[0014] The flying object 1000 acquires acquired data obtained by sensing information on a target area from above using sensors including optical cameras, infrared cameras, and laser sensors such as LiDAR. The data acquisition base system 2000 is a base equipped with a remote control device that exchanges control information with the flying object when the flying object is an unmanned flying object, and is configured as a movable vehicle type or an immovable fixed type base. The data processing base system 3000 is a system that acquires the acquired data obtained by the flying object, performs data processing, and generates processed data. The distribution system 4000 acquires the processed data and distributes the processed data to the user terminal device 5000 via the Internet line.

[0015] The crisis management headquarters terminal, which is an example of a user terminal device, is a terminal device used by users such as the person in charge of the crisis management headquarters, and processed data can be viewed by accessing the distribution system from the crisis management headquarters terminal. In addition, the data acquisition base system 2000 and the data processing base system 3000 are provided with display devices and user input devices for users who manage or operate these systems.

[0016] (A-1-2. Aircraft 1000) 2 is a functional configuration diagram of an aircraft according to one embodiment of the present invention. Aircraft 1000 is a system constituting the aircraft, and includes a flight unit 1100 with flight capabilities, a sensing unit 1200 that performs sensing using sensors, a communication unit 1300 that communicates with a data acquisition base system via a satellite system 6000 within control system 1 or a mobile phone communication network, a status determination unit 1400 that determines each status of flight, sensing, and positioning, a sensing data recording unit 1500 that records acquired data, and a status data recording unit 1600 that records information related to flight control and measurement control, as well as status determination results.

[0017] In this specification, the term "aircraft" refers to any aircraft capable of autonomous attitude control, regardless of the power source (electric power, prime mover, etc.), the control method (wireless or wired, fully autonomous flight or partially manual flight, etc.), or whether the aircraft is manned or unmanned. Aircraft may also be referred to as an unmanned aerial vehicle (UAV), multicopter, RPAS (Remote Piloted Aircraft Systems), or UAS (Unmanned Aircraft Systems). Aircraft may be a fixed-wing aircraft with fixed wings, a multicopter with multiple propellers, or a VTOL aircraft with both fixed wings and multiple propellers. The term "aircraft" also refers to any device that flies in the sky, such as a balloon or a satellite (including a low-earth orbit satellite or a geostationary orbit satellite).

[0018] (A-1-2-1. Flight section 1100) The flight unit 1100 includes a self-position / velocity determination unit 1110 , an attitude determination unit 1120 , and a flight control unit 1130 .

[0019] The self-position / speed determination unit 1110 has a function of determining the position and speed of the flying object 1000. For example, the position (absolute position) of the aircraft is measured based on satellite signals received from artificial satellites by a satellite signal antenna mounted on the flying object using a GPS (Global Positioning System) or a GNSS (Global Navigation Satellite System). The self-position and speed can also be determined using, for example, an RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System). The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information.

[0020] Next, the attitude determination unit 1120 has a function of determining the attitude (orientation) of the flying object 1000. Specifically, the attitude of the aircraft is determined by a GPS compass consisting of a pair of satellite antennas mounted on the flying object, a geomagnetic sensor, or the like.

[0021] Flight control section 1130, in turn, includes a processing unit, also referred to as a flight controller. The processing unit may include one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to memory. The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps.

[0022] The processing unit has the function of controlling the flight state of the aircraft by controlling the output of the thrust generating device. Specifically, the processing unit controls the thrust generated from the propeller using the propeller, the motor that drives the propeller, and the motor control unit that controls the motor output, thereby adjusting the spatial arrangement, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the aircraft, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). In other words, the flight control unit 1130 controls the aircraft's flight state by causing it to perform various operations such as liftoff, forward movement, turning, and landing, and by controlling the attitude angle and flight operations of the aircraft from takeoff through flight and landing.

[0023] (A-1-2-2. Sensing unit 1200) The sensing unit 1200 includes a sensor 1210 , a sensor attitude control unit 1220 , and a sensor control unit 1230 .

[0024] The sensor 1210 is composed of, for example, an optical camera that acquires optical images, an infrared camera that acquires infrared images, a laser sensor such as LiDAR that acquires point cloud data, etc. The sensor acquires optical images, infrared images, point cloud data, etc. of the ground surface of a sensing target area such as a disaster area from the sky above the target area.

[0025] The sensor attitude control unit 1220 controls the relative angle of the sensor with respect to the airframe of the flying object. For example, it controls the relative angle of the sensor with respect to the airframe by controlling at least one of the angles around the three axes of a gimbal that is provided between the sensor and the airframe and supports the sensor.

[0026] The sensor control unit 1230 has a function of changing the timing and measurement parameters of sensing execution by the sensor 1210. For example, if the sensor is an optical camera, it controls the image acquisition timing, exposure, shutter speed, aperture (F-number), zoom amount, ISO, resolution, etc.

[0027] (A-1-2-3. Communications Department 1300) The communication unit 1300 includes a control communication unit 1310, a data communication unit 1320, and a status data communication unit 1330. The communication unit 1300 is capable of direct communication, radio wave communication via a satellite system, or radio wave communication via a communication network NW, and includes, for example, a radio wave communication module. The communication unit 1300 is capable of communication with a data acquisition site system, etc. The communication unit 1300 includes a communication function for wirelessly communicating with the data acquisition site system, etc., using, for example, Wi-Fi, 2.4 GHz, or a frequency band of 5.6 to 5.8 GHz. The communication unit 1300 also includes a wireless communication function for communicating with the data acquisition site system, etc., via a communication network NW using a communication standard such as LTE (Long Term Evolution).

[0028] The control communication unit 1310 performs communication related to flight control of the aircraft 1000 exchanged between the aircraft 1000 and the data acquisition base system 2000. For example, if the aircraft is an unmanned aerial vehicle and is remotely controlled from the data acquisition base system, the control communication unit 1310 receives flight control commands transmitted from the data acquisition base system to the aircraft. The control communication unit 1310 also transmits information related to the flight status of the aircraft, including the position, speed, and attitude of the aircraft determined by the flight unit 1100, from the aircraft to the data acquisition base system.

[0029] The data communication unit 1320 transmits the acquired data acquired by the sensing unit 1200 to a data acquisition base system, etc. The status data communication unit 1330 transmits the determination results of the flight status, sensing status, and communication status determined by the status determination unit 1400 (described later) to the data acquisition base system.

[0030] (A-1-2-4. Status determination unit 1400) The state determination unit 1400 includes a flight state determination unit 1410, a sensing state determination unit 1420, and a communication state determination unit 1430.

[0031] The flight state determination unit 1410 determines that a flight abnormality has occurred when the position or speed of the flying object determined by the self-position / speed determination unit 1110, or the attitude of the flying object determined by the attitude determination unit 1120, deviates from a preset normal range. In addition, even if the position, speed, or attitude does not deviate from the normal range, the flight state determination unit 1410 also determines that a flight abnormality has occurred when at least one of the position, speed, or attitude does not normally follow the control command value from the flight control unit 1130.

[0032] The sensing state determination unit 1420 determines that a sensing abnormality has occurred when no data has been acquired by the sensor 1210 of the sensing unit 120 or when an abnormality is found in the acquired data. Alternatively, a sensing abnormality may be determined when the sensor attitude control unit 1220 does not operate correctly in response to a control command value.

[0033] The communication status determination unit 1430 determines the communication status between the aircraft and an external device (such as a data acquisition base system or a satellite system) via the communication unit 1300. The communication status to be determined is, specifically, the communication speed (e.g., bps), and may include communication strength (e.g., dB) in addition to the communication speed. The communication status determination unit 1430 particularly determines the data communication speed of data acquired by the data communication unit 1320. Furthermore, when the self-position / velocity determination unit 1110 uses GPS or GNSS, or when the attitude determination unit 1120 uses a GPS compass, the communication status determination unit 1430 may determine the reception strength, positioning accuracy, and measurement accuracy of satellite signals received from artificial satellites via GPS or GNSS. Here, the positioning accuracy can be determined based on the strength of the satellite signals, the number of satellites from which satellite signals can be received, and the elevation angle of the satellites.

[0034] (A-1-2-5. Sensing data recording unit 1500) The sensing data recording unit 1500 records the acquired data acquired by the sensing unit 1200. The recorded acquired data is transmitted to a data acquisition site system or the like by the data communication unit 1320. The sensing data recording unit 1500 is configured by a storage device such as a RAM or a ROM.

[0035] (A-1-2-6. Status data recording unit 1600) The status data recording unit 1600 includes a flight control information recording unit 1610, a measurement control information recording unit 1620, and a status determination result recording unit 1630. The status data recording unit 1600 is configured by a storage device such as a RAM or a ROM.

[0036] The flight control information recording unit 1610 records information on the position and speed of the flying object determined by the self-position / speed determining unit 1110, and the attitude of the flying object determined by the attitude determining unit 1120. In addition to the position, speed, and attitude of the flying object, other state quantities measured by the flight unit 1100, such as acceleration, angular velocity, and angular acceleration, may also be recorded.

[0037] The measurement control information recording unit 1620 records status information of the relative angle of the sensor with respect to the airframe of the flying object, which is acquired by the sensor attitude control unit 1220. It also records information on the timing of sensing execution by the sensor 1210 controlled by the sensor control unit 1230 and measurement parameters (image acquisition timing, exposure, shutter speed, aperture (F-number), zoom amount, ISO, resolution). It also acquires information on whether sensing is being executed by the sensor 1210 from the sensor control unit 1230, or acquires information on whether sensing data is being recorded from the sensing data recording unit 1500, and records this as sensing data presence / absence information.

[0038] The state determination result recording unit 1630 records the determination results determined by the flight state determination unit 1410 , the sensing state determination unit 1420 , and the communication state determination unit 1430 of the state determination unit 1400 .

[0039] (A-1-3. Data Acquisition Center System 2000) 3 is a configuration diagram of a data acquisition base system according to one embodiment of the present invention. The data acquisition base system 2000 includes a communications infrastructure management system 2100, an aircraft flight operation system 2200, a data management system 2300, an aircraft flight management system 2500, and an airspace monitoring system 2600.

[0040] When the aircraft is an unmanned aircraft, the data acquisition base system 2000 is a system that exchanges control information regarding the aircraft and the flight section and the sensing section with the aircraft, enabling remote control or data acquisition work by a pilot (user) using the data acquisition base system. On the other hand, when the aircraft is a manned aircraft, the system exchanges control information and aircraft status information regarding the aircraft and the flight section and the sensing section with the aircraft, supporting data acquisition work by the aircraft. The data acquisition base system may be composed of a mobile vehicle, ship, aircraft, etc., or may be composed of an immovable building (fixed type).

[0041] The communication infrastructure management system 2100 is a system that manages communication means for data (such as acquired data acquired by sensing operations of the aircraft) and control communication information transmitted and received between the data acquisition base system 2000, the aircraft 1000, and the data processing base system 3000. The communication infrastructure management system 2100 transmits, for example, information generated by the aircraft flight operating system 2200 and the data management system 2300 (described later) to the aircraft 1000.

[0042] Furthermore, there are multiple communication means for communicating with the flying object 1000, such as a communication means for directly communicating with the flying object 1000 using antenna equipment within the data acquisition base system 2000, a communication means via a mobile phone communication network (such as LTE), and a communication means for communicating via satellite relay via the satellite system 6000.Therefore, the communication infrastructure management system 2100 monitors the communication speed and communication strength of these communication means and switches between communication means.The transmission process may be, for example, parallel transmission or switched transmission.Furthermore, if the communication infrastructure management system 2100 determines through the data management system 2300 (described later) that it is appropriate to transmit the data by physically transporting the memory by a person, it may display this fact on an appropriate display unit and request the administrator to physically transport the memory.

[0043] The aircraft flight operating system 2200 is a system that generates a mission for the aircraft to be operated and controls the movement of the aircraft. The mission is, for example, a movement plan including the movement route and movement speed of the aircraft 1000, and the movement route is generated in an airspace at an altitude of 150 m or less to avoid interference with passenger aircraft, for example. The aircraft flight operating system 2200 transmits a control signal to the aircraft 1000 via the communication infrastructure management system 2100 to operate the aircraft automatically.

[0044] The data management system 2300 acquires acquired data and status data of the target area sensed by the aircraft via communication infrastructure management, determines the appropriateness of the content of the acquired data and the status data at the time of sensing, and then determines the remeasurement conditions. The detailed functions of the data management system 2300 will be described later.

[0045] The flight management system 2500 is a system that makes decisions and gives instructions for the operation of the aircraft 1000. The flight management system 2500 prepares plans for operations of the aircraft 1000, including, for example, sensing and flight, and transmits the plans to the aircraft 1000 via the communication infrastructure management system 2100. The flight management system 2500 may also determine the priority of operations that require processing and transmit the priorities to the aircraft 1000. The flight management system 2500 may prepare plans for multiple aircraft 1000 and transmit information about the plans to each aircraft 1000.

[0046] The airspace monitoring system 2600 is a system that monitors the airspace in which the target air vehicle 1000 is flying. The airspace monitoring system 2600 acquires information from an existing air traffic control system. The air traffic control system may be, for example, a drone traffic management system (UTM), a drone traffic management subsystem (UASSP), or an air traffic management system (ATM). That is, the airspace monitoring system 2600 measures or acquires information about the environment and other aircraft in the airspace in which the air vehicle 1000 is flying, and transmits the information to the flight management system 2500. If there is a problem with the work plan for the air vehicle 1000 based on the information from the airspace monitoring system 2600, the flight management system 2500 changes the work plan.

[0047] (A-1-4. Data processing base system 3000) 4 is a configuration diagram of a data processing base system according to one embodiment of the present invention. The data processing base system includes a communication unit 3100, a data processing unit 3200, and a data processing management unit 3300.

[0048] The communication unit 3100 transmits and receives data (acquired data) between the data processing base system 3000 and external systems (the data acquisition base system 2000 and the distribution system 4000). The communication unit 3100 receives acquired data from the data acquisition base system 2000, for example. The communication unit 3100 also transmits processed data generated in the data processing base system 3000 to the distribution system 4000.

[0049] The data processing unit 3200 executes data processing of the acquired data acquired by the flying object 1000 based on a data processing command generated by a data processing execution command unit 3310 of the data processing management unit 3300 described later.

[0050] The data processing management unit 3300 judges the suitability of the processed data and the remeasurement conditions based on the processed data generated by the data processing unit 3200 or the content of the data being processed, and then provides the processed data to the distribution system. Details of the data processing management unit 3300 will be described later.

[0051] (A-1-5. Distribution System 4000) 5 is a diagram showing the configuration of a distribution system according to one embodiment of the present invention. The distribution system 4000 includes a communication unit 4100 and a distribution data management unit 4200.

[0052] The communication unit 4100 receives processed data from the data processing base system 3000. The received processed data may be data that has undergone web optimization processing in the data processing unit 3200 of the data processing base system 3000. The communication unit 4100 also distributes the processed data to user terminal devices 5000, including crisis management headquarters terminals, via an Internet line.

[0053] The distribution data management unit 4200 records the processed data received via the communication unit 4100, and distributes the latest processed data recorded in the distribution data management unit 4200 to user terminal devices, including device countermeasure headquarters terminals, accessed via an Internet line.

[0054] (A-1-6. Data Management System 2300) 6 is a functional configuration diagram of a data management system according to one embodiment of the present invention. The data management system 2300 includes a user request acquisition unit 2310, a sensing data acquisition unit 2320, a status information acquisition unit 2330, a data suitability determination unit 2340, a corresponding action determination unit 2350, a remeasurement condition determination unit 2360, and a corresponding action command unit 2370.

[0055] (A-1-6-1. User request acquisition unit 2310) The user request acquisition unit 2310 acquires request information from a user in advance. The request information acquired by the user request acquisition unit 2310 includes suitability determination criteria for the suitability determination of sensing data executed by the data suitability determination unit 2340, which will be described later. In this case, for example, the suitability determination criteria include determination criteria related to the state of sensing data obtained by the aircraft, the state of the aircraft, the state of the sensor, etc. For example, the suitability determination criteria include each of the inappropriateness determination criteria described in the user request information shown in Table T101 in FIG. 8, which will be described later.

[0056] Here, when the sensing data is an image taken by a camera, the criteria for determining whether the sensor state is inappropriate can be obtained as user request information, such as the resolution r of the camera image being below a reference value P1, or the brightness Y of the camera image being below a reference value L1, or the camera image being out of focus, and other criteria resulting from the sensor's setting parameters.

[0057] Furthermore, as criteria for determining whether the state of the aircraft is inappropriate, criteria for determining the flight state of the aircraft can be acquired as user request information, such as the absolute value ΔX of the difference between the position X of the aircraft and the target position X1 being equal to or greater than a reference value, the absolute value ΔV of the difference between the flight speed V of the aircraft and the target speed V1 being equal to or greater than a reference value, the absolute value Δh of the difference between the flight altitude h of the aircraft and the target altitude h1 being equal to or greater than a reference value, or the angular difference between the attitude angle (Yaw, Roll, Pitch) of the aircraft and the target angle being equal to or greater than a reference value (for example, 15 degrees).

[0058] In addition, as criteria for determining whether the state of sensing data is inappropriate, for example, the sensing operation was not performed properly and there is missing data in the camera image, there are water droplets, condensation, or dirt on the lens when the camera image was acquired, or there are defects in the camera image due to clouds or fog, or the image of the target area cannot be acquired because the lens cover of the camera image is closed, etc., judgment criteria regarding the content of the sensing data can be obtained as user request information.

[0059] The request information acquired by the user request acquisition unit 2310 may include suitability determination criteria for determining suitability of the processed data, which is executed by the data suitability determination unit 3320, which will be described later. In this case, for example, the suitability determination criteria include inappropriateness determination criteria regarding the state of the processed data.

[0060] The request information acquired by the user request acquisition unit 2310 includes criteria for determining a response action to be executed by the response action determination unit 2350, which will be described later. In this case, for example, the criteria for determining a response action include multiple criteria related to the state of sensing data from the aircraft, the state of the aircraft, the state of the sensor, etc., and a response action corresponding to each criterion. For example, as shown in Table T102 in Fig. 9, which will be described later, the user request acquisition unit 2310 acquires response action determination criteria that can determine a response action depending on the determination result by the data appropriateness determination unit 2340, which will be described later.

[0061] Here, when the sensing data is an image captured by a camera, the corresponding action according to the sensor state caused by the sensor setting parameters, such as the resolution r of the camera image, the brightness Y of the camera image, and the focus error of the camera image, can be obtained as user request information as a criterion for determining the corresponding action according to the sensor state.

[0062] Furthermore, as a criterion for determining a corresponding action according to the state of the flying object, for example, the difference ΔX between the position X of the flying object and the target position, the difference ΔV between the flight speed V of the flying object and the target speed, the difference Δh between the flight altitude h of the flying object and the target altitude, the angular difference between the attitude angle (Yaw, Roll, Pitch) of the flying object and the target angle, etc., can be acquired as user request information indicating a corresponding action according to the flight state of the flying object.

[0063] In addition, as a criterion for determining response actions according to the state of the sensing data, response actions according to defects in the content of the sensing data can be obtained as user request information, such as missing data in the camera image due to the influence of clouds or fog, defects in the camera image due to water droplets, condensation, or dirt on the lens when the camera image was acquired, or the inability to acquire an image of the target area due to the camera lens cover being closed.

[0064] The above-mentioned response actions may include response actions that allow the use of inappropriate data that has been determined to be inappropriate (for example, "notifying restrictions on the use of processed data," "notifying that inappropriate data is included," "specifying the data processing method," "specifying the display method of processed data," etc.), and may also include response actions that prohibit the use of inappropriate data that has been determined to be inappropriate (for example, "not using inappropriate data," "substituting other data for inappropriate data," "re-measurement by an aircraft"). In other words, the request information acquired by the user request acquisition unit 2310 may include usage conditions for inappropriate data that allow the use of inappropriate data that has been determined to be inappropriate.

[0065] (A-1-6-2. Sensing data acquisition unit 2320) The sensing data acquisition unit 2320 acquires sensing data sensed by the sensing unit 1200 mounted on the aircraft via the communication infrastructure management system 2100. The acquired sensing data is, for example, optical image data or point cloud data acquired by a laser sensor such as LiDAR.

[0066] (A-1-6-3. Status information acquisition unit 2330) The status information acquisition unit 2330 acquires the information recorded in the flight control information recording unit 1610, the measurement control information recording unit 1620, and the status determination result recording unit 1630 of the aircraft via the communication infrastructure management system 2100.

[0067] FIG. 10 is a table showing an example of information acquired by the status information acquisition unit according to an embodiment of the present invention. Information acquired from the flight control information recording unit 1610 includes, for example, the position, flight speed, flight altitude, and attitude angle (Yaw, Roll, Pitch) of the aircraft, as well as state quantities such as acceleration, angular velocity, and angular acceleration, as shown in Table T103 of FIG. 10 . Information acquired from the measurement control information recording unit 1620 includes, for example, the image acquisition timing of the sensor, exposure, shutter speed, aperture (F-number), zoom amount (affecting focus), ISO, resolution, and brightness, as well as other measurement parameters (contrast, histogram, etc.), timing of sensing execution, and status information regarding the relative angle (Yaw, Roll, Pitch) of the sensor with respect to the aircraft body, as shown in Table T103 of FIG. 10 . Sensing data presence / absence information regarding the presence or absence of sensing data may also be acquired from the measurement control information recording unit 1620.

[0068] The information acquired from the state determination result recording unit 1630 is information on the determination results made by the flight state determination unit 1410, the sensing state determination unit 1420, and the communication state determination unit 1430 of the state determination unit 1400. In particular, the acquired determination results include the reception strength of satellite signals, positioning accuracy, and measurement accuracy.

[0069] (A-1-6-4. Data suitability determination unit 2340) The data suitability determination unit 2340 determines the suitability of the sensing data based on the content of the sensing data acquired by the sensing data acquisition unit 2320, or various state information (including the control state of the aircraft and the measurement control state of the sensors) acquired by the state information acquisition unit 2330, or a combination of these, and displays the determination result. The determination may be based on suitability determination criteria such as those shown in FIG. 8, or suitability determination may be performed using artificial intelligence. Alternatively, the determination may be based on input information of a determination result that the user has determined to be inappropriate. The data suitability determination unit 2340 includes a data analysis unit 2341, a state information analysis unit 2342, and a suitability determination result display unit 2343.

[0070] Fig. 8 is a table showing an example of criteria for determining whether or not sensing data is appropriate according to one embodiment of the present invention. Table T101 in Fig. 8 shows an example of criteria for determining whether or not sensing data is appropriate when the sensing data is an image captured by a camera. The criteria for determining whether or not sensing data is appropriate include criteria for whether or not sensing data is appropriate for the state of measurement control by the sensor, the state of flight control of the aircraft, and the state of the sensing data.

[0071] The appropriateness determination criteria can be set based on input information from the user acquired by the user request acquisition unit 2310, but are not limited to this and may be set in advance by the system. Alternatively, the user may select the criteria to be applied by inputting into a selection field on the left side of T101 from default setting criteria for each target state quantity that are set in advance by the system, or may set any conditions by inputting into a criteria field for inappropriateness determination on the right side of T101.

[0072] As an example of criteria for determining whether sensing data is appropriate or not that differs from that shown in Figure 8, the user may input the intended use of the sensing data or the processed data (such as for inspecting features or for forest surveying), and the system may automatically generate criteria for determining whether the data is appropriate or not based on the intended use entered.

[0073] First, the data analysis unit 2341 analyzes the content of the sensing data acquired by the sensing data acquisition unit 2320, and determines whether the sensing data is inappropriate in light of the criteria for determining inappropriateness regarding the state of the sensing data, such as those shown in Table T101 in Fig. 8. Table T101 lists criteria that can be used to determine inappropriateness regarding the state of the sensing data, based on an analysis of the content of the sensing data, such as missing data in part of the camera image due to the influence of clouds or fog, a defect in part of the camera image due to water droplets, condensation, or dirt on the lens when the camera image was acquired, or local insufficient brightness or blown-out highlights, failure to acquire an image of the target area due to the lens cover of the camera image being closed, the brightness Y of the camera image being equal to or less than a reference value L1 or equal to or greater than a reference value L2, or the camera image being out of focus.

[0074] In addition to the above, the criteria for judgment may include distortion of the image, blurring of the image, the image being of an unnecessary position outside the sensing target area (such as the return path of the aircraft), the resolution, contrast, or histogram being below a predetermined value, receiving information that there is no sensing data, or the relative angle of the sensor to the aircraft body (Yaw, Roll, Pitch) exceeding a reference value.

[0075] Furthermore, when sensing is performed using a laser sensor, the criteria for determining inappropriateness may include when the point cloud data does not contain specified feature points, when the laser reflectivity is below a specified value, when there is an area within the sensing target area where the point cloud has not been acquired, or when the point cloud is from an unnecessary position outside the sensing target area (such as the return path of the aircraft).

[0076] When the data analysis unit 2341 makes an inappropriate judgment based on the state of the sensing data, in addition to the judgment method of comparing the state quantity related to the sensing data with a reference value as shown in T101 as described above, it is also possible to make the judgment by comparing the state quantity of sensing data for the same area acquired in the past, or by comparing the state quantity of the sensing data with a target value, or by the acquisition rate of sensing data for the target area to be sensed, etc.

[0077] Next, the status information analysis unit 2342 analyzes the contents of the various status information acquired by the status information acquisition unit 2330, and determines whether the sensing data is inappropriate in light of the criteria for determining inappropriateness related to the measurement control status of the sensor or the flight control status of the aircraft, as shown in Table T101 in Fig. 8. Table T101 shows criteria for determining inappropriateness related to the measurement control status of the sensor, such as the resolution r of the camera image being equal to or less than a reference value P1, the brightness Y of the camera image being equal to or less than a reference value L1 or being equal to or greater than a reference value L2, or the camera image being out of focus, which are determined as being due to the setting parameters of the sensor.

[0078] In addition to the above, the judgment criteria may include receiving information that there is no sensing data, the relative angle (Yaw, Roll, Pitch) of the sensor to the aircraft body being outside the reference range, the absolute angle (Yaw, Roll, Pitch) of the sensor being outside the reference range, and data not being obtained at the point that should be sensed.

[0079] Table T101 also shows criteria for determining whether the state of an aircraft is inappropriate, such as the difference ΔX between the position of the aircraft and the target position (or target route) being outside the reference value range, the difference ΔV between the flight speed V of the aircraft and the target speed V1 being outside the reference value range, the difference Δh between the flight altitude of the aircraft and the target altitude being outside the reference value range, and the difference in angle between the attitude angle (Yaw, Roll, Pitch) of the aircraft and the target angle being outside the reference range (for example, 15 degrees or more).

[0080] In addition to the above, the criteria may include the reception strength of the satellite signal received by the flying object falling below a reference value, the measurement accuracy of the positioning accuracy or speed of the flying object falling below a reference value, the state quantities of the flying object such as acceleration, angular velocity, and angular acceleration falling outside a reference range, etc. Furthermore, particularly when sensing is performed using a laser sensor, the criteria for determining inappropriateness may include the flight altitude being higher than a predetermined value, because reflected laser light cannot be obtained if the distance to the sensing object is too great.

[0081] In addition, when the state information analysis unit 2342 makes an inappropriate judgment based on the flight control state of the aircraft, in addition to the judgment method of comparing the state quantity of the aircraft with a reference value such as that shown in T101, as described above, the inappropriate judgment may also be made by comparing with past control actual values ​​for the aircraft, or by comparing with the control target value of the state quantity to be controlled for the aircraft.

[0082] The method for determining inappropriateness described above may be a method in which an image is determined to be inappropriate if it deviates from the judgment criteria even once, or a method in which an image is determined to be inappropriate if the number of times the judgment criteria have been deviated exceeds a predetermined number, if there is an area in part of the image that deviates from the judgment criteria and the area that deviates from the judgment criteria exceeds a predetermined area, or if the proportion of the area that occupies the entire image exceeds a predetermined proportion.

[0083] Next, the appropriateness judgment result display unit 2343 displays the result of the above-mentioned inappropriateness judgment. FIG. 11 is a diagram showing an example of a display when the appropriateness judgment result display unit of the present invention displays the result of the appropriateness judgment. In the example shown in FIG. 11, inappropriateness judgment candidate data that has been judged to be inappropriate is displayed in an identifiable manner with respect to the sensing data that is the subject of the appropriateness judgment. A user of the data management system 2300 can select any inappropriateness judgment candidate data using the user input accepting unit 2344. When any inappropriateness judgment candidate data is selected, the data content of the selected inappropriateness judgment candidate data (image data if the inappropriate data is an image) is displayed, along with the reason for the judgment as inappropriate (e.g., out of focus), the degree of exceedance of the reference value (e.g., large, medium, or small), the time when the inappropriateness judgment candidate data was sensed, and coordinate information of the ground area corresponding to the inappropriateness judgment candidate data.

[0084] In addition, the suitability judgment result display unit 2343 may display or output in audio the inappropriate data that is usable as determined based on the usage conditions of the inappropriate data received by the user request acquisition unit 2310, or the corresponding action that is the usage condition of the inappropriate data that is determined to be usable.

[0085] Furthermore, when any candidate data for inappropriate determination is selected by the user input accepting unit 2344, the appropriateness determination result display unit 2343 displays on the screen the above-mentioned information and an input button that allows the user to select whether or not to confirm the data as inappropriate data. By operating the input button by the user input accepting unit 2344, the user can confirm the selected candidate data for inappropriate determination as inappropriate data.

[0086] (A-1-6-5. Corresponding action determination unit 2350) The response action determination unit 2350 determines a response action from among multiple candidate response actions, including remeasurement, based on the contents of the sensing data acquired by the sensing data acquisition unit 2320, or the various status information (including the flight control status of the aircraft and the measurement control status of the sensors) acquired by the status information acquisition unit 2330, or a combination of these, or the result of the inappropriateness determination by the data appropriateness determination unit 2340 described above, or transmits the determination result of the response action to the outside, or displays or outputs the candidate response action or the determination result as audio. The response action determination unit 2350 includes a determination processing unit 2351 and a response action determination result display unit 2352. The determination processing unit 2351 determines a response action to be taken, particularly when the data appropriateness determination unit 2340 determines that the data is inappropriate.

[0087] 9 is a diagram showing an example of response action determination criteria according to one embodiment of the present invention. The determination processing unit 2351 may determine a response action based on the response action determination criteria such as those shown in Table T102 in FIG. 9, or may determine a response action using artificial intelligence or the like. Alternatively, the response action may be determined based on command input information for the response action received from the user.

[0088] Table T102 in Figure 9 shows an example of criteria for determining a response action when it is assumed that the sensing data is an image captured by a camera. The response action criteria include criteria for determining a response action related to the measurement control state of the sensor, the flight control state of the aircraft, and the content of the sensing data. However, the criteria for determining a response action are not limited to this, and criteria for determining a response action corresponding to the result of an inappropriate judgment or the content of the inappropriate judgment (reason for judgment) may also be set.

[0089] This determination criterion can be set based on input information from the user acquired by the user request acquisition unit 2310, but is not limited to this and may be set in advance by the system. Alternatively, the user may select a criterion to be applied by inputting a value into the selection field on the left side of T102 from default setting criteria for each target state quantity that are set in advance by the system, or may set an arbitrary condition by inputting a value into the corresponding action determination field on the right side of T102.

[0090] As an example of criteria for determining corresponding actions for sensing data different from those shown in Figure 9, the user may input the intended use of the sensing data or the processed data (such as for feature inspection or forest surveying), and the system may automatically generate criteria for determining corresponding actions and determination results according to the intended use input.

[0091] Response actions determined by the determination processing unit 2351 include, for example, denial of permission to use inappropriate data determined to be inappropriate, substitution of other data for inappropriate data, notification of restrictions on use of processed data, notification that inappropriate data is included, specification of a data processing method, remeasurement, specification of a data processing method according to the intended use specified by the user, specification of a display method for processed data according to the intended use specified by the user, and the like.

[0092] The above-mentioned "specifying a data processing method according to the intended use specified by the user" and "specifying a display method for processed data according to the intended use specified by the user" mean, for example, specifying a data processing method or a display method for processed data according to the intended use input acquired by the user request acquisition unit 2310.

[0093] Here, the determined response actions include a response action that does not use inappropriate data determined to be inappropriate, and a response action that allows the use of inappropriate data. Response actions that do not use inappropriate data include, for example, "denying the use of inappropriate data determined to be inappropriate," "substituting other data for inappropriate data," and "remeasurement." Response actions that allow the use of inappropriate data include, for example, "notifying restrictions on the use of processed data," "notifying that inappropriate data is included," "designating a data processing method," "designating a data processing method according to the use specified by the user," and "designating a display method for processed data according to the use specified by the user."

[0094] As a criterion for determining a response action, for example, for inappropriate data whose degree of exceedance from the inappropriateness judgment criterion is less than a predetermined value, the response action can be to allow the use of the inappropriate data, and for inappropriate data whose degree of exceedance from the inappropriateness judgment criterion is more than the predetermined value, the response action can be to not use the inappropriate data.

[0095] First, in Table T102, as a criterion for determining the response action regarding the state of sensing data, for example, when there is data loss in part of the camera image due to disturbances such as clouds or fog, the response action is determined to be "substitute other data" or "remeasurement." Here, "substitute other data" means substituting other data (such as geographical or map information or past sensing data) in place of inappropriate data. Also, "remeasurement" means re-performing sensing using an aircraft.

[0096] Furthermore, in Table T102, if there is a defect in part of the camera image due to water droplets, condensation, or dirt on the lens when the camera image was acquired, or due to local insufficient brightness or blown-out highlights, or if an image of the target area could not be acquired due to the camera lens cover being closed, the response actions that do not allow the use of inappropriate data are determined to be "substitution of other data" and "remeasurement."

[0097] Furthermore, if the brightness Y of the camera image is equal to or less than the reference value L1 or is equal to or greater than the reference value L2, the response action is determined to be "image adjustment by adjusting brightness." That is, the response action for allowing the use of inappropriate data determined to be inappropriate by the data appropriateness determination unit 2340 is determined to be "designation of data processing method." In other words, it is determined that the use of inappropriate data for data processing is allowed on the condition that image adjustment by adjusting brightness is performed as the data processing method to be specified.

[0098] Furthermore, if the camera image is out of focus due to factors such as the camera focus setting, the response actions for allowing the use of inappropriate data are "notification that inappropriate data is included" and "unavailable for use in person determination." Here, "notification that inappropriate data is included" means allowing the use of inappropriate data for data processing on the condition that the data processor or the user viewing the processed data is notified that inappropriate data is included. Furthermore, "unavailable for use in person determination" means allowing the use of inappropriate data for data processing on the condition that the data processor or the user viewing the processed data is notified that the processed data will not be used for purposes such as determining the presence of people from image data, as a usage restriction on the processed data.

[0099] Next, in Table T102, as a criterion for determining a response action regarding the state of the flying object, for example, when the resolution r of the camera image is equal to or less than the reference value P1 and equal to or greater than the reference value P2, the response actions that allow the use of inappropriate data are determined to be "notification that inappropriate data is included" and "unusable for use in human detection." Also, when the resolution r of the camera image is less than the reference value P2, the response actions that do not allow the use of inappropriate data are determined to be "substitute other data" and "remeasure."

[0100] Next, in Table T102, as a criterion for determining a response action related to the state of the aircraft, for example, if the absolute value ΔX of the difference between the aircraft's position and the target position (or target route) is equal to or greater than a reference value X1 and equal to or less than a reference value X2, the response action for allowing the use of inappropriate data is determined to be "position correction processing of the ground surface photographed area according to the position error." In other words, it is determined that the use of inappropriate data for data processing is allowed on the condition that "position correction processing of the ground surface photographed area according to the position error" is performed as the specified data processing method. On the other hand, if the absolute value ΔX of the difference between the aircraft's position and the target position (or target route) is greater than X2, the response actions for not allowing the use of inappropriate data are determined to be "substitution of other data" and "remeasurement."

[0101] Furthermore, if the absolute value ΔV of the difference between the flight speed of the aircraft and the target flight speed is equal to or greater than the reference value V1 and equal to or less than the reference value V2, the response action to allow the use of inappropriate data is determined to be "notification that inappropriate data is included." On the other hand, if the absolute value ΔV of the difference between the flight speed of the aircraft and the target flight speed is greater than V2, the response actions to not allow the use of inappropriate data are determined to be "substitution of other data" and "remeasurement."

[0102] Furthermore, if the absolute value Δh of the difference between the flight altitude of the aircraft and the target flight altitude is equal to or greater than the reference value h1 and equal to or less than the reference value h2, the response action to allow the use of inappropriate data is determined to be "notification that inappropriate data is included." On the other hand, if the absolute value Δh of the difference between the flight altitude of the aircraft and the target flight altitude is greater than h2, the response actions to not allow the use of inappropriate data are determined to be "substitution of other data" and "remeasurement."

[0103] Furthermore, if the angular difference between the aircraft's attitude angle (Yaw, Roll, Pitch) and the target angle is within a predetermined range (for example, 15 degrees or more and 30 degrees or less), the system determines that the response action to allow the use of inappropriate data is "position correction processing of the ground surface photographed area according to the angle error." In other words, the system determines that the use of inappropriate data for data processing is allowed on the condition that "position correction processing of the ground surface photographed area according to the angle error" is performed as the specified data processing method. On the other hand, if the angular difference between the aircraft's attitude angle (Yaw, Roll, Pitch) and the target angle is greater than 30 degrees, the system determines that the response actions to not allow the use of inappropriate data are "substitution of other data" and "remeasurement."

[0104] As described above, in the example shown in Table T102 of Figure 9, the judgment processing unit 2351 determines the inappropriate data that can be used by determining either a response action that does not allow the use of inappropriate data or a response action that allows the use of inappropriate data.

[0105] In the example shown in Table T102 of FIG. 9, the criteria for determining a response action when determining a response action after the data acquisition process by the aircraft is completed are shown, but when the data suitability is determined during the execution of the measurement process (during flight), the response action can be determined based on criteria different from those in Table T102. In other words, the response action can be determined according to the result of the data suitability determination and the execution timing of the response action determination. Note that the execution timing of the data suitability determination is intended to include during measurement (during flight), before data processing, during data processing, and after data processing.

[0106] For example, if the timing for determining whether the data is appropriate is during measurement by an aircraft (during flight), the determination processing unit 2351 can determine that the corresponding action is to stop measurement, change the measurement parameters of the sensor during flight, or switch the sensor.

[0107] Specifically, if it is determined during measurement (flight) by the aircraft that the camera image acquired as measurement information does not satisfy the suitability criteria for resolution r, brightness Y, focus, or other suitability criteria (exposure, exposure, gain, contrast, etc.) shown in Table T101, the determination processing unit 2351 of the response action determination unit 2350 determines, as the response action during measurement (flight) by the aircraft, that the measurement parameters (resolution, brightness, focus, etc.) of the sensor (camera) be changed or the sensor (camera) be switched to another sensor (camera), and causes the response action command unit 2370, described later, to execute the response action. Furthermore, if it is determined that the camera image acquired after executing the response action still does not satisfy the suitability criteria, the response action is to abort the measurement and perform remeasurement. Note that when aborting the measurement and performing remeasurement, the aircraft may land once after aborting the measurement and then perform remeasurement, or the aircraft may continue flying without landing after aborting the measurement and fly a remeasurement mission.

[0108] Next, the response action determination result display unit 2352 displays or outputs audibly the determination result of the response action determined by the determination processing unit 2351. Fig. 12 is a diagram showing an example of a display when the response action determination result display unit of the present invention displays the determination result of the response action. In the example shown in Fig. 12, "remeasurement" is displayed as the determination result of the response action. Other candidate response actions (denying data use, substituting other data, allowing data use, notifying the user of the inappropriateness determination content, use only for orthoimages, use only for 3D model generation, disallowing use for human determination processing, etc.) are also displayed.

[0109] A user of the data management system 2300 can select any inappropriate data using the user input accepting unit 2353, and the determination result of the response action corresponding to the selected inappropriate data is displayed. Furthermore, when any inappropriate data is selected, the determination result of the response action described above and other candidates are displayed so that they can be selected, and a confirmation button for the response action is also displayed. The user can select any response action using the user input accepting unit 2353, and with a specific response action selected, can confirm the response action by operating the confirmation button for the response action.

[0110] (A-1-6-6. Remeasurement condition determination unit 2360) The remeasurement condition determination unit 2360 is a functional unit that determines various conditions for performing remeasurement when the response action determined by the response action determination unit 2350 is remeasurement, displays candidate conditions, and accepts selection input for the candidate conditions from the user. The remeasurement condition determination unit 2360 includes a condition determination unit 2361, a determination condition display unit 2362, and a user input acceptance unit 2363.

[0111] The condition determination unit 2361 is a functional unit that determines the re-measurement conditions. For example, the condition determination unit 2361 determines the re-measurement conditions based on the determination criteria for the re-measurement conditions as shown in Fig. 13 and the result of the data suitability determination or the determination content (reason for determination). Fig. 13 is a diagram showing the determination criteria for the re-measurement conditions when the re-measurement condition determination unit determines the re-measurement conditions. In the example of the determination criteria shown in Fig. 13, the re-measurement conditions are set in advance according to the content of the suitability determination by the data suitability determination unit 2340.

[0112] As an example, if the data is determined to be inappropriate because the resolution r of the camera image is smaller than the reference value P2, changing the camera lens to one that can acquire images with a higher resolution is determined to be a remeasurement condition.

[0113] In addition, if the data is determined to be inappropriate because the position x of the aircraft is greater than the reference value X2, changing the aircraft from a fixed-wing aircraft to a multicopter aircraft is determined to be a condition for re-measurement.

[0114] Furthermore, if the data is determined to be inappropriate because the flight speed V of the aircraft is greater than the reference value V2, a remeasurement condition is determined to be that the target flight speed of the aircraft be reduced. Furthermore, if the data is determined to be inappropriate because the flight altitude h of the aircraft is greater than the reference value h2, a remeasurement condition is determined to be that the target flight altitude of the aircraft be reduced.

[0115] In addition, if the data is determined to be inappropriate due to an angle difference of 30 degrees or more between the attitude angle of the aircraft and the target angle, changing the aircraft from a fixed-wing aircraft to a multicopter aircraft is determined to be a condition for re-measurement.

[0116] In addition, if the data is determined to be inappropriate due to missing data in the camera image, the remeasurement condition is determined to be that remeasurement be performed using a flight mission (including the flight path) that takes multiple photographs of the location where the missing data was detected.

[0117] Furthermore, if the data is determined to be inappropriate due to the detection of water droplets on the lens, condensation on the lens, or dirt on the lens from the camera image, it is determined that the condition for remeasurement is to change the sensor from a camera to a laser sensor, or to use a laser sensor in addition to a camera.Also, if the data is determined to be inappropriate due to the detection of a lens cover being left on the camera image, it is determined that the condition for remeasurement is to change the sensor from a camera to a laser sensor, or to use a laser sensor in addition to a camera.

[0118] As described above, the remeasurement conditions can include flight conditions related to at least one of the flight path, flight altitude, flight speed, and flight attitude angle of the aircraft, the type of sensor (optical camera, laser sensor, etc.), and the type of aircraft (fixed-wing aircraft, multicopter, etc.). In addition to the examples shown in Table T104, the remeasurement conditions can also include setting conditions for the optical camera related to at least one of the exposure, gain, contrast, focus, aperture, and shutter speed of the optical camera.

[0119] As shown in Table T104, in order to prevent sensing data obtained by remeasurement from being judged as inappropriate again in the suitability judgment, the judgment criteria for the remeasurement conditions are set in advance according to the judgment content of the suitability judgment so that the factors that were judged as inappropriate in the suitability judgment are resolved. Therefore, the measurement conditions that are not related to the factors that were judged as inappropriate can be the same as the previous measurement, and the measurement conditions related to the factors that were judged as inappropriate can be changed to judgment criteria that are different from the conditions in the previous measurement.

[0120] The judgment condition display unit 2362 is a functional unit that displays or outputs by voice the remeasurement conditions judged by the condition judgment unit 2361. Furthermore, the user input accepting unit 2363 is a functional unit that accepts approval input or change input for the remeasurement conditions judged by the condition judgment unit 2361. Here, a display example of the judged remeasurement conditions will be described with reference to Fig. 14. Fig. 14 is a diagram showing a display example when the judgment condition display unit displays the judgment result of the remeasurement conditions.

[0121] 14, sensing data determined to be inappropriate by the data suitability determination unit 2340 ("NG data" in the figure) is associated with the photographed area of ​​the ground surface and displayed on the map information or photographed image. Furthermore, for each sensing data determined to be inappropriate, the reason for the determination of inappropriateness ("missing data" in the example shown in the figure) and information on the determined re-measurement conditions are displayed. Furthermore, information on the flight mission including the flight path connecting the re-measurement positions may be displayed as the re-measurement conditions.

[0122] 14 displays a re-measurement condition confirmation button as user input acceptance section 2363, and the user can approve and confirm the judgment result of the displayed re-measurement condition by operating this button. Furthermore, the user can correct and input the displayed re-measurement condition via user input acceptance section 2363.

[0123] As a modified example, the condition determination unit 2361 may not determine the remeasurement condition, but the user input reception unit 2363 may receive the input of the remeasurement condition, thereby determining the remeasurement condition.

[0124] (A-1-6-7. Response Action Command Unit 2370) Next, the response action command unit 2370 outputs an execution command for the response action determined by the response action determination unit 2350, causing the response action to be executed. Furthermore, if remeasurement is determined as the response action, the response action command unit 2370 outputs an execution command for remeasurement according to the remeasurement condition determined by the remeasurement condition determination unit 2360, causing the remeasurement to be executed. Furthermore, the response action command unit 2370 may transmit information on the determined response action or information on the remeasurement condition to an external system other than the data management system 2300 (such as the aircraft operation system 2200), an external system other than the data acquisition base system 2000 (such as a terminal of a person involved in the measurement work), or an external system other than the control system 1. The information to be executed or transmitted to the external system may include usable inappropriate data determined based on the determination criteria for allowing the use of inappropriate data received by the user request acquisition unit 2310, or a response action determined based on the usage conditions for the inappropriate data determined to be usable.

[0125] (A-1-7. Data Processing Management Department 3300) The data processing management unit 3300 determines the appropriateness of the processed data, the corresponding action, and the remeasurement conditions if remeasurement is necessary based on the processed data generated by the data processing unit 3200 or the contents of the data being processed, and then outputs a command for the corresponding action, generates display information for the processed data, and provides the display information for the processed data to the distribution system. FIG. 7 is a functional configuration diagram of a data acquisition management unit according to one embodiment of the present invention. As shown in FIG. 7, the data processing management unit 3300 includes a data processing execution command unit 3310, a data appropriateness determination unit 3320, a corresponding action determination unit 3330, a remeasurement condition determination unit 3340, a corresponding action command unit 3350, and a processed data display information generation unit 3360.

[0126] (A-1-7-1. Data processing execution command unit 3310) The data processing execution command unit 3310 notifies the user of the content of the response action command received from the response action command unit 2370 of the data management system 2300, and determines the content of the data processing to be executed by the data processing unit 3200, and sends an execution command for the data processing to the data processing unit 3200. The data processing execution command unit 3310 includes a response action display unit 3311 and a data processing command determination unit 3312.

[0127] The response action display unit 3311 is a functional unit that notifies the user of the content of the response action command received from the response action command unit 2370 of the data management system 2300 by displaying it on a display unit or outputting it as audio. By notifying the user (data processing officer) of the content of the response action command by the response action display unit 3311, the user can understand the content of the response action command, such as denial of permission to use inappropriate data determined to be inappropriate, substitution of other data in place of inappropriate data, notification of restrictions on use of processed data, notification that inappropriate data is included, specification of a data processing method, remeasurement, specification of a data processing method according to the use purpose specified by the user, and specification of a display method for processed data according to the use purpose specified by the user.

[0128] 15 and 16, a display example of a response action command displayed in the response action display unit 3311 will be described. Fig. 15 is a diagram showing a display example when the response action display unit displays a response action in association with a sensing target area on the ground. Fig. 16 is a diagram showing a display example when the response action display unit displays a response action in association with the position of the flying object.

[0129] First, in the example shown in Fig. 15, the response actions are displayed in map format on the screen of the display device in association with the sensing target areas on the ground. In Fig. 15, the sensing range on the ground for each piece of sensing data is shown by a dotted line, and the response actions are displayed in association with the two sensing target areas on the ground corresponding to the inappropriate data. In addition to the response actions, the reason for the inappropriateness determination is also displayed.

[0130] In the example shown in FIG. 16, the response actions are displayed in map format on the screen of the display device in association with the position of the aircraft. In FIG. 16, the position of the aircraft when sensing of each sensing data was performed (for example, the coordinates of the actual measured position) is indicated by a circle, and the response actions are displayed in association with the positions of the aircraft at two locations corresponding to the inappropriate data. In addition to the response actions, the reason for the inappropriateness determination is also displayed. Note that the positions indicated by the circles in FIG. 16 may be the control target positions on the control target path of the aircraft.

[0131] Data processing command determination unit 3312 is a functional unit that determines the content of data processing to be executed by data processing unit 3200 and sends an execution command for that data processing to data processing unit 3200. Fig. 17 is a table showing an example of data processing content that the data processing execution command unit of the present invention commands to the data processing unit. Data processing execution command unit 3310 can generate execution commands for various data processing such as those shown in Table T105 of Fig. 17. Furthermore, the content of the data processing for which an execution command is issued may be set in advance, or may be determined by receiving a request to execute a specific data processing selected and input by the user from the list of data processing shown in Table T105.

[0132] The contents of each data processing shown in Table T105 are as follows: Identification No. 001: A process of acquiring multiple optical images as acquired data, converting the acquired optical images into orthoimages, and generating an orthoimage by integrating the multiple orthoimages. Identification No. 002: A process in which multiple optical images are acquired as data and three-dimensional spatial data is generated from the multiple optical images using SfM (Structure from Motion) processing. Identification No. 003: A process of acquiring point cloud data as acquired data and generating three-dimensional spatial data from the point cloud data using DSM (Digital Surface Model) or DEM (Digital Elevation Model). Identification No. 004: Processing to generate the orthoimage shown in No. 001, obtain geographic data as processing data, and generate map integrated data that integrates the orthoimage and geographic data. Identification No. 005: Processing to generate the map integrated image shown in No. 004, obtain past map integrated images as processing data, and perform differential analysis between the current and past map integrated images. Identification No. 006: Processing to generate the orthoimage shown in No. 001, obtain disaster information as processing data, and generate an integrated disaster information image that integrates the orthoimage and disaster information. · Identification No. 007: Processing to generate the orthoimage shown in No. 001, obtain on-site unit information as processing data, and generate an on-site unit information integrated image that integrates the orthoimage and on-site unit information.

[0133] 17, the data processing command determination unit 3312 may automatically determine or receive from the user data processing conditions other than the data processing types shown in FIG. 17, and determine the content of the data processing based on the data processing conditions. For example, if the content of the response action command displayed by the response action display unit 3311 is "Not permitting the use of inappropriate data determined to be inappropriate," the data processing command determination unit 3312 determines not to use the inappropriate data for data processing based on an automatic determination or manual input information from the user. If the content of the response action command displayed by the response action display unit 3311 is "Substitute other data for inappropriate data," the data processing command determination unit 3312 determines automatically or based on manual input information from the user to substitute other data for the inappropriate data. If the content of the response action command displayed by the response action display unit 3311 is "Specify data processing method," the data processing command determination unit 3312 determines automatically or based on manual input information from the user to execute the specified data processing method.

[0134] (A-1-7-2. Data suitability determination unit 3320) The data appropriateness determination unit 3320 determines whether the data is appropriate or not, based on the contents of the data being processed that is generated during the data processing by the data processing unit 3200, or the contents of the processed data that is generated when the data processing is completed, and displays the appropriateness determination result. The data appropriateness determination unit 3320 includes a data analysis unit 3321, an appropriateness determination result display unit 3322, and a user input reception unit 3323.

[0135] The data analysis unit 3321 analyzes the contents of the data in progress and the processed data acquired from the data processing unit 3200, and judges inappropriate data based on the judgment criteria related to the contents of the data in progress and the processed data. The judgment criteria used by the data analysis unit 3321 to judge data as inappropriate can be set for each of the above-mentioned data processing contents, for example, as follows:

[0136] The criteria for determining inappropriateness in data processing identification Nos. 001, 004, 005, and 006, which generate orthoimages, can include, for example, when lens distortion of a predetermined value or more is detected from the image data, when blurring or focus shift of the image data is detected, when a portion of the captured image is detected as missing, when an image in an unnecessary position (such as the return path of the aircraft) is detected, when insufficient brightness or blown-out highlights are detected in the image data, etc.

[0137] The criteria for determining inappropriateness in data processing identification No. 002, which performs SfM processing using multiple images, can be, for example, when the object for which three-dimensional information is to be generated is not included in the image data, or when feature points of the object cannot be obtained. Alternatively, the criteria can include when lens distortion of a predetermined value or more is detected, when ground control points (GCPs (Ground Control Points), CPs (Check Points), etc.) are not captured in the image data, when doming of a predetermined value or more is detected, when a partial image is not captured, when an image of an unnecessary position (such as the return path of an aircraft) is detected, when insufficient luminosity or blown-out highlights are detected in the image data, etc. Furthermore, the criteria for determining inappropriateness are not limited to those described above, and criteria can be set for each processing step of SfM processing.

[0138] The criteria for determining inappropriateness in data processing identification No. 003, which generates three-dimensional spatial data from a measurement point cloud, can be, for example, when an area is detected where the laser does not reach the ground surface due to topography such as a cliff, or where point cloud data cannot be acquired due to water, fog, or other objects that absorb the laser. It can also include when a part of the point cloud data is missing, or when feature points cannot be detected from the point cloud data.

[0139] The criteria for determining inappropriateness in the data analysis unit 3321 may include the following criteria in addition to those mentioned above. - When the data processing content cannot detect a pair of images that can be used as a stereo image for measuring distances on the ground included in the image data. When processing both image data and point cloud data, if point cloud data is not available and the presence of clouds or fog is detected from the image.

[0140] The inappropriateness determination of the data in process and the processed data in the data analysis unit 3321 may be determined by comparing the state quantities of the data in process and the processed data with the reference values ​​as described above, but may also be determined by comparing the state quantities with the state quantities of past processed data or with the target values.

[0141] When the data analysis unit 3321 makes an inappropriate judgment based on the state of the data being processed and the processed data, in addition to the judgment method of comparing the state quantities related to the data being processed and the processed data with the judgment reference value (reference value of a specified parameter) as described above, the judgment can also be made by comparing the state quantities of the data being processed and the processed data for the same area generated in the past with each other, or by comparing the state quantities of the data being processed and the processed data with target values ​​for the state quantities of the data being processed.

[0142] Next, the appropriateness judgment result display unit 3322 displays the result of the above-mentioned inappropriateness judgment. Specifically, the display can be similar to the display of the inappropriateness judgment result displayed in the appropriateness judgment result display unit 2343 of the data management system 2300 shown in Fig. 11. In other words, the inappropriate data judged to be inappropriate is displayed in an identifiable manner compared to the in-process data or processed data that is the subject of the appropriateness judgment. A user of the data processing management unit 3300 can select any data from the inappropriateness judgment candidate data displayed in the appropriateness judgment result display unit 3322 via the user input acceptance unit 3323 and confirm the inappropriate data.

[0143] (A-1-7-3. Corresponding action determination unit 3330) The corresponding action determination unit 3330 determines a corresponding action or displays candidate corresponding actions based on the content of the data being processed or the processed data, or a combination of these, acquired from the data processing unit 3200. The corresponding action determination unit 3330 includes a determination processing unit 3331 and a corresponding action determination result display unit 3332. The determination processing unit 3331 determines a corresponding action to be taken, particularly when the data appropriateness determination unit 3320 determines that the data is inappropriate.

[0144] The response action determination unit 3330 determines a response action based on state quantities related to the data being processed and the processed data, and the determination result of the inappropriateness determination by the data appropriateness determination unit 3320. Similar to the determination processing unit 2351 of the data management system 2300, the response actions determined by the determination processing unit 3331 include, for example, not using the inappropriate data determined to be inappropriate, substituting other data for the inappropriate data, notifying of restrictions on the use of the processed data, notifying that the processed data contains inappropriate data, specifying a data processing method, re-performing sensing by an aircraft, specifying a data processing method according to the use specified by the user, and specifying a display method for the processed data according to the use specified by the user.

[0145] Here, when performing data processing identification No. 002, which performs SfM processing using multiple images, and doming of a predetermined value or more is detected due to an inappropriate judgment, the response action judgment unit 3330 judges that the response action is to take oblique images of the target area by "re-performing sensing using an aircraft" and to perform doming response processing by "specifying a data processing method."

[0146] The judgment criteria of the corresponding action judgment unit 3330 can be set based on input information from the user acquired by the user request acquisition unit 2310, but are not limited to this and may be set in advance by the system. Alternatively, the user may select a criterion to be applied by inputting a value into the selection field on the left side of T102 from default setting criteria for each target state quantity that are set in advance by the system, or may set an arbitrary condition by inputting a value into the corresponding action judgment field on the right side of T102.

[0147] As an example of criteria for determining a response action, the user may input the intended use of the processed data (such as for feature inspection or forest surveying), and the system may automatically generate criteria and results for determining a response action based on the intended use entered.

[0148] Next, the response action determination result display unit 3332 displays the determination result of the response action determined by the determination processing unit 3331. The content displayed by the response action determination result display unit 3332 may be similar to the content displayed by the response action determination result display unit 2352 of the data management system 2300, as shown in FIG. 12 . A user of the data processing management unit 3300 can select any inappropriate data from the multiple inappropriate data displayed by the response action determination result display unit 3332 via the user input accepting unit 3333, and the determination result of the response action corresponding to the selected inappropriate data is displayed. Furthermore, when any inappropriate data is selected, the above-mentioned response action determination result and other candidates are displayed so that they can be selected, and a confirmation button for the response action is also displayed. The user can select any response action via the user input accepting unit 3333, and with a specific response action selected, can confirm the response action by operating the confirmation button for the response action.

[0149] (A-1-7-4. Remeasurement condition determination unit 3340) The remeasurement condition determination unit 3340 is a functional unit that determines various conditions for performing remeasurement when the response action determined by the response action determination unit 3330 is remeasurement, displays candidate conditions, and accepts selection input for the candidate conditions from the user. The remeasurement condition determination unit 3340 includes a condition determination unit 3341, a determination condition display unit 3342, and a user input acceptance unit 3343.

[0150] The condition determination unit 3341 is a functional unit that determines the remeasurement condition, and has the same function as the condition determination unit 2361 of the data management system 2300 .

[0151] The judgment condition display unit 3342 is a functional unit that displays or outputs by voice the remeasurement conditions judged by the condition judgment unit 3341 , and has the same function as the condition judgment unit 3341 of the data management system 2300 .

[0152] The user input accepting unit 3343 is a functional unit that accepts approval input or correction input for the judgment result of the remeasurement conditions displayed in the judgment condition display unit 3342, and has the same function as the user input accepting unit 2363 of the data management system 2300.

[0153] (A-1-7-5. Response Action Command Unit 3350) Next, the response action command unit 3350 outputs an execution command for the response action determined by the response action determination unit 3330, causing the response action to be executed. Furthermore, if re-measurement is determined as the response action, the response action command unit 3350 outputs an execution command for re-measurement in accordance with the re-measurement condition determined by the re-measurement condition determination unit 3340, causing re-measurement to be executed. Furthermore, the response action command unit 3350 may transmit information on the determined response action or information on the re-measurement condition to an external system other than the data management system 2300, an external system other than the data processing base system 3000, or an external system other than the control system 1.

[0154] (A-1-7-6. Post-processing data display information generation unit 3360) The processed data display information generation unit 3360 generates display information for the processed data to be distributed to the user terminal device 5000 via the distribution system 4000 based on the response action taken in response to the suitability determination of the sensing data determined by the response action determination unit 2350 and the response action taken in response to the suitability determination of the data being processed or the processed data determined by the response action determination unit 3330.

[0155] As an example, when the response action determined by the response action determination unit 2350 or the response action determination unit 3330 is "notification of restrictions on the use of the processed data," the processed data display information generation unit 3360 generates display information that includes information restricting the use of the processed data. Similarly, when the response action is "notification that inappropriate data is included," the processed data display information generation unit 3360 generates display information that includes information that inappropriate data is included.

[0156] In addition, the processed data display information generation unit 3360 can generate display information of the processed data in map format, associating it with the inappropriate data determined to be inappropriate, or the ground position of the target position related to the inappropriate determination, or the position of the aircraft, or the control target position of the aircraft, or the control target route of the aircraft.

[0157] An example of display information of processed data generated by the processed data display information generation unit 3360 will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of display information generated by the processed data display information generation unit by associating the processed data with the terrestrial sensing target area.

[0158] 18, the processed data is displayed in map format, and information is displayed that identifies whether the sensing data is data obtained by initial measurement or re-measurement, in association with the sensing target area on the ground that corresponds to each piece of sensing data, and further displays information on the measurement date and time when each piece of data was acquired. In other words, display information for processed data is generated that is a mixture of processed data based on initial measurement data and processed data based on re-measurement data.

[0159] <Hardware configuration> (A-1-8. Hardware configuration) 19 is a diagram showing an example of the hardware configuration of a data management system 2300 and a data processing management unit 3300. Here, the data management system 2300 and the data processing management unit 3300 in the present invention are information processing devices such as a server device or a PC. As shown in the figure, the data management system 2300 and the data processing management unit 3300 have an input device 100, an output device 200, a processing device 300, a main memory device 400, an auxiliary memory device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0160] The input device 100 is a device that allows a user to input information and instructions to the data management system 2300 and the data processing management unit 3300, and can constitute the user request acquisition unit 2310 and the user input acceptance units 2344, 2353, 2363, 3323, 3333, and 3343. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or an audio input device such as a microphone.

[0161] The output device 200 is a device that outputs information generated by the data management system 2300 and the data processing management unit 3300, and can configure a suitability judgment result display unit 2343, a corresponding action judgment result display unit 2352, a judgment condition display unit 2362, a corresponding action display unit 3311, a suitability judgment result display unit 3322, a corresponding action judgment result display unit 3332, and a judgment condition display unit 3342. Specifically, the output device 200 is a display device (including eyewear, AR, VR display devices, etc.), a printer, or a speaker.

[0162] The processing device 300 is, for example, a device that performs arithmetic processing. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing arithmetic processing.

[0163] The main storage device 400 is a memory device such as a RAM that temporarily stores various types of read information and a ROM that stores programs, application programs, and other various information executed by the processing device 300. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory that can store digital information.

[0164] The communication device 600 is a device that performs wireless or wired information communication with an external device.

[0165] Note that Figure 19 describes the hardware configuration of the data management system 2300 and the data processing management unit 3300, but other systems within the data acquisition base system 2000 (communication infrastructure management system 2100, aircraft operation operating system 2200, operation management system 2500, airspace monitoring system 2600), other systems within the data processing base system 3000 (communication unit 3100, data processing unit 3200), the distribution system 4000, and the user terminal device 5000 can also be realized with a hardware configuration similar to that shown in Figure 19 above.

[0166] <Control flow> (A-1-9. Overview of the control flow of the control system) Next, the control flow of the control system will be explained with reference to the drawings. Fig. 20 is a flowchart showing the processing flow by the processing system 1. In the flowchart in this figure, the processing of the first half, from processing steps 101 to 109, is executed by the data management system 2300, and the processing of the second half, from processing steps 110 to 121, is executed by the data processing management unit 3300.

[0167] First, the user request information is acquired by the user request acquisition unit 2310 of the data management system 2300 (step 101). Note that this step may be performed before starting the sensing operation by the aircraft, or the user request may be acquired while the sensing operation is being performed.

[0168] Next, the sensing data acquisition unit 2320 and the status information acquisition unit 2330 of the flying object 1000 and the data management system 2300 acquire sensing data and status information from the flying object 1000 (step 102). As an example of the status information to be acquired, status information such as that shown in Table T103 in FIG. 10 is acquired.

[0169] Next, the data suitability determination unit 2340 of the data management system 2300 determines whether the pre-processed data is inappropriate based on the acquired sensing data, status information, and other information (step 103). Here, the data suitability determination in this step may be performed within the data acquisition base system 2000 after the aircraft has completed its flight for measurement operations, based on data received via wireless communication or read from the aircraft's sensing data recording unit 1500 and status data recording unit 1600. However, it is also possible to receive sensing data and status information in real time via wireless communication while the aircraft is performing measurement operations, and perform the data suitability determination in this step in parallel with the aircraft's measurement operations. This embodiment will be described later with reference to FIG. 25.

[0170] Next, as a result of the judgment of the pre-processing data by the data suitability judgment unit 2340, it is judged whether there is any inappropriate data judged to be inappropriate, and the processing step to transition to is determined according to the judgment result (step 104). If there is any inappropriate data, transition is made to step 105, and if there is no inappropriate data, transition is made to step 110.

[0171] Next, in step 105, the response action determination unit 2350 of the data management system 2300 determines the response action to be taken when the data is determined to be inappropriate (step 105). This step may be performed within the data acquisition base system 2000 after the flying object 1000 has completed its flight for sensing work. However, it is also possible to receive sensing data and status information in real time via wireless communication while the flying object is performing measurement work, and perform the response action determination in this step in parallel with the flying object's measurement work. This embodiment will be described later with reference to FIG. 25.

[0172] Next, the corresponding action determination unit 2350 determines whether or not remeasurement is required as a result of the corresponding action determination, and determines the processing step to transition to depending on the determination result (step 106). If it is determined that remeasurement is required, transition is made to step 107, and if it is determined that remeasurement is not required, transition is made to step 110.

[0173] Next, in step 107, the remeasurement condition determination unit 2360 of the data management system 2300 determines whether the remeasurement condition is met (step 107).

[0174] Next, the remeasurement condition determination unit 2360 of the data management system 2300 displays the determined remeasurement conditions on the determination condition display unit 2362, and accepts approval or correction input for the displayed remeasurement conditions from the user via the user input acceptance unit 2363, thereby finalizing the remeasurement conditions (step 108). The display contents of the remeasurement conditions in this step can be, for example, as shown in Fig. 14.

[0175] Next, the response action command unit 2370 of the data management system 2300 outputs a command to perform remeasurement based on the determined remeasurement conditions, and executes remeasurement (step 109). Here, the remeasurement command can be transmitted to, for example, a pilot who operates the aircraft using the aircraft flight operating system 2200, or to on-site personnel who operate the aircraft.

[0176] Next, in step 110, the response action display unit 3311 of the data processing management unit 3300 displays the response action determined by the response action determination unit 2350 to the user (i.e., the data processing staff) (step 110). The display content of the response action in this step can be, for example, as shown in Fig. 15 or 16.

[0177] Next, the data processing command determining section 3312 of the data processing management section 3300 determines the content of the data processing to be executed by the data processing section 3200, and commands the data processing section 3200 to execute the data processing (step 111).

[0178] Next, the data suitability determination unit 3320 of the data processing management unit 3300 determines whether the data being processed is suitable (step 112).

[0179] Next, the data suitability determination unit 3320 of the data processing management unit 3300 determines the suitability of the processed data generated by the data processing by the data processing unit 3200 (step 113).

[0180] Next, as a result of the data suitability determination by the data suitability determination unit 3320 of the data processing management unit 3300, it is determined whether or not there is inappropriate data determined to be inappropriate in the data being processed or the processed data, and the step to transition to is determined according to the determination result (step 114). If there is inappropriate data, the process transitions to processing step 115, and if there is no inappropriate data, the process transitions to processing step 120.

[0181] Next, in step 115, the corresponding action determination unit 3330 of the data processing management unit 3300 determines the corresponding action to be taken when the data in process or the processed data is determined to be inappropriate (step 115).

[0182] Next, the corresponding action determination unit 3330 determines whether or not remeasurement is required as a result of the corresponding action determination, and determines the processing step to transition to depending on the determination result (step 116). If it is determined that remeasurement is required, the process transitions to step 117, and if it is determined that remeasurement is not required, the process transitions to step 120.

[0183] Next, in step 117, the remeasurement condition determination unit 3340 of the data processing management unit 3300 determines whether the remeasurement condition is met (step 117).

[0184] Next, the remeasurement condition determination unit 3340 of the data processing management unit 3300 displays the determined remeasurement conditions on the determination condition display unit 3342, and accepts approval or correction input for the displayed remeasurement conditions from the user via the user input acceptance unit 3343, thereby finalizing the remeasurement conditions (step 118). The display contents of the remeasurement conditions in this step can be, for example, as shown in Fig. 14.

[0185] Next, the corresponding action command unit 3350 of the data processing management unit 3300 outputs a command to perform remeasurement based on the determined remeasurement conditions, and executes remeasurement (step 119). Here, the remeasurement command can be transmitted to the aircraft operation system 2200, which operates the aircraft using the aircraft operation system 2200, or to on-site personnel operating the aircraft.

[0186] Next, in step 120, the processed data display information generation unit 3360 of the data processing management unit 3300 generates display information for the processed data to be distributed to the user terminal device 5000 via the distribution system 4000 based on the response action in response to the judgment on the suitability of the sensing data determined by the response action judgment unit 2350 and the response action in response to the judgment on the suitability of the data being processed or the processed data determined by the response action judgment unit 3330 (step 120).

[0187] (A-1-10. Processing flow of data suitability determination by the data management system 2300) FIG. 21 is a flowchart showing the processing flow when the data appropriateness determination unit 2340 of the present invention determines the appropriateness of data before processing.

[0188] First, the data analysis unit 2341 determines whether there is any missing data in the sensing data acquired from the sensing data acquisition unit 2320 (step 201). For example, if the sensing operation is not being performed and no sensing data can be obtained, if the number of pieces of sensing data is clearly small, or if the lens cover remains closed and no ground image can be captured, it can be determined that the data is inappropriate due to missing data.

[0189] Next, the data analysis unit 2341 determines whether the data is appropriate based on the content of the sensing data before processing (step 202). For example, as shown in Fig. 8, if there are water droplets, condensation, or dirt on the lens when the camera image is acquired, or if there is a defective area in the camera image where the ground is not captured due to clouds or fog, etc., a determination of inappropriateness can be made based on the sensing data.

[0190] Next, the state information analysis unit 2342 determines whether the aircraft is suitable or not based on its state (step 203). For example, as shown in Fig. 8, the determination can be made based on state quantities such as the position, flight speed, flight altitude, and attitude of the aircraft.

[0191] Next, the state information analysis unit 2342 determines whether the sensor is suitable or not based on the measurement control state of the sensor (step 204). For example, as shown in Fig. 8, the determination can be made based on the sensor settings such as the resolution, focus, and brightness of the camera.

[0192] Next, it is determined whether or not there was an inappropriate judgment as a result of the suitability judgments made in steps 201 to 204, and a process to transition to is determined based on the judgment result (step 205). If there is an inappropriate judgment, transition is made to step 206, and if there is no inappropriate judgment, transition is made to step 208.

[0193] Next, the inappropriate data that has been judged to be inappropriate and the details of the inappropriate judgment are displayed by the appropriateness judgment result display unit 2343 (step 206). For example, the inappropriate data as shown in Fig. 12, the reason for the inappropriateness judgment for the inappropriate data, the degree of exceedance of the reference value, the sensing time, the location of the target ground area, etc. are displayed.

[0194] Next, the user input receiving unit 2353 receives a confirmation input from the user for the displayed inappropriateness determination result (step 207). That is, by operating the confirmation button for inappropriate data as shown in Fig. 11, the inappropriateness determination result is confirmed for the sensing data determined to be inappropriate data by the inappropriateness determination and the content of the determination.

[0195] Next, the display device displays the sensing data, including the sensing data that has not been determined to be inappropriate in the data appropriateness judgment, to the user, regardless of the result of the appropriateness judgment (step 208).

[0196] Next, the user input receiving unit 2353 receives a selection input of sensing data to be determined as inappropriate from the user (step 209). In steps 208 and 209, the user can manually determine as inappropriate sensing data even for sensing data that has not been determined as inappropriate by the data appropriateness determining unit 2340, and input the data into the system.

[0197] Next, the data appropriateness judgment unit 2340 determines the inappropriate data and the content of the inappropriateness judgment based on the input information obtained from the user in steps 207 and 209, and records the determined inappropriate data and the content of the inappropriateness judgment (step 210).

[0198] (A-1-11. Processing flow of response action determination by the data management system 2300) FIG. 22 is a flowchart showing the processing flow performed by the corresponding action determining unit 2350 of the present invention when determining a corresponding action to be taken on pre-processing data.

[0199] First, the determination processing unit 2351 determines an action to be taken based on the data content of the sensing data acquired from the sensing data acquisition unit 2320 (step 301). For example, as shown in Fig. 9, it is possible to determine a corresponding action depending on each state, such as whether there is data loss, or if there is a defective area in the camera image where the ground is not captured due to water droplets, condensation, or dirt on the lens when the camera image was acquired, or clouds or fog.

[0200] Next, the determination processing unit 2351 determines a corresponding action based on the flight control status of the aircraft (step 302). For example, as shown in Fig. 9, a corresponding action can be determined according to state quantities such as the position, flight speed, flight altitude, and attitude of the aircraft.

[0201] Next, the determination processing unit 2351 determines a corresponding action based on the measurement control state of the sensor (step 303). For example, as shown in Fig. 9, a corresponding action can be determined according to sensor setting values ​​such as the camera resolution, focus, and brightness.

[0202] Next, the response action determination result display unit 2352 displays the results of the response action determination made in steps 301 to 303 (step 304). For example, the inappropriate data, the reason for the inappropriate determination, the response action for the inappropriate data, etc., as shown in Fig. 12 are displayed.

[0203] Next, the user input accepting unit 2353 accepts a confirmation input from the user for the displayed response action determination result (step 305). That is, by operating a response action confirmation button as shown in Fig. 12, a confirmation input for the response action determined by the response action determination is accepted. Alternatively, regardless of the response action determination result, a selection input of a specific response action from a plurality of predetermined response action candidates may be accepted from the user.

[0204] Next, the corresponding action determination unit 2350 determines the corresponding action based on the input information acquired from the user in step 305, and records the determined corresponding action (step 306).

[0205] Note that, in Figure 22, the processing flow of the response action determination unit 2350 of the data management system 2300 is explained, but in the processing flow of the response action determination unit 3330 of the data processing management unit 3300, steps 302 and 303 in Figure 22 can be omitted.

[0206] (A-1-12. Processing flow of determining remeasurement conditions by the data management system 2300) Next, FIG. 23 is a flowchart showing the processing flow performed by the remeasurement condition determination unit 2360 when determining whether the remeasurement condition is met.

[0207] First, the condition determination unit 2361 of the remeasurement condition determination unit 2360 determines the equipment and devices used for measurement (step 401). For example, based on the content of the data suitability determination by the data suitability determination unit 2340, the type of sensor used for measurement (optical camera, LiDAR, other laser sensor, etc.), in the case of an optical camera, the sensor performance such as resolution and lens focal length, or the type of aircraft (fixed-wing aircraft, multicopter aircraft, VTOL aircraft, etc.) is determined.

[0208] Next, the condition determination unit 2361 of the re-measurement condition determination unit 2360 determines the setting parameters of the measurement sensor used for measurement (step 402). For example, if the measurement sensor is an optical camera, the focus, aperture (f-number), exposure, gain, contrast, and shutter speed are set.

[0209] Next, the condition determination unit 2361 of the re-measurement condition determination unit 2360 determines the flight conditions of the aircraft used for measurement (step 403). For example, the flight path, flight altitude, flight speed, flight attitude angle, etc. of the aircraft are set. Note that, when setting a flight path with a round-trip route, the setting of the flight path can include setting the width between adjacent routes of the round-trip route and the route length of the flight direction of the round-trip route.

[0210] (A-1-13. Processing flow of suitability determination in the data processing management unit 3300) FIG. 24 is a flowchart showing the processing flow performed when the data appropriateness determining unit 3320 in the data processing management unit of the present invention determines whether data is appropriate.

[0211] First, the data analysis unit 3321 judges whether the data being processed obtained from the data processing unit 3200 is appropriate or not based on the data content (step 501).

[0212] Next, the data analysis unit 3321 judges the suitability of the processed data obtained from the data processing unit 3200 based on the data content (step 502).

[0213] Next, the data appropriateness determination unit 3320 determines whether or not there was an inappropriate determination as a result of the appropriateness determination made in steps 501 and 502, and determines the process to transition to based on the determination result (step 503). If there is an inappropriate determination, transition is made to step 504, and if there is no inappropriate determination, transition is made to step 506.

[0214] Next, the inappropriate data that was judged to be inappropriate and the details of the inappropriate judgment are displayed by the appropriateness judgment result display unit 3322 (step 504). For example, the inappropriate data as shown in Fig. 11, the reason for the inappropriateness judgment for the inappropriate data, the degree of exceedance of the standard value, the sensing time, the location of the target ground area, etc. are displayed.

[0215] Next, the user input receiving unit 3323 receives a confirmation input from the user for the displayed inappropriateness determination result (step 505). That is, by operating the confirmation button for inappropriate data as shown in Fig. 11, the inappropriateness determination result is confirmed for the data in process or the processed data that has been determined to be inappropriate data by the inappropriateness determination and the contents of the determination.

[0216] Next, regardless of the result of the suitability judgment, the display device displays to the user the data in process and the processed data that have not been judged to be inappropriate in the data suitability judgment (step 506).

[0217] Next, the user input receiving unit 3323 receives a selection input from the user of the data in process or the processed data that should be determined to be inappropriate (step 507). For data that has not been determined to be inappropriate in steps 501 and 502, the user can manually determine that the data is inappropriate and input the result into the system.

[0218] Next, the data appropriateness determination unit 3320 determines the inappropriate data and the content of the inappropriateness determination based on the input information acquired from the user in step 507, and records the determined inappropriate data and the content of the inappropriateness determination (step 508).

[0219] (A-1-13. Processing flow when determining whether data is appropriate during measurement) Next, a process flow will be described for when the data suitability determination unit 2340 determines whether data is appropriate during measurement by the flying object. Fig. 25 is a flowchart showing the process flow for when the data suitability determination unit 2340 determines whether data is appropriate during measurement by the flying object. This flowchart particularly shows a modified example of steps 101 to 109 shown in Fig. 20.

[0220] First, the user request acquisition unit 2310 of the data management system 2300 acquires user request information (step 601). Note that this step may be performed before starting sensing work by the aircraft, or the user request may be acquired while the sensing work is being performed.

[0221] Next, the sensing data acquisition unit 2320 and the status information acquisition unit 2330 of the flying object 1000 and the data management system 2300 acquire sensing data and status information from the flying object 1000 (step 602). As an example of the status information to be acquired, status information such as that shown in Table T103 in FIG. 10 is acquired.

[0222] Next, the data suitability determination unit 2340 of the data management system 2300 determines whether the pre-processed data is inappropriate based on the acquired sensing data, status information, and other information (step 603). Here, sensing data and status information are received in real time via wireless communication during measurement work by the aircraft, and data suitability determination in this step is performed in parallel with the measurement work by the aircraft.

[0223] Next, as a result of the judgment of the pre-processing data by the data suitability judgment unit 2340, it is judged whether or not there is inappropriate data judged to be inappropriate, and the processing step to transition to is determined according to the judgment result (step 604). If there is inappropriate data, the process transitions to step 605, and if there is no inappropriate data, the process transitions to step 110 shown in Figure 20.

[0224] Next, in step 605, the corresponding action determination unit 2350 determines a proposed change in the measurement conditions of the measurement sensor mounted on the flying object during flight (step 605). For example, in this step, a change in the measurement parameters of the sensor during flight, a switch in the sensor, etc. are determined as a proposed change in the measurement conditions.

[0225] Next, the corresponding action determination unit 2350 of the data management system 2300 displays the determined proposed changes to the measurement conditions on the determination condition display unit 2362, and accepts approval or correction input for the displayed proposed changes to the measurement conditions from the user via the user input acceptance unit 2353, thereby finalizing the proposed changes to the measurement conditions (step 606).

[0226] Next, the corresponding action command unit 2370 sends a command to change the measurement conditions to the flying object 1000 or the aircraft flight operating system 2200, causing measurement to be performed under the changed measurement conditions (step 607).

[0227] Next, the data suitability determination unit 2340 determines whether the sensing data measured under the changed measurement conditions is suitable (step 608).

[0228] Next, as a result of the judgment of the pre-processing data by the data suitability judgment unit 2340, it is judged whether or not there is inappropriate data judged to be inappropriate, and the processing step to transition to is determined according to the judgment result (step 609). If there is inappropriate data, the process transitions to step 610, and if there is no inappropriate data, the process transitions to step 110 shown in Figure 20.

[0229] Next, in step 610, the measurement by the aircraft is stopped (step 610). When the measurement is stopped in this step, the flight and measurement according to the original flight mission may be stopped and the aircraft may be landed once, and then taken off again when the measurement is to be retaken, or the flight and measurement according to the original flight mission may be stopped and the aircraft may wait in the sky by a waiting flight without landing, and then the aircraft may start flying according to a newly generated mission for remeasurement when the measurement is to be retaken.

[0230] Steps 611 to 615 in this flowchart are the same as steps 105 to 109 in FIG. 20, and therefore their explanation will be omitted.

[0231] (A-1-14. Processing flow for determining suitability when SfM processing is performed in the data processing unit 3200) 26 is a flowchart showing the processing flow for data processing and data suitability determination when the data processing center system of the present invention executes SfM processing. Using this diagram, we will explain each processing step when executing SfM processing in the data processing unit 3200, and an example of how the data suitability determination unit 3320 determines suitability based on multiple pieces of data being processed.

[0232] First, the data processing unit 3200 detects feature points from an image and estimates the three-dimensional distribution of the feature points (step 701).

[0233] Next, the data appropriateness determination unit 3320 determines whether or not feature points were extracted in step 701, and determines the process to transition to based on the determination result (step 702). If feature points were extracted, the process transitions to step 703, and if feature points were not extracted, the process transitions to step 712, where it is determined that the data is inappropriate.

[0234] Next, the data processing unit 3200 determines tie points, which are points that can be recognized from both of the multiple images, from the image feature points (step 703).

[0235] Next, the data processing unit 3200 estimates the position and attitude of the aircraft at the time of photographing as exterior orientation parameters (step 704).

[0236] Next, the data processing unit 3200 estimates the lens distortion as an internal orientation parameter (step 705).

[0237] Next, the data suitability determination unit 3320 determines whether or not the parameters (aperture (f-number), focus amount, etc.) used to estimate the lens distortion in step 705 have changed, and determines the process to transition to based on the determination result (step 706). If the parameters have not changed, the process transitions to step 707, and if the parameters have changed, the process transitions to step 712, where it is determined that the data is inappropriate.

[0238] Next, the data processing unit 3200 detects ground control points (GCPs) and accuracy verification points (CPs) (step 707).

[0239] Next, the data suitability determination unit 3320 determines whether or not ground control points (GCPs) and accuracy verification points (CPs) were detected in the processing step of step 707, and determines the processing to transition to based on the determination result (step 708). If ground control points (GCPs) and accuracy verification points (CPs) were detected, the process transitions to step 709, and if ground control points (GCPs) and accuracy verification points (CPs) could not be detected, the process transitions to step 712, where it is determined that the data is inappropriate.

[0240] Next, the data processing unit 3200 performs a doming countermeasure process (step 709). The doming countermeasure process will be described in detail later.

[0241] Next, the data processing unit 3200 constructs a point cloud model from the image feature points and a plurality of stereo pair images (step 710).

[0242] Next, the data appropriateness determining unit 3320 determines that the data is appropriate (step 710).

[0243] (A-1-15. Explanation of doming prevention measures) The concept of the doming countermeasure processing executed in step 709 of the flowchart shown in Fig. 26 will be briefly explained with reference to Fig. 27. Fig. 27 is a diagram showing an example of the doming countermeasure processing executed by the data processing unit of the present invention.

[0244] As shown in the upper diagram of Fig. 27, doming in this embodiment refers to an error that occurs when SfM processing is performed using multiple nadir images taken from the sky toward the ground, causing the shape of the ground surface estimated by SfM processing to be more curved than the actual shape of the ground surface. Such doming occurs when SfM processing is performed using only nadir images or when there is an error in the interior orientation parameters.

[0245] As a countermeasure against doming that takes such doming into account, SfM processing is performed using multiple images, including a nadir image and an image facing diagonally downward, which can reduce the estimation error of the ground surface shape due to doming, as shown in the lower diagram of Figure 27.

[0246] In each of the above-described embodiments, the data management system 2300 is implemented within the data acquisition base system 2000, but all or part of the functions of the data management system 2300 can also be implemented in the aircraft 1000, the data processing base system 3000, the distribution system 4000, or other systems.

[0247] The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

[0248] [A-2. Effects of this embodiment] In the above-described embodiment, flight conditions and measurement conditions for remeasurement can be appropriately set, particularly when there are deficiencies in the measurement data, so that measurement results can be quickly provided to the user using the user terminal device 5000. Furthermore, since there is a function to determine whether there are deficiencies in the measurement data, the risk that the user will overlook deficiencies in the measurement data can be reduced. [Explanation of symbols]

[0249] 1...Control system 100...Input device 200...output device 300...processing device 400…Main storage device 500…Auxiliary storage device 600...Communication equipment 700...Bus 1000...Flying object 1100...Flight section 1110...Self position / speed determination section 1120: Attitude determination unit 1130: Flight control unit 1200: Sensing unit 1210: Sensor 1220: Sensor attitude control unit 1230: Sensor control unit 1300...Communication unit 1310...Control communication unit 1320...Data communication unit 1330...Status data communication unit 1400...State determination unit 1410...Flight state determination unit 1420: Sensing state determination unit 1430: Communication state determination unit 1500...Sensing data recording unit 1600...Status data recording unit 1610...Flight control information recording unit 1620... Measurement control information recording unit 1630... Status determination result recording unit 2000...Data acquisition base system 2100...Communication Infrastructure Management System 2200...Aircraft Operation System 2300...Data Management System 2310...User request acquisition unit 2320: Sensing data acquisition unit 2330...Status information acquisition unit 2340: Data suitability determination unit 2341: Data analysis unit 2342...Status information analysis unit 2343...Property judgment result display unit 2344...User input reception unit 2350: Corresponding action determination unit 2351: Determination processing unit 2352…Response action determination result display section 2353...User input reception section 2360... Remeasurement condition determination unit 2361... Condition determination unit 2362: Determination condition display section 2363: User input reception section 2370...Response Action Command 2500...Flight Management System 2600...Airspace Monitoring System 3000...Data processing base system 3100...Communication unit 3200...Data processing unit 3300...Data Processing Management Department 3310: Data processing execution command unit 3311: Corresponding action display unit 3312...Data processing command determination unit 3320: Data suitability determination unit 3321: Data analysis unit 3322: Appropriateness judgment result display unit 3323: User input reception unit 3330: Corresponding action determination unit 3331: Determination processing unit 3332…Response action determination result display section 3333...User input reception unit 3340... Remeasurement condition determination unit 3341... Condition determination unit 3342: Determination condition display section 3343: User input reception section 3350...Response Action Command 3360...Processed data display information generation unit 4000...Distribution system 4100: Communications department 4200: Distribution data management department 5000...User terminal device

Claims

1. a data acquisition unit that acquires at least one of the following data to be determined: sensing data including an image or point cloud information of a target area acquired using a sensor mounted on an aircraft; in-process data generated in the process of data processing of the sensing data; and processed data generated by data processing of the sensing data; a data suitability determination unit that determines suitability of the judgment target data acquired by the data acquisition unit; a remeasurement condition determination unit that determines a condition for remeasurement of the sensing data using the aircraft according to a result of the suitability determination by the data suitability determination unit, A control system comprising at least one of a command unit that executes remeasurement based on the remeasurement condition determined by the remeasurement condition determination unit, a transmission unit that transmits the remeasurement condition to an external system, or an output unit that displays or outputs the remeasurement condition by voice.

2. 2. The control system of claim 1, The re-measurement condition determination unit determines the re-measurement condition according to the determination result of the data suitability determination unit.

3. 2. The control system of claim 1, A control system, wherein the remeasurement conditions determined by the remeasurement condition determination unit include flight conditions related to at least one of the flight path, flight altitude, flight speed, and flight attitude angle of the aircraft.

4. 2. The control system of claim 1, A control system in which the re-measurement conditions determined by the re-measurement condition determination unit include setting conditions regarding at least one of the exposure, exposure, gain, contrast, focus, aperture, and shutter speed of the optical camera when an optical camera is used as the sensor.

5. 2. The control system of claim 1, A control system, wherein the remeasurement conditions determined by the remeasurement condition determination unit include at least one of the type of the sensor and the type of the flying object.

6. 2. The control system of claim 1, The re-measurement condition determination unit determines the re-measurement condition in accordance with a preset determination criterion in addition to the result of suitability determination by the data suitability determination unit when determining the re-measurement condition.

7. 7. The control system according to claim 2, When the remeasurement condition is output from a display device or an audio output device based on the remeasurement condition determined by the remeasurement condition determination unit, a re-measurement condition input receiving unit that receives user input including changes or approvals to the output re-measurement conditions;

8. 2. The control system of claim 1, a corresponding action determination unit that determines a corresponding action according to a result of the suitability determination by the data suitability determination unit; A control system in which the command unit executes the response action, the transmission unit transmits information about the response action to the outside, or the output unit displays or outputs the response action by voice.

9. 9. The control system of claim 8, The response action determination unit determines the response action depending on the content of the suitability determination made by the data suitability determination unit.

10. 9. The control system of claim 8, The response action determination unit determines the response action depending on the result of the suitability determination by the data suitability determination unit and the timing of execution of the suitability determination.

11. 11. The control system of claim 10, A control system in which, when the timing of executing the suitability judgment is during measurement by the aircraft, the response action judgment unit judges that the response action will be at least one of stopping the measurement, changing the measurement parameters of the sensor during flight, or switching the sensor.

12. 12. The control system of claim 11, The command unit executes the corresponding action of changing the measurement parameters of the sensor or switching the sensor during flight, and if the sensing data measured after the measurement parameter change or sensor switching is determined to be inappropriate by the data suitability determination unit, the control system stops the measurement.

13. 9. The control system of claim 8, A control system in which the response action determined by the response action determination unit includes at least one of remeasurement, denial of permission to use the sensing data, substitution of other data for the sensing data, notification to the user that the sensing data contains inappropriate data determined to be inappropriate, and specification of a data processing method for the sensing data.

14. 9. The control system of claim 8, a usability determination criterion receiving unit that receives, from a user, a determination criterion for permitting use of the inappropriate data determined to be inappropriate by the data appropriateness determining unit; A control system in which, when the data suitability determination unit determines that the data to be determined is inappropriate, the output unit displays or outputs the inappropriate data as usable based on the information received by the usability determination criteria receiving unit, or the transmitting unit transmits the inappropriate data to the outside.

15. 9. The control system of claim 8, an inappropriate data use condition receiving unit that receives from a user a use condition for the inappropriate data that permits use of the inappropriate data determined to be inappropriate by the data appropriateness determining unit; A control system in which, when the data suitability determination unit determines that the data to be determined is inappropriate, the command unit executes the response action determined based on the information received by the inappropriate data usage conditions reception unit, or the output unit displays or outputs the action as sound, or the transmission unit transmits the action to the outside.

16. 9. The control system of claim 8, a response action judgment criterion receiving unit that receives, from a user, a judgment criterion for determining the response action according to the result of the suitability judgment by the data suitability judgment unit; A control system in which the response action determined in accordance with the judgment criteria received by the response action judgment criteria receiving unit is executed by the command unit, or is displayed or output as audio by the output unit, or is transmitted to the outside by the transmission unit.

17. 2. The control system of claim 1, A control system in which the suitability determination by the data suitability determination unit is performed automatically in accordance with suitability determination criteria, or based on information on the suitability determination result received from a user.

18. 2. The control system of claim 1, A control system in which the suitability judgment by the data suitability judgment unit is automatically made in accordance with suitability judgment criteria regarding the flight control state of the aircraft or the measurement control state by the sensor.

19. 2. The control system of claim 1, A control system in which the suitability judgment by the data suitability judgment unit is automatically made based on the results of a comparison with a control target value or past control actual value regarding the flight control state of the aircraft or the measured control state by the sensor.

20. 2. The control system of claim 1, A control system in which the suitability determination by the data suitability determination unit is automatically determined in accordance with suitability determination criteria regarding the data content of at least one of the sensing data, the data in process, and the processed data.

21. 2. The control system of claim 1, A control system in which the suitability judgment by the data suitability judgment unit is automatically made based on the reference value of a predetermined parameter related to the data content of at least one of the sensing data, the data being processed, and the processed data, or the result of comparison with the predetermined parameter of past data for the same target area.

22. The computer a data acquisition step of acquiring data to be determined, which includes sensing data including an image or point cloud information of a target area acquired using a sensor mounted on an aircraft, data in process generated in the process of data processing of the sensing data, and / or post-processing data generated by data processing of the sensing data; a data suitability determination step of determining suitability of the acquired judgment target data; a remeasurement condition determination step of determining a condition for remeasurement of the sensing data using the aircraft according to the result of the suitability determination; a step of executing a remeasurement based on the determined remeasurement condition, or outputting the remeasurement condition from a display device or an audio output device; A control method for performing

23. On the computer, a data acquisition command to acquire data to be determined, which is at least one of sensing data including an image or point cloud information of a target area acquired using a sensor mounted on an aircraft, in-process data generated in the process of data processing of the sensing data, and processed data generated by the data processing of the sensing data; a data suitability determination command for determining suitability of the acquired judgment target data; a remeasurement condition determination command for determining a condition for remeasurement of the sensing data using the aircraft according to the result of the suitability determination; a command to execute remeasurement based on the determined remeasurement condition, or a command to output the remeasurement condition from a display device or an audio output device; A control program that executes the above.

Images