Information processing device, control method, and program

Abstract

To provide an information processing device, control method, and program capable of preventing a human error from being made by an operator at a time of deciding takeoff.SOLUTION: A management server MS displays on a drone monitoring screen an operation button (for example, a takeoff decision button BO3 or takeoff determination button BO4) for use in receiving a takeoff permission of a drone Dn from an operator OPm, determines whether the decision timing for takeoff permission of the drone Dn to be given by the operator OPm has come, and controls the operator button so as to prevent the drone Dn from receiving the takeoff permission prior to the decision timing. When the drone Dn is in a flyable condition after the decision timing has come, the management server controls the operation button so that the takeoff permission of the drone Dn can be received.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to the technical field of systems such as those that allow an operator to monitor multiple unmanned aerial vehicles. [Background technology]

[0002] Conventionally, flight management systems for managing the flights of unmanned aircraft such as drones have been known. For example, the flight management system disclosed in Patent Document 1 calculates pre-flight risks before the flight of the unmanned aircraft, outputs the processing results via an input / output interface, and receives commands. This allows the flight manager or pilot of the unmanned aircraft to take action based on the pre-flight risks. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-24475 Summary of the Invention [Problem to be solved by the invention]

[0004] As services develop in the future, it is expected that one operator will make takeoff decisions for multiple unmanned aircraft, taking into account the pre-flight risks for each. In this case, the burden on the operator will increase, making it difficult to make appropriate takeoff decisions taking into account the pre-flight risks for each aircraft, raising concerns about the occurrence of human error.

[0005] Therefore, the present invention has been made in consideration of the above points and has as one example an object thereof to provide an information processing device, a control method, and a program that can prevent human error when an operator makes a takeoff decision. [Means for solving the problem]

[0006] (Application Example 1) In order to solve the above problem, the information processing device of this application example is characterized by comprising: a display control means for displaying an operation button for receiving takeoff permission for the unmanned aircraft from an operator on the screen of a terminal used by the operator; a first judgment means for judging whether the timing for the operator to decide whether to allow the unmanned aircraft to take off has arrived; and a button control means for controlling the operation button so that takeoff permission for the unmanned aircraft cannot be accepted before the judgment timing arrives, while controlling the operation button so that takeoff permission for the unmanned aircraft can be accepted after the judgment timing arrives if the unmanned aircraft is in a state capable of flight.

[0007] (Application Example 2) The control method according to this application example is characterized by including the steps of: displaying an operation button for receiving takeoff permission for the unmanned aircraft from an operator on the screen of a terminal used by the operator; determining whether the timing for the operator to decide whether to allow the unmanned aircraft to take off has arrived; and controlling the operation button so that takeoff permission for the unmanned aircraft cannot be accepted before the decision timing arrives, while controlling the operation button so that takeoff permission for the unmanned aircraft can be accepted after the decision timing arrives if the unmanned aircraft is in a flyable state.

[0008] (Application Example 3) The program of this application example is characterized by causing a computer included in a terminal used by an operator to execute the following steps: displaying an operation button on the screen of the terminal used by the operator for accepting takeoff permission for the unmanned aircraft from the operator; determining whether the timing for the operator to decide whether to allow the unmanned aircraft to take off has arrived; and controlling the operation button so that takeoff permission for the unmanned aircraft cannot be accepted before the judgment timing arrives, while controlling the operation button so that takeoff permission for the unmanned aircraft can be accepted after the judgment timing arrives if the unmanned aircraft is in a state capable of flight. [Effects of the Invention]

[0009] According to the present invention, it is possible to prevent human error when an operator makes a decision to take off. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a remote monitoring system S. [Figure 2] FIG. 1 is a diagram illustrating an example of the general configuration of a drone Dn. [Figure 3] FIG. 2 is a diagram illustrating an example of a schematic configuration of an operator terminal Tm. [Figure 4] FIG. 10 is a diagram showing an example of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 5] FIG. 10 is a diagram showing an example of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 6] FIG. 10 is a diagram showing an example of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 7] FIG. 10 is a diagram showing an example of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 8] FIG. 10 is a diagram showing an example of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 9] FIG. 10 is a diagram showing an example of a pop-up screen that is displayed when the takeoff decision button BO3 displaying "Takeoff Planned" is selected. [Figure 10] FIG. 2 is a diagram illustrating an example of a schematic configuration of a management server MS. [Figure 11] FIG. 2 is a diagram illustrating an example of functional blocks in a control unit 33. [Figure 12] 10 is a flowchart showing an example of a monitoring information display process executed by a control unit 33 of a management server MS. [Figure 13] 10 is a flowchart showing an example of an abnormality determination process executed by a control unit 33 of a management server MS. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described below with reference to the drawings. The following embodiment is an embodiment in which the present invention is applied to a remote monitoring system capable of remotely monitoring drones. Note that in this embodiment, a drone will be used as an example of an unmanned aerial vehicle, but the present invention can also be applied to flying robots other than drones.

[0012] [ 1. Overview of the configuration and operation of the remote monitoring system S ]

[0013] First, with reference to Fig. 1, an overview of the configuration and operation of a remote monitoring system S according to this embodiment will be described. Fig. 1 is a diagram showing an example of the schematic configuration of the remote monitoring system S. As shown in Fig. 1, the remote monitoring system S is configured to include a plurality of drones Dn (n = 1, 2, 3, etc.), a plurality of operator terminals Tm (m = 1, 2, etc.), and a management server MS (an example of an information processing device and a predetermined server). The drones Dn, the operator terminals Tm, and the management server MS are each connected to a communication network NW. The communication network NW is configured, for example, by the Internet, a mobile communication network, its wireless base stations, etc.

[0014] The drone Dn is an example of an unmanned aerial vehicle, also known as a multicopter or UAV. The drone Dn takes off in response to a takeoff command from a GCS (Ground Control Station) and is capable of flying autonomously, and is used for, for example, delivery, surveying, photography, and monitoring. The GCS is installed as an application on, for example, an operator terminal Tm, and is designed to link with a management server MS. The drone can also fly under remote control from the ground using a control terminal (equipped with a GCS) used by the operator.

[0015] Drone Dn is under the jurisdiction of one of multiple drone bases Bm (in other words, it belongs to one of the drone bases Bm). Drone base Bm is a base (e.g., a facility) from which drone Dn can take off and land. In the example of FIG. 1, drones D1 to D4 are each under the jurisdiction of drone base B1, and are configured to depart from and return to drone base B1. Drones D5 to D9 are each under the jurisdiction of drone base B2, and are configured to depart from and return to drone base B2. However, one drone Dn may be under the jurisdiction of multiple drone bases Bm. For example, drone D11 (not shown) may depart from drone base B2 and return to drone base B1. The number of drones Dn under the jurisdiction of one drone base Bm is not particularly limited.

[0016] Furthermore, at the drone base Bm, a port Pm used for takeoff and landing of the drone Dn and a base device Em (an example of a device) used to monitor the drone Dn are installed. At the drone base Bm, base staff (manually) perform a pre-flight inspection (aircraft condition check) of the drone Dn. For example, the base staff visually inspects a predetermined part of the drone Dn for each inspection item in the manual inspection, or inspects the drone Dn by touching a predetermined part of the drone Dn. Then, manual inspection result information (an example of base information) indicating the results of the inspection by the base staff is transmitted from the base staff's terminal (an example of a device), such as a smartphone, to the management server MS via the communication network NW. Furthermore, the drone Dn that has undergone the pre-flight inspection is placed at the port Pm and takes off (departs) from the port Pm according to a predetermined drone schedule. Furthermore, the drone Dn that returns to the drone base Bm lands at the port Pm. Note that multiple ports Pm may be installed at one drone base Bm.

[0017] The base device Em is connected to the communication network NW and includes a base camera (e.g., an RGB camera or an infrared camera) for monitoring the drone Dn. The base camera stores a base ID (identification information) for identifying the drone base Bm. The base camera, for example, continuously captures images of the drone Dn placed at the port Pm. Base image information (an example of base information) representing base images (still images or moving images) captured by the base camera is transmitted from the base device Em to the management server MS along with the base ID. Note that multiple base cameras may be installed at one drone base Bm. The base device Em may also be equipped with a wind sensor that detects (measures) at least one of wind speed and wind direction. The measurement information (an example of base information) measured by the wind sensor is transmitted from the base device Em to the management server MS. The base device Em may also be equipped with at least one of a temperature sensor, a humidity sensor, a precipitation (snow) amount sensor, and an atmospheric pressure sensor. The measurement information (an example of base information) measured by these sensors is transmitted from the base device Em to the management server MS.

[0018] The operator terminal Tm is a terminal used by an operator OPm who remotely monitors multiple drones Dn. The operator OPm monitors the drones Dn under his / her care while viewing information displayed on the screen (user interface screen) of the operator terminal Tm. Such monitoring includes, for example, checking the pre-flight inspection status of the multiple drones Dn, monitoring the aircraft status before flight, and monitoring the aircraft status during flight. Monitoring may also include actions accompanying monitoring (such as operations by the operator OPm). The operator OPm determines whether or not to allow the drone Dn to take off (takeoff determination) before the drone Dn takes off. For this reason, a takeoff determination button for receiving takeoff permission for the drone Dn from the operator OPm is displayed on the screen of the operator terminal Tm. The takeoff determination button is controlled so that takeoff permission for the drone Dn can be received only if the drone Dn is in a flyable state after the timing for the operator OPm to determine whether or not to allow the takeoff arrives. Control of the takeoff determination button includes, for example, control to enable or disable the takeoff determination button (i.e., to activate or deactivate it). The control to disable the takeoff decision button may include display control to gray out the takeoff decision button (to make it difficult to see). The timing to determine whether or not to permit takeoff may be set to multiple stages.

[0019] The management server MS is composed of one or more server computers that manage information about drone bases Bm for each drone base Bm. The information about drone bases Bm includes, for example, information about drones Dn under the jurisdiction of the drone base Bm, information about operators OPm that monitor the drones Dn, and information about operator terminals Tm used by the operators OPm. The management server MS identifies multiple drones Dn that are managed by an operator OPm who logs in by operating the operator terminal Tm, and displays a list showing the identified multiple drones Dn (hereinafter referred to as the "drone list") on the operator terminal Tm of the operator OPm. The management server MS also receives detailed weather information about drone bases Bm from a weather management server (not shown) via a communication network NW on a regular or irregular basis.

[0020] [ 1-1. Drone Dn configuration and functions ] Next, the configuration and functions of the drone Dn will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the schematic configuration of the drone Dn. As shown in Fig. 2, the drone Dn includes a power supply unit 11, a drive unit 12, a positioning unit 13, a communication unit 14, a sensor unit 15, a memory unit 16, and a control unit 17. Furthermore, the drone Dn includes a propeller (rotor) which is a horizontal rotor, and an arm pipe (including an arm joint) for attaching the propeller to the drone body (housing). When the drone Dn is used to deliver goods, the drone Dn is provided with a holding mechanism for holding the goods.

[0021] The power supply unit 11 includes a detachable battery (power storage device) and the like. The power supply unit 11 supplies (feeds) the power stored in the battery to each part of the drone Dn. The power supply unit 11 also continuously measures the remaining battery charge. Battery information indicating the remaining battery charge measured by the power supply unit 11 is output to the control unit 17. The drive unit 12 includes a motor, a rotating shaft, and the like. The drive unit 12 rotates multiple rotors using the motor, rotating shaft, and the like that are driven in accordance with control signals output from the control unit 17.

[0022] The positioning unit 13 includes a radio wave receiver, an altitude sensor, and the like. The positioning unit 13 receives radio waves transmitted from positioning satellites of a Global Navigation Satellite System (GNSS), such as a Global Positioning System (GPS), using the radio wave receiver, and sequentially detects the current position of the drone Dn based on the radio waves. The current position of the UAV 1 may be represented by the latitude and longitude of the UAV 1, or by the latitude, longitude, and altitude of the UAV 1. Here, the positioning satellites may include satellites used by multiple satellite positioning systems, such as GPS satellites, Michibiki satellites, and Galileo satellites. Position information indicating the current position detected by the positioning unit 13 is continuously output to the control unit 17. At this time, acquisition number information indicating the number of acquisitions of positioning satellites (satellite acquisition number) acquired by the positioning unit 13 is continuously output to the control unit 17. The positioning unit 13 may also detect the altitude of the drone Dn using an altitude sensor. In this case, the location information indicating the current location of the drone Dn includes the altitude detected by the altitude sensor. The communication unit 14 has an antenna and a wireless communication function, and controls communication performed via the communication network NW.

[0023] The sensor unit 15 includes various sensors used to control the drone Dn. The various sensors include, for example, a compass (geomagnetic sensor), a gyro (three-axis angular velocity sensor), a three-axis acceleration sensor, a barometric pressure sensor, a gimbal, an optical sensor, and a rangefinder. The optical sensor includes an on-board camera (for example, an RGB camera or an infrared camera). The on-board camera is configured to, for example, continuously capture images of the surroundings of the drone Dn (for example, in front of or below the drone Dn). The orientation of the on-board camera (in front of or below the drone Dn) can be controlled by the control unit 17. The sensing information sensed by the sensor unit 15 is output to the control unit 17. The storage unit 16 is configured with a non-volatile memory or the like and stores various programs and data. The storage unit 16 also stores an on-board ID (identification information) for identifying the drone Dn.

[0024] The control unit 17 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and controls the drone Dn based on position information from the positioning unit 13 and sensing information from the sensor unit 15. Such control includes control of the propeller rotation speed, and control of the position, attitude, and direction of travel of the drone Dn. The position information of the drone Dn (i.e., the position information from the positioning unit 13) is transmitted to the management server MS via the communication network NW together with aircraft image information (an example of aircraft information) representing an aircraft image (a still image or a moving image) captured by the aircraft camera, battery information (an example of aircraft information) from the power supply unit 11, capture number information (an example of aircraft information) from the positioning unit 13, and the aircraft ID of the drone Dn.

[0025] The control unit 17 also has a self-diagnosis function and performs automatic inspections to check whether certain parts of the drone Dn (e.g., the power supply unit 11, drive unit 12, positioning unit 13, communication unit 14, and sensor unit 15) are operating normally. Items inspected in the automatic inspection include, for example, remaining battery power, battery cell balance, GPS, compass, gyro, acceleration sensor, barometric pressure sensor, gimbal, optical sensor, and range finder. Automatic inspection result information indicating the results of the automatic inspection by the drone Dn (i.e., the control unit 17) is transmitted to the management server MS via the communication network NW. Note that the automatic inspection by the drone Dn is performed before the drone Dn takes flight, but some of the inspection items of the automatic inspection are also performed while the drone Dn is in flight.

[0026] [ 1-2.Configuration and functions of operator terminal Tm ] Next, the configuration and functions of the operator terminal Tm will be described with reference to FIG. 3. FIG. 3 is a diagram illustrating an example of a schematic configuration of the operator terminal Tm. The operator terminal Tm includes an operation and display unit 21, a communication unit 22, a storage unit 23, and a control unit 24. For example, a personal computer can be applied to the operator terminal Tm. The operator terminal Tm may also include a sound processing unit and a speaker. The operation and display unit 21 has an input function for accepting input (selection) by the operator OPm's finger, pen, or mouse, and a display function for displaying various screens on the display. The operator terminal Tm may be provided with multiple displays. The various screens include a login screen for the operator OPm to log in, a drone monitoring screen for the operator OPm to monitor the drone Dn, and an action request notification screen for notifying the operator OPm of a request for a predetermined action. The drone monitoring screen and the action request notification screen may be simultaneously displayed on separate displays.

[0027] The communication unit 22 is responsible for controlling communications carried out via the communication network NW. The storage unit 23 is composed of non-volatile memory and the like, and stores various programs (program code groups) and data. The various programs include an operating system (OS), a monitoring application, a GCS, and a web browser. The monitoring application is a program that mainly acquires and displays information about multiple drones Dn managed by an operator OPm from the management server MS. The monitoring application may be downloaded to the operator terminal Tm from a specified server.

[0028] The control unit 24 (an example of a computer) includes a CPU, a ROM, a RAM, etc., and executes processing in accordance with a monitoring application stored in the ROM (or the storage unit 23). When the monitoring application is started in response to an instruction from the operator OPm, the control unit 24 displays a login screen on the display. Then, when the operator OPm inputs a user ID and a password through the login screen, the control unit 24 transmits a login request including the user ID and the password to the management server MS via the communication unit 22 and the communication network NW. The user ID is identification information for identifying the operator OPm.

[0029] When an operator OPm logs in in response to the login request, the management server MS transmits display control data for displaying a drone list showing the multiple drones Dn managed by the logged-in operator OPm. This causes the control unit 24 to display a drone monitoring screen including the drone list on the display. Note that the login process determines whether or not the pair of user ID and password included in the login request has been registered. If the pair of user ID and password has been registered, the operator OPm using the operator terminal Tm that transmitted the login request is identified, and the operator OPm logs in.

[0030] Here, the multiple drones Dn shown in the drone list may be under the jurisdiction of the same drone base Bm, or may be under the jurisdiction of different drone bases Bm. In the latter case, the drone list shows the drones Dn (e.g., drones D1 to D4 and drones D5 to D9) belonging to each of the multiple drone bases Bm (e.g., drone base B1 and drone base B2). This allows the operator OPm to monitor each drone Dn under the jurisdiction of the multiple drone bases Bm. Note that the display control data may be data for a web page displayed by a web browser. Display data for the drone monitoring screen may be incorporated into the monitoring application.

[0031] 4 to 8 are diagrams showing examples of a drone monitoring screen displayed on the operator terminal T1 of operator OP1. As shown in FIGS. 4 to 8, the drone monitoring screen displays a drone list L showing multiple drones D1 to D4 (drone names: Drone A to D), and further includes a drone basic information display area A1, a drone detailed information display area A2, an aircraft camera display area A3, and a base camera display area A4. The drone list L is composed of simple information display areas LA1 to LA4 corresponding to the drones D1 to D4 respectively managed by operator OP1. The simple information display areas LA1 to LA4 are scrollable. The names (drone names) of the drones D1 to D4 are displayed in the simple information display areas LA1 to LA4. Note that the drone list L may also include simple information display areas corresponding to the drones D5 to D9 respectively managed by operator OP1.

[0032] The simple information display areas LA1 to LA4 can be selected by the operator OP1. In other words, the drones D1 to D4 can be selected via the simple information display areas LA1 to LA4. In the display examples of FIGS. 4 to 8, the drone D1 is selected by the operator OP1 selecting (pressing) the simple information display area LA1. Therefore, the monitoring information (obtained as needed from the management server MS) of the selected drone D1 is displayed in the drone basic information display area A1, the drone detailed information display area A2, the aircraft camera display area A3, and the base camera display area A4. Here, the monitoring information includes, for example, manual inspection result information, base image information, aircraft data information, aircraft image information, detailed weather information, etc. In the display examples of FIGS. 4 to 8, the drone D1 is placed at port P1 at the drone base B1 for takeoff and is waiting for the operator OP1 to decide whether to take off.

[0033] The drone basic information display area A1 displays basic information about the drone D1, the status of the drone D1 (hereinafter referred to as the "drone status"), the weather status at the drone base B1 (hereinafter referred to as the "weather status"), and the drone schedule for the drone D1. The drone basic information display area A1 may also display measurement information measured by a sensor provided in the base device Em. The basic information about the drone D1 includes, but is not limited to, the drone name (the name of the drone D1), type, model number, manufacturer, base name (the name of the drone base B1), and port name (the name of the port P1). The drone status of the drone D1 indicates the current state (status) of the drone D1. The weather status indicates the current state (status) of the weather at the drone base B1 based on detailed weather information. The drone schedule includes, for example, the scheduled times for the drone D1 to take off and arrive at the destination (e.g., arrival at the delivery destination).

[0034] Furthermore, the drone basic information display area A1 is provided with a drone status button BO1, a weather status button BO2, and a takeoff decision button BO3. In the display examples of FIGS. 4 to 6, the drone status button BO1 displays "Flyable" as the drone status. "Flyable" indicates that the drone D1 is in a state where it can fly based on the aircraft's status. On the other hand, in the display examples of FIGS. 7 and 8, the drone status button BO1 displays "Not Flyable" as the drone status. "Not Flyable" indicates that the drone D1 is not in a state where it can fly based on the aircraft's status (e.g., an abnormality has occurred in the aircraft, the aircraft is improperly placed, etc.). The aircraft's status may be identified from manual inspection result information or automatic inspection result information. Also, in the display examples of FIGS. 4 to 8, the weather status button BO2 displays "OK" as the weather status, indicating good weather. Good weather indicates that the drone D1 is in a state where it can fly based on the weather conditions. Note that, although not shown, the weather status button BO2 may also display "NG" as the weather status, indicating bad weather. Bad weather indicates that drone D1 is not in a condition to fly due to weather conditions (e.g., strong winds, heavy rainfall, etc.).

[0035] The takeoff decision button BO3 is an example of an operation button for accepting takeoff permission for drone D1 from operator OP1. In the display example of FIG. 4, because the timing for deciding whether to permit or deny takeoff has not yet arrived, the takeoff decision button BO3 is grayed out and displayed as disabled so that takeoff permission cannot be accepted. In other words, operator OP1 cannot select (press) the takeoff decision button BO3. At this time, “Takeoff Decision Before” is displayed on the takeoff decision button BO3. In the display example of FIG. 5, the timing for deciding whether to permit or deny takeoff (e.g., the first decision timing 15 minutes before takeoff) has arrived and drone D1 is in a flyable state, so the takeoff decision button BO3 is displayed as enabled so that the initial takeoff permission can be accepted. In other words, operator OP1 can select the takeoff decision button BO3. At this time, “Takeoff Decision Before” is also displayed on the takeoff decision button BO3. When the takeoff decision button BO3 is selected in the display example of FIG. 5, the display shown in FIG. 6 is displayed. At this time, “Takeoff Planned” is displayed on the takeoff decision button BO3. 7 and 8, the time has come to decide whether or not to allow takeoff, but the drone D1 is not in a flight-ready state, so the takeoff decision button BO3 is displayed as disabled so that takeoff permission cannot be accepted. At this time, "Takeoff not permitted" is displayed on the takeoff decision button BO3.

[0036] The drone detailed information display area A2 displays an aircraft status check tab TB1, an aircraft data tab TB2, and a weather status tab TB3. In the display examples of FIGS. 4 to 8, the aircraft status check tab TB1 is selected and displayed among the tabs TB1 to TB3, and therefore aircraft status check information (list) is displayed below the tabs TB1 to TB3. The aircraft status check information is based on the manual inspection result information described above. In the display examples of FIGS. 4 to 6, "OK" displayed in association with the inspection item "arm joint looseness" included in the aircraft status check information indicates, for example, that the result (inspection result) of this manual inspection (aircraft status check) is good. Note that if the result of this manual inspection is not good (i.e., it is poor), for example, "NG" is displayed in association with that inspection item.

[0037] On the other hand, when the operator OP1 selects the aircraft data tab TB2 in the drone detail information display area A2, aircraft data information (list) (not shown) is displayed below the tabs TB1 to TB3. The aircraft data information is based on the above-mentioned battery information (remaining battery capacity) and acquisition number information (number of acquired satellites). Furthermore, when the operator OP1 selects the weather status tab TB3 in the drone detail information display area A2, detailed weather information for the drone base B1 is displayed below the tabs TB1 to TB3. Note that while the tabs TB1 to TB3 can each be manually selected by the operator OP1, they are automatically selected (i.e., switched) in response to the selection of the drone status button BO1 displaying "No Flight" or the weather status button BO2 displaying "NG." Furthermore, when the drone status button BO1 or the weather status button BO2 is selected, the tabs TB1 to TB3 may be switched regardless of the drone status.

[0038] For example, when the aircraft status check tab TB1 is selected as shown in Figure 7, if the drone status button BO1 displaying "Not flyable" is selected, the inspection item "Drone location" with an unsatisfactory result is displayed in the aircraft status check information as shown in Figure 8. In other words, the display transitions to a display area where the factors that caused the drone to be unflyable can be confirmed. This allows the operator OP1 to quickly identify which inspection item resulted in an "NG" result. On the other hand, when the aircraft data tab TB2 is selected and the drone status button BO1 displaying "Not flyable" is selected, the display switches to the aircraft status check tab TB1 (i.e., transitions to a tab where the factors that caused the drone to be unflyable can be confirmed), and the inspection item with an unsatisfactory result is displayed in the aircraft status check information. Note that when the aircraft status check tab TB1 is selected and the weather status button BO2 displaying "NG" is selected, the display switches to the weather status tab TB3, which displays detailed weather information for the drone base B1.

[0039] The on-board camera display area A3 displays a video captured by the on-board camera of the drone D1 (hereinafter referred to as "on-board camera video"). Meanwhile, the base camera display area A4 displays a video captured by the base camera of the base device E1 (hereinafter referred to as "base camera video"). In the display example of FIG. 5, the timing for the first decision on whether to allow takeoff has arrived and the drone D1 is in a flight-ready state, so the base camera video is highlighted (for example, the frame A41 in the base camera display area A4 is displayed in red) to prompt the operator OP1 to check the base camera video. At this time, the on-board camera video may be highlighted along with the base camera video. Highlighting refers to displaying the image in a conspicuous display manner (for example, a conspicuous display color or display size). In particular, examples of highlighting an image include displaying the frame of the image's display area in a conspicuous display manner (for example, displaying the frame in a more conspicuous color than usual, displaying the frame in a thicker color than usual, or displaying the frame in a blinking manner), or displaying a message (for example, text strings) within the image's display area. In addition to the base camera image, basic information about drone D1 may also be highlighted.

[0040] When the initial timing for determining whether or not to permit takeoff arrives and the drone D1 is in a flight-ready state, if the operator OP1 selects (presses) the takeoff decision button BO3 displaying "Before Takeoff Decision," a pop-up screen (not shown) with an OK (Yes) button and an NG (No) button is displayed on the drone monitoring screen. When the operator OP1 selects the OK button displayed on the pop-up screen, the initial decision permission is accepted by the control unit 24, and takeoff permission information indicating the takeoff permission is transmitted to the management server MS. In this manner, when takeoff permission for the drone D1 is accepted via the takeoff decision button BO3 displaying "Before Takeoff Decision," the display on the takeoff decision button BO3 switches to "Takeoff Planned," and the highlighting of the base camera image, etc., is stopped. Note that when the takeoff decision button BO3 displaying "Before Takeoff Decision" is displayed on the drone monitoring screen, monitoring information including information about the drone Dn being monitored (e.g., the drone name) along with information prompting the operator OP1 to make a takeoff decision may be displayed on the action request notification screen. Thereafter, drone D1 continues to be in a state where it can fly, and after a predetermined time has passed, when the time comes for a final decision on whether or not to allow takeoff (final takeoff decision), it will be able to accept final takeoff permission.

[0041] Here, final takeoff permission may be given on a pop-up screen that is displayed when the takeoff decision button BO3 displaying "Takeoff Planned" is selected, as shown in FIG. 6. FIG. 9 is a diagram showing an example of a pop-up screen that is displayed when the takeoff decision button BO3 displaying "Takeoff Planned" is selected. The takeoff decision button BO4 displayed on each of the pop-up screens Pa and Pb shown in FIG. 9 is an example of an operation button for accepting takeoff permission for drone D1 from operator OP1. The takeoff decision button BO4 displayed on the pop-up screen Pa shown in FIG. 9 is grayed out and displayed as disabled so that final takeoff permission cannot be accepted. This is because the final decision timing for whether to allow or deny takeoff has not yet arrived.

[0042] Meanwhile, the takeoff decision button BO4 displayed on the pop-up screen Pb shown in FIG. 9 is enabled so that final takeoff permission can be accepted. This is because the final decision timing for whether or not to allow takeoff has arrived and the drone D1 is in a flight-ready state. The takeoff prohibition button BO5 displayed on each of the pop-up screens Pa and Pb is displayed selectable. When the final decision timing for whether or not to allow takeoff arrives (only when the drone D1 is in a flight-ready state), the base camera image, etc. is highlighted again to prompt the operator OP1 to check the base camera image, etc. Then, when the takeoff decision button BO4 displayed on the pop-up screen Pb shown in FIG. 9 is selected by the operator OP1, takeoff permission information is transmitted to the management server MS, the GCS, or the drone D1. This causes the drone D1 to take off from port P1. When a pilot of the drone Dn is present, takeoff permission may be notified to the pilot terminal used by the pilot when the operator OP1 selects the takeoff decision button BO4 displayed on the pop-up screen Pb shown in FIG. 9.

[0043] [ 1-3.Configuration and Functions of Management Server MS ] Next, the configuration and functions of the management server MS will be described with reference to FIG. 10. FIG. 10 is a diagram illustrating an example of a schematic configuration of the management server MS. As illustrated in FIG. 10, the management server MS includes a communication unit 31, a storage unit 32, a control unit 33, and the like. The communication unit 31 controls communication via the communication network NW. The communication unit 31 receives manual inspection result information, base image information, measurement information, and base ID transmitted from the base device Em. The communication unit 31 receives battery information, capture number information, aircraft image information, position information of the drone Dn, automatic inspection result information, and aircraft ID transmitted from the drone Dn. The management server MS can recognize the current position of the drone Dn based on the position information of the drone Dn. The communication unit 31 also receives a login request transmitted from the operator terminal Tm. The communication unit 31 also receives detailed weather information transmitted from the weather management server.

[0044] The storage unit 32 is configured, for example, with a hard disk drive or the like, and stores various programs including an operating system and applications. Here, the applications include programs for executing control methods. Furthermore, the storage unit 32 has constructed therein a base management database (DB) 321, a drone management database (DB) 322, and an operator management database (DB) 323. The base management database 321 is a database for managing information related to drone bases Bm.

[0045] The base management database 321 stores, for example, the base ID of the drone base Bm, base image information, measurement information, detailed weather information, weather status, and the aircraft ID of the drone Dn under the jurisdiction of the drone base Bm, in association with each drone base Bm. Note that the base image information, measurement information, and detailed weather information may be updated as appropriate each time they are received by the communication unit 31. The weather status is also updated as appropriate according to the detailed weather information.

[0046] The drone management database 322 is a database for managing information related to drones Dn. The drone management database 322 stores the drone Dn's aircraft ID, location information, basic information, drone status, drone schedule, manual inspection result information, aircraft data information, and aircraft image information, etc., in association with each drone Dn. The aircraft data information includes, for example, battery information and capture count information. The manual inspection result information, aircraft data information, and aircraft image information may be updated as appropriate each time they are received by the communication unit 31. The drone status is also updated as appropriate, for example, in accordance with the manual inspection result information or automatic inspection result information received by the communication unit 31.

[0047] The operator management database 323 is a database for managing information related to the operator OPm. The operator management database 323 stores the user ID, password, login status, and the aircraft ID and name of each of the multiple drones Dn that the operator OPm is in charge of, in association with each operator OPm. Here, the login status indicates whether the operator OPm is logged in or not.

[0048] The control unit 33 (an example of a computer) includes a CPU, a ROM, a RAM, and the like. FIG. 11 is a diagram illustrating an example of functional blocks in the control unit 33. The CPU may be a general-purpose processor, a special-purpose processor, or a processor including transistors and other integrated circuits (electrical circuits or electronic circuits). In accordance with a program (a group of program codes) stored in, for example, the ROM or the storage unit 32, the control unit 33 functions as a display control unit 331 (an example of a display control means), a timing arrival determination unit 332 (an example of a first determination means), a button control unit 333 (an example of a button control means), a status identification unit 334 (an example of a first identification means and a second identification means), a flight-safe state determination unit 335 (an example of a second determination means and a third determination means), an abnormality determination unit 336 (an example of a fourth determination means), and the like, as shown in FIG. 11 .

[0049] The display control unit 331 generates a drone list showing, for example, multiple drones Dn that are managed by the logged-in operator OPm. Then, the display control unit 331 transmits display control data for displaying the generated drone list to the operator terminal Tm of the operator OPm, thereby displaying the drone list on the drone monitoring screen of the operator terminal Tm. When the operator OPm selects a drone Dn from the drone list, the selection request (including the aircraft ID of the selected drone Dn) is transmitted to the management server MS and received by the communication unit 31.

[0050] Then, in response to a selection request from the management server MS, the display control unit 331 transmits display control data for displaying the monitoring information and takeoff decision button BO3 of the selected drone Dn to the operator terminal Tm, thereby displaying the monitoring information together with the takeoff decision button BO3 on the drone monitoring screen of the operator terminal Tm. This allows the operator OPm to quickly input the takeoff decision result (i.e., takeoff permission) using the takeoff decision button BO3 while viewing the monitoring information of the drone Dn. Note that the drone Dn may be automatically selected according to the drone status of the drone Dn, and the monitoring information and takeoff decision button BO3 of the selected drone Dn may be automatically displayed on the drone monitoring screen.

[0051] The timing arrival determination unit 332 determines at predetermined time intervals whether the timing for the operator OPm to determine whether or not to allow the drone Dn to take off has arrived (i.e., the current time has become the determination timing). The timing for determining whether or not to allow the takeoff may be set, for example, T1 (e.g., 15) minutes before the scheduled takeoff time. As described above, the timing for determining whether or not to allow the takeoff may be set in two or more stages, such as the first determination timing (e.g., T1 minute before the scheduled takeoff time) and the final determination timing (e.g., T2 (e.g., 5) minutes before the scheduled takeoff time (T1>T2)). Here, T1 and T2 may be set based on the time required for the operator OPm to determine whether or not to allow the takeoff.

[0052] Alternatively, depending on the operator OPm, in addition to determining whether or not to permit takeoff, there may be cases where the operator OPm performs tasks such as specifying the flight route of the drone Dn or communicating with the pilot of the drone Dn. Therefore, in addition to the time required for determining whether or not to permit takeoff, the determination timing (e.g., T1, T2) may be set based on the time required for the above tasks. In this case, the first determination timing is set to be longer than the above T1, for example, T3 (e.g., 25) minutes before the scheduled takeoff time. Note that information indicating the task and the time required for the task may be associated with the user ID of the operator OPm and stored in the operator management database 323.

[0053] The button control unit 333 controls (e.g., disables) the takeoff decision button BO3 (or the takeoff decision button BO4 in the case of the final decision timing) so as not to accept takeoff permission for the drone Dn before the decision timing for whether or not to allow takeoff for the drone Dn (the selected drone Dn) arrives, in accordance with the decision of the timing arrival by the timing arrival decision unit 332. On the other hand, after the decision timing for whether or not to allow takeoff arrives, the button control unit 333 controls (e.g., enables) the takeoff decision button BO3 (or the takeoff decision button BO4 in the case of the final decision timing) so as to accept takeoff permission for the drone Dn if the drone Dn is in a flyable state. Whether or not the drone Dn is in a flyable state is determined by the flyable state decision unit 335, as will be described later.

[0054] For example, when a takeoff decision button BO3 corresponding to a drone Dn is displayed on the drone monitoring screen before the initial decision timing for the drone Dn to approve or reject takeoff arrives (see FIG. 4), the button control unit 333 may disable the takeoff decision button BO3 by setting, in the display control data, disabling control data for disabling the takeoff decision button BO3. At this time, the display control unit 331 may transmit, to the operator terminal Tm, the display control data for graying out the takeoff decision button BO3, as shown in FIG. 4. Furthermore, when a takeoff decision button BO4 corresponding to a drone Dn is displayed on the drone monitoring screen before the final decision timing for the drone Dn to approve or reject takeoff arrives (see FIG. 9), the button control unit 333 may disable the takeoff decision button BO4 by setting, in the display control data, disabling control data for disabling the takeoff decision button BO4. At this time, the display control unit 331 may transmit, to the operator terminal Tm, the display control data for graying out the takeoff decision button BO4, as shown in the pop-up screen Pa of FIG. 9.

[0055] On the other hand, when the takeoff decision button BO3 corresponding to the drone Dn is displayed on the drone monitoring screen after the arrival of the first decision timing for whether to allow or deny takeoff of the drone Dn and the drone Dn is in a flyable state (see FIG. 5), the button control unit 333 may set the activation control data for activating the takeoff decision button BO3 to the display control data, thereby activating the takeoff decision button BO3. Also, when the takeoff decision button BO3 corresponding to the drone Dn is displayed on the drone monitoring screen (see FIG. 4) and the first decision timing for whether to allow or deny takeoff of the drone Dn arrives and the drone Dn is in a flyable state (see FIG. 5), the button control unit 333 may activate the takeoff decision button BO3 by transmitting the activation control data for activating the takeoff decision button BO3 to the operator terminal Tm.

[0056] Furthermore, when the takeoff decision button BO4 corresponding to the drone Dn is displayed on the drone monitoring screen after the final decision timing for whether to permit takeoff of the drone Dn has arrived and the drone Dn is in a flyable state (see FIG. 9), the button control unit 333 may set the enablement control data for enabling the takeoff decision button BO4 to the display control data, thereby enabling the takeoff decision button BO4. Furthermore, when the takeoff decision button BO4 corresponding to the drone Dn is displayed on the drone monitoring screen and the final decision timing for whether to permit takeoff of the drone Dn has arrived and the drone Dn is in a flyable state (see FIG. 9), the button control unit 333 may enable the takeoff decision button BO4 by transmitting the enablement control data for enabling the takeoff decision button BO4 to the operator terminal Tm.

[0057] Furthermore, when the takeoff decision button BO3 (or the takeoff decision button BO4) is controlled to accept takeoff permission for the drone Dn (that is, when the takeoff decision button BO3 or the takeoff decision button BO4 is activated), the display control unit 331 may transmit display control data to the operator terminal Tm for highlighting at least one of the aircraft image (for example, aircraft camera image) and the base image (for example, base camera image) to prompt the operator OPm to check that image. This makes it possible to quickly notify the operator OPm that the time has come to make a takeoff decision.

[0058] For example, the frame A41 of the base camera display area A4 that displays the base camera video is displayed in red (for example, changed from black to red). Then, when the operator OPm selects the takeoff decision button BO3 (or the takeoff decision button BO4), takeoff permission information is transmitted to the management server MS and received by the communication unit 31. In this way, when takeoff permission for the drone Dn is accepted via the takeoff decision button BO3 (or the takeoff decision button BO4), the display control unit 331 may transmit highlighting stop control data for stopping the highlighting of the image to the operator terminal Tm. This makes it possible to quickly notify the operator OPm that the takeoff decision has been accepted.

[0059] The status identification unit 334 identifies the current drone status (e.g., flyable or non-flyable) of the selected drone Dn, for example, from the drone management database 322. The status identification unit 334 also identifies the current weather status (e.g., good weather or bad weather) of the drone base Bm to which the selected drone Dn belongs, for example, from the base management database 321. The flight feasibility determination unit 335 determines whether the selected drone Dn is in a flight feasible state based on at least one of the drone status identified by the status identification unit 334 and the weather status identified by the status identification unit 334. This makes it possible to accurately determine whether the drone Dn is flight feasible from the perspective of the drone status or the weather status. For example, if the drone status indicates "flyable," the drone Dn is determined to be in a flight feasible state. Alternatively, if the drone status indicates "flyable" and the weather status indicates "OK (good weather)," the drone Dn is determined to be flight feasible.

[0060] When the flight-ready state determination unit 335 determines that the drone Dn is not in a flight-ready state based on the drone status, the display control unit 331 may transmit display control data to the operator terminal Tm to display information indicating the factors that led to the determination that the drone Dn is not in a flight-ready state on the drone monitoring screen. As a result, for example, as shown in FIG. 8, information indicating the factors that led to the determination that the drone Dn is not in a flight-ready state is displayed on the drone monitoring screen in response to selection of the drone status button BO1 that displays "Not flyable." Therefore, the operator OPm can be quickly informed of the aircraft factors that led to the drone Dn not being able to fly. Note that, even if the drone status button BO1 that displays "Not flyable" is not selected, information indicating the factors that led to the determination that the drone Dn is not in a flight-ready state may be displayed on the drone monitoring screen (for example, transitioning to a display area where the factors that led to the drone Dn not being in a flight-ready state can be confirmed).

[0061] Furthermore, if the flight-safe state determination unit 335 determines that the drone Dn is not in a flight-safe state based on the weather status, the display control unit 331 may transmit display control data to the operator terminal Tm to display detailed information about the weather at the drone base Bm to which the drone Dn belongs on the drone monitoring screen. For example, this allows detailed information about the weather at the drone base Bm to which the drone Dn belongs to be displayed in response to selection of the weather status button BO2 displaying "NG." This allows the operator OPm to be quickly informed of the type of poor weather that prevents the drone Dn from flying. Note that even if the weather status button BO2 displaying "NG" is not selected, detailed information about the weather at the drone base Bm to which the drone Dn belongs may be displayed on the drone monitoring screen (for example, transitioning to a display area where detailed weather information can be confirmed).

[0062] When the timing for determining whether or not to allow takeoff of the selected drone Dn arrives (i.e., when the timing arrival determination unit 332 determines that the timing for determining whether or not to allow takeoff has arrived), the abnormality determination unit 336 analyzes at least one of the drone Dn's aircraft image and the drone base image of the drone base Bm to which the drone Dn belongs to determine whether or not an abnormality exists. For example, the abnormality determination unit 336 determines that an abnormality (e.g., an image transmission abnormality) exists when the brightness of the drone image or the base image is below a threshold (e.g., black (minimum brightness)). Alternatively, the abnormality determination unit 336 checks for the presence or absence of the drone by analyzing the base image of the drone base Bm (e.g., base camera video), and determines that an abnormality exists when the drone cannot be detected from the base image (i.e., when the drone is not visible in the base image). Here, the presence or absence of the drone may be confirmed based on a pre-registered drone exterior image (or feature points).

[0063] If the abnormality determination unit 336 determines that an abnormality exists, the display control unit 331 may transmit display control data to the operator terminal Tm for highlighting the image determined to have the abnormality among the aircraft image and the base image to indicate to the operator OPm that an abnormality exists. This allows the operator OPm to be quickly notified that an abnormality has been detected regarding the drone Dn based on analysis of the aircraft image or the base image. The highlighting at this time may be different from the highlighting displayed when the takeoff decision button BO3 (or the takeoff decision button BO4) is controlled to accept takeoff permission for the drone Dn. For example, if an abnormality is determined based on analysis of the base camera image, the frame A41 of the base camera display area A4 displaying the base camera image may be displayed flashing in red (e.g., changing from black to red).

[0064] On the other hand, if the abnormality determination unit 336 determines that there is no abnormality, the display control unit 331 displays at least one of the aircraft image and the base image to indicate to the operator OPm that there is no abnormality. This allows the operator OPm to be quickly informed that there is no abnormality regarding the drone Dn based on the analysis of the aircraft image or the base image. For example, it is preferable that the frame 31 of the aircraft camera display area A3 that displays the aircraft camera image and the frame 41 of the base camera display area A4 that displays the base camera image are displayed in yellow (for example, changed from black to yellow).

[0065] [ 2. Operation of the remote monitoring system S ] Next, the operation of the remote monitoring system S will be described with reference to Figs. 12 and 13. Fig. 12 is a flowchart showing an example of a monitoring information display process executed by the control unit 33 of the management server MS. Fig. 13 is a flowchart showing an example of an abnormality determination process executed by the control unit 33 of the management server MS. In the operation example described below, it is assumed that an operator OP1 of the operator terminal T1 is logged in, that multiple drones Dn that the logged-in operator OP1 is in charge of are identified, and that a drone monitoring screen including a drone list showing the identified multiple drones Dn is displayed on the operator terminal T1. When the operator OP1 logs in in this way, the management server MS continuously recognizes (monitors) the display status of the drone monitoring screen on the operator terminal T1 (including the display content of the monitoring information).

[0066] (2.1. Monitoring information display processing) First, the monitoring quantity information display process shown in Fig. 12 is started, for example, when the monitoring information of drone Dn selected from the drone list by operator OP1 is displayed on the drone monitoring screen. For convenience of explanation, it is assumed that operator OP1 selects drone D1, for which the timing for the first decision on whether or not to allow takeoff has not yet arrived, and that the monitoring information of drone D1 is displayed as shown in Fig. 4, and the takeoff decision button BO3 (before takeoff decision) is disabled.

[0067] 12, when the process starts, the control unit 33 determines whether the first determination timing has arrived using the timing arrival determination unit 332 (step S1). If it is determined that the first determination timing has not arrived (step S1: NO), the process proceeds to step S2. On the other hand, if it is determined that the first determination timing has arrived (for example, 15 minutes before the scheduled takeoff time) (step S1: YES), the process proceeds to step S3.

[0068] In step S2, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1. Examples of the predetermined instruction include an instruction to select another drone Dn in the drone list, an instruction to select buttons BO1 and BO2 shown in FIG. 4, etc., an instruction to select tabs TB1 to TB3 shown in FIG. 4, etc., and an instruction to log out. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S2: NO), the process returns to step S1. At this time, the process may return to step S1 after a predetermined waiting time (e.g., 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S2: YES), the process proceeds to another process. Here, the other process is a process (e.g., display process) in accordance with the above-mentioned predetermined instruction.

[0069] In step S3, the control unit 33 determines at least one of the current drone status of the drone D1 and the current weather status of the drone base B1 of the drone D1 using the status determination unit 334. Next, the control unit 33 determines whether the drone D1 is in a flight-ready state based on at least one of the drone status and the weather status determined in step S3 using the flight-ready state determination unit 335 (step S4). If it is determined that the drone D1 is not in a flight-ready state (step S4: NO), the process proceeds to step S5. On the other hand, if it is determined that the drone D1 is in a flight-ready state (step S4: YES), the process proceeds to step S6.

[0070] In step S4, if it is determined based on the drone status that drone D1 is not in a flight-ready state, the display control unit 331 may display information indicating the reason for determining that drone D1 is not in a flight-ready state on the drone monitoring screen, as described above. Also, if it is determined based on the weather status that drone D1 is not in a flight-ready state, the display control unit 331 may display detailed information about the weather at drone base B1 of drone D1 on the drone monitoring screen, as described above.

[0071] In step S5, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1, similar to step S2. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S5: NO), the process returns to step S3. At this time, the process may return to step S3 after a predetermined waiting time (e.g., 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S5: YES), the process proceeds to another process.

[0072] In step S6, the control unit 33 causes the button control unit 333 to activate the grayed-out takeoff judgment button BO3 (before takeoff judgment) displayed on the drone monitoring screen, and proceeds to step S7. Such activation is performed, for example, by transmitting activation control data for activating the takeoff judgment button BO3 (before takeoff judgment) to the operator terminal T1. This cancels the graying out of the takeoff judgment button BO3 (before takeoff judgment), making it possible to accept takeoff permission for the drone D1. When the takeoff judgment button BO3 (before takeoff judgment) is activated in this manner, as described above, the display control unit 331 may highlight at least one of the aircraft image and the base image to prompt the operator OP1 to confirm that image.

[0073] In step S7, the control unit 33 determines whether or not the takeoff decision button BO3 (before takeoff decision) has been selected. If it is determined that the takeoff decision button BO3 (before takeoff decision) has not been selected (step S7: NO), the process proceeds to step S8. On the other hand, if the takeoff decision button BO3 (before takeoff decision) has been selected by the operator OP1, takeoff permission information is transmitted to the management server MS. The control unit 33 determines that the takeoff decision button BO3 (before takeoff decision) has been selected by receiving the takeoff permission information via the communication unit 31. If it is determined that the takeoff decision button BO3 (before takeoff decision) has been selected (step S7: YES), the process proceeds to step S9.

[0074] In step S8, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1, similar to step S2. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S8: NO), the process returns to step S7. At this time, the process may return to step S7 after a predetermined waiting time (for example, 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S8: YES), the process proceeds to another process.

[0075] In step S9, the control unit 33 changes the display on the takeoff judgment button BO3 from "before takeoff judgment" to "takeoff planned," and proceeds to step S10. This change is made, for example, by sending change control data to the operator terminal T1 to change the display on the takeoff judgment button BO3 from "before takeoff judgment" to "takeoff planned." Note that, in step S6, if at least one of the aircraft image and the base image is highlighted to prompt confirmation of that image, the control unit 33 stops the highlighting of that image. This stop is made, for example, by sending highlighting stop control data to the operator terminal T1 to stop the highlighting of that image.

[0076] In step S10, the control unit 33 determines whether the takeoff judgment button BO3 (takeoff planned) has been selected. If it is determined that the takeoff judgment button BO3 (takeoff planned) has not been selected (step S10: NO), the process proceeds to step S11. For example, if the takeoff judgment button BO3 (takeoff planned) is selected by the operator OP1, takeoff permission information is transmitted to the management server MS. The control unit 33 determines that the takeoff judgment button BO3 (takeoff planned) has been selected by receiving the takeoff permission information via the communication unit 31. If it is determined that the takeoff judgment button BO3 (takeoff planned) has been selected (step S10: YES), the process proceeds to step S12.

[0077] In step S11, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1, similar to step S2. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S11: NO), the process returns to step S10. At this time, the process may return to step S10 after a predetermined waiting time (for example, 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S11: YES), the process proceeds to another process.

[0078] In step S12, the control unit 33 determines whether the final determination timing has arrived using the timing arrival determination unit 332. If it is determined that the final determination timing has not arrived (step S12: NO), the process proceeds to step S13. On the other hand, if it is determined that the final determination timing has arrived (step S12: YES), the process proceeds to step S14.

[0079] In step S13, the control unit 33 displays a pop-up screen Pa on the drone monitoring screen, on which a disabled takeoff decision button BO4 is set, as shown in Fig. 9 (step S13), and proceeds to step S15. Such display is performed by transmitting display control data for displaying the pop-up screen Pa to the operator terminal Tm. Meanwhile, in step S14, the control unit 33 displays a pop-up screen Pb on which an enabled takeoff decision button BO4 is set, on the drone monitoring screen, as shown in Fig. 9 (step S14), and proceeds to step S18.

[0080] In step S15, the control unit 33 determines whether the final determination timing has arrived using the timing arrival determination unit 332. If it is determined that the final determination timing has not arrived (step S15: NO), the process proceeds to step S16. On the other hand, if it is determined that the final determination timing has arrived (step S15: YES), the process proceeds to step S17.

[0081] In step S16, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1, similar to step S2. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S16: NO), the process returns to step S15. At this time, the process may return to step S15 after a predetermined waiting time (e.g., 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S16: YES), the process proceeds to another process.

[0082] In step S17, the control unit 33 activates the takeoff decision button BO4 using the button control unit 333, and proceeds to step S18. This activation is performed, for example, by transmitting activation control data for activating the takeoff decision button BO4 to the operator terminal T1. This causes the takeoff decision button BO4 to no longer be grayed out, making it possible to accept the final takeoff permission. When the takeoff decision button BO4 is activated in this manner, as described above, the display control unit 331 may highlight at least one of the aircraft image and the base image to prompt the operator OP1 to confirm that image.

[0083] In step S18, the control unit 33 determines whether the takeoff decision button BO4 has been selected. If it is determined that the takeoff decision button BO4 has not been selected (step S18: NO), the process proceeds to step S19. For example, if the takeoff decision button BO4 has been selected by the operator OP1, takeoff permission information is transmitted to the management server MS. The control unit 33 determines that the takeoff decision button BO4 has been selected by receiving the takeoff permission information via the communication unit 31. If it is determined that the takeoff decision button BO4 has been selected (step S18: YES), the process proceeds to step S20.

[0084] In step S19, the control unit 33 determines whether or not a predetermined instruction has been received from the operator OP1, similar to step S2. If it is determined that a predetermined instruction has not been received from the operator OP1 (step S19: NO), the process returns to step S18. At this time, the process may return to step S18 after a predetermined waiting time (for example, 30 seconds) has elapsed. On the other hand, if it is determined that a predetermined instruction has been received from the operator OP1 (step S19: YES), the process proceeds to another process.

[0085] In step S20, the control unit 33 records the takeoff permission. Then, the control unit 33 sends a takeoff command to the drone D1 or notifies the control terminal of the pilot of the drone D1 of the takeoff permission. Note that if at least one of the aircraft image and the base image is highlighted in step S17 to prompt confirmation of that image, the control unit 33 stops highlighting that image.

[0086] Note that even after the processing of step S6 above, the control unit 33 may periodically or irregularly determine whether the drone D1 is in a flight-ready state based on at least one of the drone status and the weather status. In this case, if it is determined that the drone D1 is not in a flight-ready state, the takeoff determination button BO3 (before takeoff determination) is disabled, and the display on the takeoff determination button BO3 is changed to "Takeoff not allowed" as shown in FIG. 7.

[0087] (2.2. Abnormality Judgment Processing) Next, the abnormality determination process shown in FIG. 13 may be performed before the process of step S3 when it is determined that the first determination timing has arrived in step S1 shown in FIG. 12 (step S1: YES), for example.

[0088] 13 starts, the control unit 33 acquires an image of the drone D1 (step S31). Next, the control unit 33 determines whether or not there is an abnormality in the image of the drone D1 acquired in step S31 using the abnormality determination unit 336 (step S32). If it is determined that there is an abnormality in the image of the drone D1 (step S32: YES), the process proceeds to step S33. On the other hand, if it is determined that there is no abnormality in the image of the drone D1 (step S32: NO), the process proceeds to step S34.

[0089] In step S33, the control unit 33 highlights the aircraft image so as to indicate to the operator OP1 that there is an abnormality (for example, by displaying the frame 31 of the aircraft camera display area A3 in red). Such highlighting is performed, for example, by transmitting display control data to the operator terminal T1 for highlighting the aircraft image so as to indicate that there is an abnormality.

[0090] As another example of displaying the aircraft image to indicate that an abnormality exists, a message indicating that an abnormality exists may be displayed on the aircraft image. Furthermore, if it is determined that an abnormality exists in the aircraft image, the control unit 33 may change the drone status of the drone D1 to "abnormality occurred" and associate the changed drone status with the aircraft ID of the drone D1 and update and register it in the drone management database 322. In this case, it is determined in step S4 shown in FIG. 12 that the drone is not in a flyable state.

[0091] In step S34, the control unit 33 acquires a base image of the drone base B1 of the drone D1. Next, the control unit 33 determines whether or not there is an abnormality in the base image acquired in step S34 using the abnormality determination unit 336 (step S35). If it is determined that there is an abnormality in the base image (step S35: YES), the process proceeds to step S36. On the other hand, if it is determined that there is no abnormality in the base image (step S35: NO), the process proceeds to step S37.

[0092] In step S36, the control unit 33 highlights the base image so as to indicate to the operator OP1 that there is an abnormality (for example, by displaying the frame 41 of the base camera display area A4 in red). Such highlighting is performed, for example, by transmitting display control data to the operator terminal T1 for highlighting the base image so as to indicate that there is an abnormality.

[0093] As another example of displaying the base image to indicate that an abnormality exists, a message indicating that an abnormality exists may be displayed on the base image. Furthermore, if it is determined that an abnormality exists in the base image, the control unit 33 may change the drone status of the drone D1 to "abnormality occurred" and associate the changed drone status with the aircraft ID of the drone D1 and update and register it in the drone management database 322. In this case, it is determined in step S4 shown in FIG. 12 that the drone is not in a flyable state.

[0094] In step S37, the control unit 33 displays the aircraft image and the base image to indicate to the operator OP1 that there is no abnormality (for example, by displaying the frame 31 of the aircraft camera display area A3 and the frame 41 of the base camera display area A4 in yellow). Such display may be performed by transmitting display control data to the operator terminal T1 for displaying the aircraft image and the base image to indicate that there is no abnormality.

[0095] The abnormality determination process shown in Fig. 13 may be performed before the process of step S14 when it is determined that the final determination timing has arrived in step S12 shown in Fig. 12 (step S12: YES). Also, the abnormality determination process shown in Fig. 13 may be performed before the process of step S17 when it is determined that the final determination timing has arrived in step S15 shown in Fig. 12 (step S15: YES).

[0096] As described above, according to the above embodiment, the management server MS displays an operation button (for example, a takeoff decision button BO3 or a takeoff decision button BO4) on the drone monitoring screen for accepting takeoff permission for the drone Dn from the operator OPm, determines whether the time has come for the operator OPm to decide whether to allow or deny the takeoff, and controls the operation button so that takeoff permission for the drone Dn cannot be accepted before the decision time arrives, while controlling the operation button so that takeoff permission for the drone Dn can be accepted after the decision time arrives if the drone Dn is in a state where it can fly.This makes it possible to appropriately support the operator OPm, who makes takeoff decisions based on the pre-flight risks of each of the multiple drones Dn, thereby reducing the burden on the operator OPm and preventing human error when the operator OPm makes takeoff decisions.

[0097] The above embodiment is merely an embodiment of the present invention, and the present invention is not limited to the above embodiment. Various configurations and other changes may be made to the above embodiment without departing from the spirit of the present invention, and such changes are still within the technical scope of the present invention. In the above embodiment, instead of the management server MS, the operator terminal Tm (control unit 24) may be configured to display operation buttons for receiving takeoff permission for the drone Dn from the operator OPm on the drone monitoring screen in accordance with a monitoring application, determine whether the timing for the operator OPm to determine whether or not to allow the drone Dn to take off has arrived, and control the operation buttons so that takeoff permission for the drone Dn cannot be received before the arrival of the determination timing, but control the operation buttons so that takeoff permission for the drone Dn can be received after the arrival of the determination timing if the drone Dn is in a flight-ready state. In this case, the operator terminal Tm may execute the monitoring information display process shown in FIG. 12 and the abnormality determination process shown in FIG. 13 while appropriately acquiring necessary information from the management server MS.

[0098] <Additional Notes> [1] The information processing device according to the present disclosure is characterized by comprising: a display control means for displaying an operation button for receiving takeoff permission for an unmanned aerial vehicle from an operator on a screen of a terminal used by the operator; a first determination means for determining whether the timing for the operator to determine whether to allow the unmanned aerial vehicle to take off has arrived; and a button control means for controlling the operation button so that takeoff permission for the unmanned aerial vehicle cannot be accepted before the arrival of the determination timing, and controlling the operation button so that takeoff permission for the unmanned aerial vehicle can be accepted after the arrival of the determination timing if the unmanned aerial vehicle is in a flyable state. This prevents human error when the operator makes a takeoff decision.

[0099] [2] The information processing device described in [1] above is characterized in that it further comprises a first identification means for identifying the status of the unmanned aerial vehicle, and a second determination means for determining whether the unmanned aerial vehicle is in a flyable state based on the status identified by the first identification means. This makes it possible to accurately determine whether the unmanned aerial vehicle is flyable from the perspective of the status of the unmanned aerial vehicle.

[0100] [3] In the information processing device described in [2] above, the display control means is characterized in that, when the second determination means determines that the unmanned aerial vehicle is not in a flight-ready state, displays information indicating the cause of the determination that the unmanned aerial vehicle is not in a flight-ready state on the screen, thereby enabling the operator to quickly be informed of the aircraft factors that prevent the unmanned aerial vehicle from flying.

[0101] [4] The information processing device according to [1] or [2] above is characterized in that it further comprises a second identification means for identifying the weather status of a base where the unmanned aerial vehicle can take off and land, and a third determination means for determining whether the unmanned aerial vehicle is in a flight-ready state based on the status identified by the second identification means. This makes it possible to accurately determine whether the unmanned aerial vehicle can fly from the perspective of the weather status.

[0102] [5] In the information processing device described in [4] above, the display control means is characterized in that when the third determination means determines that the unmanned aerial vehicle is not in a flyable state, detailed information about the weather is displayed on the screen, thereby enabling the operator to quickly be informed of the type of poor weather that is preventing the unmanned aerial vehicle from flying.

[0103] [6] In the information processing device described in any one of [1] to [5] above, the display control means displays, on the screen together with the operation buttons, at least one of aircraft information acquired by the unmanned aerial vehicle and base information acquired by equipment at a base from which the unmanned aerial vehicle can take off and land. This allows the operator to quickly input the result of the takeoff decision using the operation buttons while viewing at least one of the aircraft information and base information.

[0104] [7] In the information processing device described in [6] above, the aircraft information includes an aircraft image captured by a camera equipped in the unmanned aerial vehicle, the base information includes a base image captured by a camera equipped in the device, and the display control means, when the operation button is controlled to accept takeoff permission for the unmanned aerial vehicle, highlights at least one of the aircraft image and the base image to prompt the operator to check that image. This makes it possible to quickly notify the operator that the time to make a takeoff decision has arrived.

[0105] [8] In the information processing device described in [6] or [7] above, the aircraft information includes an aircraft image captured by a camera equipped on the unmanned aerial vehicle, and the base information includes a base image captured by a camera equipped on the device, and the information processing device further includes a fourth determination means that, when the determination timing arrives, analyzes at least one of the aircraft image and the base image to determine whether or not an abnormality exists, and the display control means, when the fourth determination means determines that an abnormality exists, highlights the image of the aircraft image and the base image that has been determined to have the abnormality so as to notify the operator that an abnormality exists. This makes it possible to quickly notify the operator that an abnormality has occurred in the unmanned aerial vehicle based on the analysis of the aircraft image or the base image.

[0106] [9] In the information processing device described in [8] above, when the fourth determination means determines that there is no abnormality, the display control means displays at least one image of the aircraft image and the base image to indicate to the operator that there is no abnormality. This makes it possible to quickly inform the operator that there is no abnormality regarding the unmanned aerial vehicle based on the analysis of the aircraft image or the base image.

[0107]

[10] In the information processing device described in any one of [7] to [9] above, the display control means is characterized in that when takeoff permission for the unmanned aerial vehicle is accepted via the operation button, the highlighting of the image is stopped. This allows the operator to be quickly notified that the takeoff decision has been accepted.

[0108]

[11] The control method disclosed herein is characterized by including the steps of: displaying an operation button on the screen of a terminal used by an operator for receiving takeoff permission for the unmanned aircraft from the operator; determining whether the timing for the operator to decide whether to allow the unmanned aircraft to take off has arrived; and controlling the operation button so that takeoff permission for the unmanned aircraft cannot be accepted before the decision timing arrives, while controlling the operation button so that takeoff permission for the unmanned aircraft can be accepted after the decision timing arrives if the unmanned aircraft is in a flyable state.

[0109]

[12] The program disclosed herein is characterized by causing a computer included in a terminal used by an operator to execute the following steps: displaying an operation button on the screen of the terminal used by the operator for accepting takeoff permission for the unmanned aircraft from the operator; determining whether the timing for the operator to decide whether to allow the unmanned aircraft to take off has arrived; and controlling the operation button so that takeoff permission for the unmanned aircraft cannot be accepted before the judgment timing arrives, while controlling the operation button so that takeoff permission for the unmanned aircraft can be accepted after the judgment timing arrives if the unmanned aircraft is in a state capable of flight. [Explanation of symbols]

[0110] 11 Power supply section 12 Drive unit 13 Positioning unit 14 Communications Department 15 Sensor section 16 Memory section 17 Control Unit 21 Operation / display section 22 Communications Department 23 Memory section 24 Control Unit 31 Communications Department 32 Storage section 33 Control Unit 331 Display control unit 332 Timing arrival judgment unit 333 Button control section 334 Status Identification Section 335 Flight Readiness Determination Unit 336 Abnormality determination section Dn Drone Tm Operator Terminal MS Management Server EM base equipment S Remote Monitoring System

Claims

1. a display control means for displaying operation buttons for receiving takeoff permission for the unmanned aerial vehicle from an operator on a screen of a terminal used by the operator; a first determination means for determining whether or not a time has come for the operator to determine whether or not to permit takeoff of the unmanned aerial vehicle; a button control means for controlling the operation button so that takeoff permission for the unmanned aerial vehicle cannot be accepted before the judgment timing arrives, and for controlling the operation button so that takeoff permission for the unmanned aerial vehicle can be accepted after the judgment timing arrives if the unmanned aerial vehicle is in a flight-ready state; An information processing device comprising:

2. a first identification means for identifying a status of the unmanned aerial vehicle; a second determination means for determining whether the unmanned aerial vehicle is in a flight-ready state based on the status identified by the first identification means; 2. The information processing apparatus according to claim 1, further comprising:

3. The information processing device described in claim 2, characterized in that the display control means displays on the screen information indicating the factors that determined that the unmanned aircraft is not in a flyable state when the second determination means determines that the unmanned aircraft is not in a flyable state.

4. A second identification means for identifying a weather status of a base where the unmanned aerial vehicle can take off and land; 2. The information processing device according to claim 1, further comprising a third determination means for determining whether the unmanned aerial vehicle is in a flight-ready state based on the status identified by the second identification means.

5. The information processing device described in claim 4, characterized in that the display control means displays detailed information about the weather on the screen when the third determination means determines that the unmanned aircraft is not in a state suitable for flight.

6. The information processing device described in claim 1, characterized in that the display control means displays at least one of aircraft information acquired by the unmanned aerial vehicle and base information acquired by equipment at a base from which the unmanned aerial vehicle can take off and land on the screen together with the operation buttons.

7. The aircraft information includes an aircraft image captured by a camera equipped on the unmanned aerial vehicle, The location information includes a location image captured by a camera provided in the device, The information processing device described in claim 6, characterized in that when the operation button is controlled to accept takeoff permission for the unmanned aerial vehicle, the display control means highlights the image to prompt the operator to confirm at least one of the aircraft image and the base image.

8. The aircraft information includes an aircraft image captured by a camera equipped on the unmanned aerial vehicle, The location information includes a location image captured by a camera provided in the device, a fourth determination means for determining whether or not an abnormality exists by analyzing at least one of the aircraft image and the base image when the determination timing arrives; The information processing device according to claim 6, characterized in that, when the fourth determination means determines that an abnormality exists, the display control means highlights the image of the aircraft image and the base image that is determined to contain the abnormality so as to indicate to the operator that an abnormality exists.

9. The information processing device according to claim 8, characterized in that, when the fourth determination means determines that there is no abnormality, the display control means displays at least one image of the aircraft image and the base image to indicate to the operator that there is no abnormality.

10. The information processing device according to claim 7, wherein the display control means stops highlighting the image when takeoff permission for the unmanned aerial vehicle is accepted via the operation button.

11. a step of displaying an operation button for receiving takeoff permission for the unmanned aerial vehicle from an operator on a screen of a terminal used by the operator; a step of determining whether or not a timing for the operator to determine whether or not to permit takeoff of the unmanned aerial vehicle has arrived; controlling the operation button so that takeoff permission for the unmanned aerial vehicle cannot be accepted before the judgment timing arrives, and controlling the operation button so that takeoff permission for the unmanned aerial vehicle can be accepted after the judgment timing arrives if the unmanned aerial vehicle is in a flyable state; A control method comprising:

12. The computer included in the terminal used by the operator a step of displaying an operation button for receiving takeoff permission for the unmanned aerial vehicle from the operator on a screen of a terminal used by the operator; a step of determining whether or not a timing for the operator to determine whether or not to permit takeoff of the unmanned aerial vehicle has arrived; controlling the operation button so that takeoff permission for the unmanned aerial vehicle cannot be accepted before the judgment timing arrives, and controlling the operation button so that takeoff permission for the unmanned aerial vehicle can be accepted after the judgment timing arrives if the unmanned aerial vehicle is in a flyable state; A program characterized by executing the following.

Images