Information processing device, information display method, and program

Abstract

To provide an information processing device capable of preventing a human error by an operator when a plurality of unmanned aircraft are monitored during the same period, an information display method, and a program.SOLUTION: A management server MS specifies a drone status of each of the plurality of drones Dn of which an operator OPm is in charge, specifies the drone Dn whose monitoring-required timing has arrived based on each drone status, and displays the display control data for displaying monitoring information of the specified drone Dn on a drone monitoring screen of an operator terminal Tm.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to the technical field of systems such as those that allow an operator to monitor multiple unmanned aerial vehicles. [Background technology]

[0002] Conventionally, systems that allow an operator to monitor and operate multiple unmanned aerial vehicles have been known. For example, Patent Document 1 discloses a system in which at least one operator monitors each of multiple UAVs (Unmanned Aerial Vehicles) using a computing device that can communicate with each of the UAVs and controls the takeoff, ascent, descent, landing, etc. of the UAVs. Patent Document 1 describes an example in which 10 operators monitor 500 UAVs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-529507 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when at least one operator monitors multiple unmanned aerial vehicles over the same period, the burden on the operator increases, raising concerns about the possibility of human error.

[0005] Therefore, the present invention has been made in consideration of the above points, and one example of its objective is to provide an information processing device, an information display method, and a program that can prevent human error by an operator when monitoring multiple unmanned aerial vehicles over the same period. [Means for solving the problem]

[0006] (Application Example 1) In order to solve the above problem, the information processing device of this application example is characterized by comprising: a first identification means for identifying the status of each of a plurality of unmanned aerial vehicles under the operator's responsibility; a second identification means for identifying an unmanned aerial vehicle among the plurality of unmanned aerial vehicles for which a monitoring timing has arrived that requires monitoring based on each status identified by the first identification means; and a processing means for notifying the operator via a terminal used by the operator of monitoring information that prompts monitoring of the unmanned aerial vehicle identified by the second identification means, the monitoring information including information about the unmanned aerial vehicle.

[0007] (Application Example 2) The information display method of this application example is an information display method executed by one or more computers, and is characterized by including the steps of: identifying the status of each of multiple unmanned aerial vehicles under the operator's responsibility; identifying an unmanned aerial vehicle among the multiple unmanned aerial vehicles that has reached a monitoring timing requiring monitoring based on each of the identified statuses; and notifying the operator via a terminal used by the operator of monitoring information that prompts monitoring of the identified unmanned aerial vehicle and includes information about the unmanned aerial vehicle.

[0008] (Application Example 3) The program related to this application example is characterized in that it causes a computer included in a terminal used by an operator to execute the steps of: obtaining the status of each of multiple unmanned aerial vehicles under the operator's responsibility from a specified server; identifying an unmanned aerial vehicle among the multiple unmanned aerial vehicles for which a monitoring timing has arrived, based on the status of each of the unmanned aerial vehicles; and notifying the operator of monitoring information that prompts monitoring of the identified unmanned aerial vehicle and includes information about the unmanned aerial vehicle. [Effects of the Invention]

[0009] According to the present invention, human error by an operator can be prevented when multiple unmanned aerial vehicles are monitored over the same period. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a remote monitoring system S. [Figure 2] FIG. 1 is a diagram illustrating an example of the general configuration of a drone Dn. [Figure 3] FIG. 2 is a diagram illustrating an example of a schematic configuration of an operator terminal Tm. [Figure 4] FIG. 10 is a diagram showing a display example (1) of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 5] FIG. 10 is a diagram showing a display example (2) of a drone monitoring screen displayed on the operator terminal T1 of the operator OP1. [Figure 6] FIG. 2 is a diagram illustrating an example of a schematic configuration of a management server MS. [Figure 7] FIG. 10 is a diagram illustrating an example of a monitoring priority master table. [Figure 8] FIG. 2 is a diagram illustrating an example of functional blocks in a control unit 33. [Figure 9] This is a conceptual diagram showing the timing at which drone D1 needs to be monitored and the timing at which drone D2 needs to be monitored in chronological order. [Figure 10] This is a conceptual diagram showing the timing at which drone D1, drone D2, and drone D3 need to be monitored in chronological order. [Figure 11] This is a conceptual diagram showing the timing at which drone D1, drone D2, and drone D3 need to be monitored in chronological order. [Figure 12] This figure shows an example of a drone monitoring screen display in which the onboard camera image of drone D1 and the base camera image of drone D3 are displayed in a position that does not interfere with the display of monitoring information for drone D2. [Figure 13] 10 is a flowchart showing an example of a monitoring information display process executed by a control unit 33 of a management server MS. [Figure 14] 14 is a flowchart showing an example of an abnormality detection process in step S13 shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described below with reference to the drawings. The following embodiment is an embodiment in which the present invention is applied to a remote monitoring system capable of remotely monitoring drones. Note that in this embodiment, a drone will be used as an example of an unmanned aerial vehicle, but the present invention can also be applied to flying robots other than drones.

[0012] [ 1. Overview of the configuration and operation of the remote monitoring system S ] First, with reference to Fig. 1, an overview of the configuration and operation of a remote monitoring system S according to this embodiment will be described. Fig. 1 is a diagram showing an example of the schematic configuration of the remote monitoring system S. As shown in Fig. 1, the remote monitoring system S is configured to include a plurality of drones Dn (n = 1, 2, 3, etc.), a plurality of operator terminals Tm (m = 1, 2, etc.), and a management server MS (an example of an information processing device and a predetermined server). The drones Dn, the operator terminals Tm, and the management server MS are each connected to a communication network NW. The communication network NW is configured, for example, by the Internet, a mobile communication network, its wireless base stations, etc.

[0013] The drone Dn is an example of an unmanned aerial vehicle, also known as a multicopter or UAV. The drone Dn takes off in response to a takeoff command from a GCS (Ground Control Station) and is capable of flying autonomously, and is used for, for example, delivery, surveying, photography, and monitoring. The GCS is installed as an application on, for example, an operator terminal Tm, and is configured to link with a management server MS. The drone Dn can also fly under remote control from the ground using a control terminal (equipped with a GCS) used by the operator.

[0014] Drone Dn is under the jurisdiction of one of multiple drone bases Bm (in other words, it belongs to one of the drone bases Bm). Drone base Bm is a base (e.g., a facility) from which drone Dn can take off and land. In the example of FIG. 1, drones D1 to D4 are each under the jurisdiction of drone base B1, and are configured to depart from and return to drone base B1. Drones D5 to D9 are each under the jurisdiction of drone base B2, and are configured to depart from and return to drone base B2. However, one drone Dn may be under the jurisdiction of multiple drone bases Bm. For example, drone D11 (not shown) may depart from drone base B2 and return to drone base B1. The number of drones Dn under the jurisdiction of one drone base Bm is not particularly limited.

[0015] Furthermore, at the drone base Bm, a port Pm used for takeoff and landing of the drone Dn and a base device Em (an example of a device) used to monitor the drone Dn are installed. At the drone base Bm, base staff (manually) perform a pre-flight inspection (aircraft condition check) of the drone Dn. For example, the base staff visually inspects a predetermined part of the drone Dn for each inspection item in the manual inspection, or inspects the drone Dn by touching a predetermined part of the drone Dn. Then, manual inspection result information (an example of base information) indicating the results of the inspection by the base staff is transmitted from the base staff's terminal (an example of a device), such as a smartphone, to the management server MS via the communication network NW. Furthermore, the drone Dn that has undergone the pre-flight inspection is placed at the port Pm and takes off (departs) from the port Pm according to a predetermined drone schedule. Furthermore, the drone Dn that returns to the drone base Bm lands at the port Pm. Note that multiple ports Pm may be installed at one drone base Bm.

[0016] The base device Em is connected to the communication network NW and includes a base camera (e.g., an RGB camera or an infrared camera) for monitoring the drone Dn. The base camera stores a base ID (identification information) for identifying the drone base Bm. The base camera, for example, continuously captures images of the drone Dn placed at the port Pm. Base image information (an example of base information) representing base images (still images or moving images) captured by the base camera is transmitted from the base device Em to the management server MS along with the base ID. Note that multiple base cameras may be installed at one drone base Bm. The base device Em may also be equipped with a wind sensor that detects (measures) at least one of wind speed and wind direction. The measurement information (an example of base information) measured by the wind sensor is transmitted from the base device Em to the management server MS. The base device Em may also be equipped with at least one of a temperature sensor, a humidity sensor, a precipitation (snow) amount sensor, and an atmospheric pressure sensor. The measurement information (an example of base information) measured by these sensors is transmitted from the base device Em to the management server MS.

[0017] The operator terminal Tm is a terminal used by an operator OPm who remotely monitors multiple drones Dn. The operator OPm monitors the drones Dn under his / her responsibility while viewing information displayed on the screen (user interface screen) of the operator terminal Tm. Such monitoring includes, for example, checking the pre-flight inspection status of the multiple drones Dn, monitoring the aircraft status before flight, and monitoring the aircraft status during flight. Monitoring may also include actions involving monitoring (such as operations by the operator OPm). For example, during the operation of a drone Dn, there are times when monitoring of the drone Dn is required. Such times are called monitoring-required timings (in other words, attention-required timings). The monitoring-required timings may be a point in time, but because monitoring requires a certain amount of time, it is desirable for them to have a time length (time width). Therefore, the monitoring-required timings can be called monitoring-required times. The monitoring-required timings are registered (set) for each drone status that requires monitoring among the multiple statuses of the drone Dn (hereinafter referred to as "drone statuses") Examples of drone statuses that require monitoring include "arrival at base (drone base)", "before takeoff decision (takeoff permission)", "after takeoff decision (takeoff planned or takeoff decision)", "takeoff (automatic takeoff)", and "arrived at destination".

[0018] For example, in the case of "Arrival at base," a monitoring timing is registered for the time period from t1 minutes before to t2 minutes after the drone Dn arrives at the drone base Bm. In the case of "Before takeoff decision," a monitoring timing is registered for the time period from t3 minutes before to t3 minutes after the operator OPm makes the takeoff decision. Here, the takeoff decision is made, for example, by pressing a takeoff decision button displayed on the screen of the operator terminal Tm. In the case of "After takeoff decision," a monitoring timing is registered for the time period from t4 minutes after the drone Dn starts takeoff due to the operator OPm's takeoff decision (pressing the takeoff decision button). In the case of "Takeoff (automatic takeoff)," a monitoring timing is registered for the time period from t5 minutes after the drone Dn starts automatically takeoff from the drone base Bm at the scheduled time. In the case of "Arrival at destination," a monitoring timing is registered for the time period from t6 minutes before to t7 minutes after the drone Dn arrives at the destination. t1 to t7 may be the same or different, and are set to, for example, about 1 to 5 minutes. When the drone Dn is used for delivery, the destination is the delivery destination of the goods. Another example of a drone status that requires monitoring is "abnormal occurrence."

[0019] In this embodiment, monitoring refers to, for example, the operator OPm visually gazing (watching carefully) at a specific portion of the screen. Here, the specific portion of the screen includes a portion that requires the operator OPm to pay close attention to. When a monitoring timing requiring monitoring arrives (in other words, the current time becomes the start point of the monitoring timing), monitoring information including information about the drone Dn for which the monitoring timing has arrived (e.g., the drone name) is notified to the operator OPm from the management server MS via the operator terminal Tm. The monitoring information of the drone Dn also serves to prompt the operator OPm to monitor the drone Dn for which the monitoring timing has arrived. The monitoring information of the drone Dn includes, for example, at least one of aircraft information acquired by the drone Dn and base information acquired by the base device Em. For example, the monitoring information of the drone Dn may be notified to the operator OPm by being displayed on the screen of the operator terminal Tm. Note that the monitoring information of the drone Dn may also be notified to the operator OPm by being output as audio from the operator terminal Tm (speaker).

[0020] The management server MS is composed of one or more server computers that manage information about drone bases Bm for each drone base Bm. The information about drone bases Bm includes, for example, information about drones Dn under the jurisdiction of the drone base Bm, information about operators OPm that monitor the drones Dn, and information about operator terminals Tm used by the operators OPm. The management server MS identifies the drones Dn managed by the operator OPm who logs in by operating the operator terminal Tm and displays a list (hereinafter referred to as the "drone list") showing the identified drones Dn on the operator terminal Tm of the operator OPm. When the drone list is displayed, if a monitoring timing arrives for any of the drones Dn, the management server MS can automatically display monitoring information for the drone Dn whose monitoring timing has arrived on the operator terminal Tm. The management server MS periodically or irregularly receives detailed weather information about the drone base Bm from a weather management server (not shown) via a communication network NW.

[0021] [ 1-1. Drone Dn configuration and functions ] Next, the configuration and functions of the drone Dn will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the schematic configuration of the drone Dn. As shown in Fig. 2, the drone Dn includes a power supply unit 11, a drive unit 12, a positioning unit 13, a communication unit 14, a sensor unit 15, a memory unit 16, and a control unit 17. Furthermore, the drone Dn includes a propeller (rotor) which is a horizontal rotor, and an arm pipe (including an arm joint) for attaching the propeller to the drone body (housing). When the drone Dn is used to deliver goods, the drone Dn is provided with a holding mechanism for holding the goods.

[0022] The power supply unit 11 includes a detachable battery (power storage device) and the like. The power supply unit 11 supplies (feeds) the power stored in the battery to each part of the drone Dn. The power supply unit 11 also continuously measures the remaining battery charge. Battery information indicating the remaining battery charge measured by the power supply unit 11 is output to the control unit 17. The drive unit 12 includes a motor, a rotating shaft, and the like. The drive unit 12 rotates multiple rotors using the motor, rotating shaft, and the like that are driven in accordance with control signals output from the control unit 17.

[0023] The positioning unit 13 includes a radio wave receiver, an altitude sensor, and the like. The positioning unit 13 receives radio waves transmitted from positioning satellites of a Global Navigation Satellite System (GNSS), such as a Global Positioning System (GPS), using the radio wave receiver, and sequentially detects the current position of the drone Dn based on the radio waves. The current position of the UAV 1 may be represented by the latitude and longitude of the UAV 1, or by the latitude, longitude, and altitude of the UAV 1. Here, the positioning satellites may include satellites used by multiple satellite positioning systems, such as GPS satellites, Michibiki satellites, and Galileo satellites. Position information indicating the current position detected by the positioning unit 13 is continuously output to the control unit 17. At this time, acquisition number information indicating the number of acquisitions of positioning satellites (satellite acquisition number) acquired by the positioning unit 13 is continuously output to the control unit 17. The positioning unit 13 may also detect the altitude of the drone Dn using an altitude sensor. In this case, the location information indicating the current location of the drone Dn includes the altitude detected by the altitude sensor. The communication unit 14 has an antenna and a wireless communication function, and controls communication performed via the communication network NW.

[0024] The sensor unit 15 includes various sensors used to control the drone Dn. The various sensors include, for example, a compass (geomagnetic sensor), a gyro (three-axis angular velocity sensor), a three-axis acceleration sensor, a barometric pressure sensor, a gimbal, an optical sensor, and a rangefinder. The optical sensor includes an on-board camera (for example, an RGB camera or an infrared camera). The on-board camera is configured to, for example, continuously capture images of the surroundings of the drone Dn (for example, in front of or below the drone Dn). The orientation of the on-board camera (in front of or below the drone Dn) can be controlled by the control unit 17. The sensing information sensed by the sensor unit 15 is output to the control unit 17. The storage unit 16 is configured with a non-volatile memory or the like and stores various programs and data. The storage unit 16 also stores an on-board ID (identification information) for identifying the drone Dn.

[0025] The control unit 17 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and controls the drone Dn based on position information from the positioning unit 13 and sensing information from the sensor unit 15. Such control includes control of the propeller rotation speed, and control of the position, attitude, and direction of travel of the drone Dn. The position information of the drone Dn (i.e., the position information from the positioning unit 13) is transmitted to the management server MS via the communication network NW together with aircraft image information (an example of aircraft information) representing an aircraft image (a still image or a moving image) captured by the aircraft camera, battery information (an example of aircraft information) from the power supply unit 11, capture number information (an example of aircraft information) from the positioning unit 13, and the aircraft ID of the drone Dn.

[0026] The control unit 17 also has a self-diagnosis function and performs automatic inspections to check whether certain parts of the drone Dn (e.g., the power supply unit 11, drive unit 12, positioning unit 13, communication unit 14, and sensor unit 15) are operating normally. Items inspected in the automatic inspection include, for example, remaining battery power, battery cell balance, GPS, compass, gyro, acceleration sensor, barometric pressure sensor, gimbal, optical sensor, and range finder. Automatic inspection result information indicating the results of the automatic inspection by the drone Dn (i.e., the control unit 17) is transmitted to the management server MS via the communication network NW. Note that the automatic inspection by the drone Dn is performed before the drone Dn takes flight, but some of the inspection items of the automatic inspection are also performed while the drone Dn is in flight.

[0027] [ 1-2.Configuration and functions of operator terminal Tm ] Next, the configuration and functions of the operator terminal Tm will be described with reference to FIG. 3. FIG. 3 is a diagram illustrating an example of a schematic configuration of the operator terminal Tm. The operator terminal Tm includes an operation and display unit 21, a communication unit 22, a storage unit 23, and a control unit 24. For example, a personal computer can be applied to the operator terminal Tm. The operator terminal Tm may also include a sound processing unit and a speaker. The operation and display unit 21 has an input function for accepting input (selection) by the operator OPm's finger, pen, or mouse, and a display function for displaying various screens on the display. The operator terminal Tm may be provided with multiple displays. The various screens include a login screen for the operator OPm to log in, a drone monitoring screen for the operator OPm to monitor the drone Dn, and an action request notification screen for notifying the operator OPm of a request for a predetermined action. The drone monitoring screen and the action request notification screen may be simultaneously displayed on separate displays.

[0028] The communication unit 22 is responsible for controlling communications carried out via the communication network NW. The storage unit 23 is composed of non-volatile memory and the like, and stores various programs (program code groups) and data. The various programs include an operating system (OS), a monitoring application, a GCS, and a web browser. The monitoring application is a program that mainly acquires and displays information about multiple drones Dn managed by an operator OPm from the management server MS. The monitoring application may be downloaded to the operator terminal Tm from a specified server.

[0029] The control unit 24 (an example of a computer) includes a CPU, a ROM, a RAM, etc., and executes processing in accordance with a monitoring application stored in the ROM (or the storage unit 23). When the monitoring application is started in response to an instruction from the operator OPm, the control unit 24 displays a login screen on the display. Then, when the operator OPm inputs a user ID and a password through the login screen, the control unit 24 transmits a login request including the user ID and the password to the management server MS via the communication unit 22 and the communication network NW. The user ID is identification information for identifying the operator OPm.

[0030] Then, the management server MS performs login processing in response to the login request, and when the operator OPm logs in, the management server MS transmits display control data for displaying a drone list showing the multiple drones Dn managed by the logged-in operator OPm. As a result, the control unit 24 displays a drone monitoring screen including the drone list on the display. Note that in the login processing, it is determined whether or not the pair of user ID and password included in the login request has been registered. Then, if the pair of user ID and password has been registered, the operator OPm using the operator terminal Tm that transmitted the login request is identified, and the operator OPm logs in.

[0031] Here, the multiple drones Dn shown in the drone list may be under the jurisdiction of the same drone base Bm, or may be under the jurisdiction of different drone bases Bm. In the latter case, the drone list shows the drones Dn (e.g., drones D1 to D4 and drones D5 to D9) belonging to each of the multiple drone bases Bm (e.g., drone base B1 and drone base B2). This allows the operator OPm to monitor each drone Dn under the jurisdiction of the multiple drone bases Bm. Note that the display control data may be data for a web page displayed by a web browser. Display data for the drone monitoring screen may be incorporated into the monitoring application.

[0032] 4 and 5 are diagrams showing an example of a drone monitoring screen displayed on the operator terminal T1 of operator OP1. As shown in FIGS. 4 and 5, the drone monitoring screen displays a drone list L showing multiple drones D1 to D4 (drone names: Drone A to D), and further includes a drone basic information display area A1, a drone detailed information display area A2, an aircraft camera display area A3, and a base camera display area A4. The drone list L is composed of simple information display areas LA1 to LA4 corresponding to each of the drones D1 to D4 managed by operator OP1. The simple information display areas LA1 to LA4 are scrollable. The names (drone names) of the drones D1 to D4 are displayed in the simple information display areas LA1 to LA4. Note that the drone list L may also include simple information display areas corresponding to each of the drones D5 to D9 managed by operator OP1.

[0033] The simple information display areas LA1 to LA4 can be selected by the operator OP1. In other words, the drones D1 to D4 can be selected via the simple information display areas LA1 to LA4. In the display examples of FIGS. 4 and 5, the drone D1 is selected by the operator OP1 selecting (pressing) the simple information display area LA1. Therefore, the monitoring information (obtained as needed from the management server MS) of the selected drone D1 is displayed in the drone basic information display area A1, the drone detailed information display area A2, the aircraft camera display area A3, and the base camera display area A4. Here, the monitoring information includes, for example, manual inspection result information, base image information, aircraft data information, aircraft image information, detailed weather information, etc. In the display example of FIG. 4, the drone D1 is placed at port P1 at the drone base B1 for takeoff and is waiting for the operator OP1 to decide whether to takeoff (permit takeoff). Meanwhile, in the display example of FIG. 5, the drone D1 is flying (ascending) above port P1.

[0034] The drone basic information display area A1 displays basic information about drone D1, drone status of drone D1, drone phase of drone D1, drone schedule of drone D1, weather status of drone base B1 (hereinafter referred to as "weather status"), and takeoff decision button B. Takeoff decision button B is an operation button for receiving takeoff decision (takeoff permission) for drone D1 from operator OP1. Note that measurement information measured by a sensor equipped in base device Em may also be displayed in drone basic information display area A1. In the display examples of FIGS. 4 and 5, the basic information about drone D1 includes, for example, drone name (name of drone D1), type, model number, manufacturer, base name (name of drone base B1), and port name (name of port P1), but is not limited thereto. The drone status of drone D1 indicates the current state (condition) of drone D1. In the display example of Figure 4, the drone status of drone D1 is "before takeoff decision", while in the display example of Figure 5, the drone status of drone D1 is "takeoff".

[0035] The drone phase of drone D1 roughly indicates the stage drone D1 is in. In the display examples of FIGS. 4 and 5, the drone phases of drone D1 are displayed in the phase display area A11 as "Preparing for Delivery," "In Flight," and "Return." In the display example of FIG. 4, "Preparing for Delivery" is displayed in the phase display area A11, while in the display example of FIG. 5, "In Flight" is displayed. The display content of the phase display area A11 changes as the drone phase of drone D1 changes. The weather status indicates the current weather condition (situation) at drone base B1 based on detailed weather information. The drone schedule of drone D1 includes, for example, the scheduled times for drone D1 to arrive at the base, take off, and arrive at the destination.

[0036] The drone detailed information display area A2 displays an aircraft status check tab TB1, an aircraft data tab TB2, and a weather status tab TB3. In the display example of FIG. 4, the aircraft status check tab TB1 is selected and displayed among the tabs TB1 to TB3, and therefore aircraft status check information (list) is displayed below the tabs TB1 to TB3. The aircraft status check information is based on the manual inspection result information described above. For example, "OK" displayed in association with the inspection item "arm joint looseness" included in the aircraft status check information indicates that the result (inspection result) of this manual inspection (aircraft status check) is good. If the result of this manual inspection is not good (i.e., it is bad), for example, "NG" is displayed in association with that inspection item.

[0037] Meanwhile, in the display example of FIG. 5, the aircraft data tab TB2 is selected and displayed among the tabs TB1 to TB3, and therefore aircraft data information (list) is displayed below the tabs TB1 to TB3. The aircraft data information is based on the above-mentioned battery information (remaining battery charge) and acquisition number information (number of acquired satellites), etc. A check mark displayed in association with the item "remaining battery charge" included in the aircraft data information indicates, for example, that the remaining battery charge is good. Each of the tabs TB1 to TB3 can be manually selected by the operator OP1, but they are automatically switched depending on the drone status or drone phase of drone D1. For example, when the weather status tab TB3 is selected, detailed weather information for drone base B1 is displayed below the tabs TB1 to TB3.

[0038] The on-board camera display area A3 displays video captured by the on-board camera of the drone D1 (hereinafter referred to as "on-board camera video"). Meanwhile, the base camera display area A4 displays video captured by the base camera of the base device E1 (hereinafter referred to as "base camera video"). In the display example of FIG. 4, the drone status of the drone D1 is "before takeoff determination," so the base camera video is highlighted (for example, the frame A41 in the base camera display area A4 is displayed in red). Highlighting refers to displaying in a conspicuous display manner (for example, a conspicuous display color or display size). It is preferable to highlight portions of the screen displaying the monitoring information (i.e., various information and images) that require particular attention depending on the drone status. In particular, examples of highlighting an image include displaying the frame of the display area of ​​the image in a conspicuous display manner (for example, displaying the frame in a more conspicuous color than usual, displaying the frame thicker than usual, or displaying the frame flashing), or displaying a message (for example, text strings) within the display area of ​​the image. In addition to the base camera image, basic information about drone D1 may also be highlighted.

[0039] In this way, when the drone status of drone D1 is “before takeoff determination,” operator OP1 selects (presses) the takeoff determination button B in the drone basic information display area A1, and a takeoff permission command is sent to the management server MS or drone D1. As a result, the display of the drone status of drone D1 switches to “after takeoff determination” (not shown). After that, when drone D1 takes off from port P1, as shown in FIG. 5, the display of the drone status of drone D1 switches to “in flight.” Synchronously, the image from the aircraft camera is highlighted in addition to the image from the base camera (for example, frame A41 in the base camera display area A4 and frame A31 in the aircraft camera display area A3 are displayed in red). When the takeoff determination button B is displayed on the drone monitoring screen, monitoring information including information about the drone Dn being monitored (for example, the drone name) may be displayed on the action request notification screen, along with information prompting operator OPm to make a takeoff determination.

[0040] [ 1-3.Configuration and Functions of Management Server MS ] Next, the configuration and functions of the management server MS will be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of a schematic configuration of the management server MS. As shown in FIG. 6, the management server MS includes a communication unit 31, a storage unit 32, a control unit 33, and the like. The communication unit 31 controls communication via the communication network NW. The communication unit 31 receives manual inspection result information, base image information, measurement information, and base ID transmitted from the base device E. The communication unit 31 receives battery information, capture number information, aircraft image information, position information of the drone Dn, automatic inspection result information, and aircraft ID transmitted from the drone Dn. The management server MS can recognize the current position of the drone Dn based on the position information of the drone Dn. The communication unit 31 also receives a login request transmitted from the operator terminal Tm. The communication unit 31 also receives detailed weather information transmitted from the weather management server.

[0041] The storage unit 32 is configured, for example, with a hard disk drive or the like, and stores various programs including an operating system and applications. Here, the applications include programs for executing an information display method. The storage unit 32 also stores a monitoring priority master table. The monitoring priority master table is a table that registers monitoring priorities (attention priorities) and monitoring timings (criteria) in association with each drone status that requires monitoring.

[0042] FIG. 7 is a diagram illustrating an example of a monitoring priority master table. In the example of FIG. 7, a monitoring priority of “1” is associated with “arrival at destination,” a monitoring priority of “2” is associated with “after takeoff determination,” a monitoring priority of “3” is associated with “arrival at base,” and a monitoring priority of “4” is associated with “takeoff.” In this example, a monitoring priority of “1” is the highest. Such monitoring priorities are determined, for example, taking into consideration the impact on people and objects if the monitoring information of drone Dn is not closely monitored. Note that the monitoring priority master table may also register “abnormal occurrence” as a drone status requiring monitoring, and the highest monitoring priority of “0” may be registered for “abnormal occurrence.” Information associating the monitoring priority and the monitoring timing for each drone status requiring monitoring may be written in a program for executing the information display method.

[0043] Furthermore, the monitoring priority master table may be registered with highlighting information associated with each drone status requiring monitoring to highlight portions that require particular attention. For example, "after takeoff determination" is associated with highlighting information for highlighting a base image (e.g., base camera image) and basic information about drone Dn. Furthermore, "takeoff" is associated with highlighting information for highlighting a base image and an aircraft image (e.g., aircraft camera image). Furthermore, "arrived at destination" is associated with highlighting information for highlighting an aircraft image. Note that the highlighting information for highlighting portions that require particular attention for each drone status requiring monitoring may be written in a program for executing the information display method.

[0044] Furthermore, the storage unit 32 has constructed therein a base management database (DB) 321, a drone management database (DB) 322, and an operator management database (DB) 323. The base management database 321 is a database for managing information related to drone bases Bm. The base management database 321 stores, for example, the base ID of the drone base Bm, base image information, measurement information, detailed weather information, weather status, and the aircraft ID of the drone Dn under the jurisdiction of the drone base Bm, all associated with each drone base Bm. Note that the base image information, measurement information, and detailed weather information may be updated as appropriate each time they are received by the communication unit 31.

[0045] The drone management database 322 is a database for managing information related to drones Dn. The drone management database 322 stores the drone Dn's aircraft ID, location information, basic information, drone status, drone phase, drone schedule, manual inspection result information, aircraft data information, and aircraft image information, etc., in association with each drone Dn. The aircraft data information includes, for example, battery information and capture count information. The manual inspection result information, aircraft data information, and aircraft image information may be updated as appropriate each time they are received by the communication unit 31. The drone status and drone phase are updated as appropriate, for example, in accordance with various information (including information from the drone Dn) received by the communication unit 31, the drone schedule, or instruction information from an administrator or the like.

[0046] The operator management database 323 is a database for managing information related to the operator OPm. The operator management database 323 stores the user ID, password, login status, and the aircraft ID and name of each of the multiple drones Dn that the operator OPm is in charge of, in association with each operator OPm. Here, the login status indicates whether the operator OPm is logged in or not.

[0047] The control unit 33 (an example of a computer) includes a CPU, a ROM, and a RAM. FIG. 8 is a diagram illustrating an example of functional blocks in the control unit 33. The CPU may be a general-purpose processor, a special-purpose processor, or a processor including transistors and other integrated circuits (electrical circuits or electronic circuits). As illustrated in FIG. 8, the control unit 33 functions as a status identification unit 331 (an example of a first identification unit), a monitoring-required drone identification unit 332 (an example of a second identification unit), a monitoring information display control unit 333 (an example of a processing unit), a monitoring information display determination unit 334 (an example of a first determination unit and a second determination unit), a priority acquisition unit 335 (an example of an acquisition unit), a priority determination unit 336 (an example of a third determination unit), a monitoring-required timing end determination unit 337 (an example of a fourth determination unit), a monitoring-required drone determination unit 338 (an example of a fifth determination unit), and an abnormality detection unit 339 (an example of a detection unit), in accordance with a program (a group of program codes) stored in the ROM or the storage unit 32, for example.

[0048] The status identification unit 331 identifies the current drone status of each of the multiple drones Dn managed by the logged-in operator OPm, for example, from the drone management database 322. The monitoring-required drone identification unit 332 identifies a drone Dn among the multiple drones Dn managed by the operator OPm for which a monitoring timing that requires monitoring has arrived (started) based on each drone status identified by the status identification unit 331. For example, the monitoring-required drone identification unit 332 may identify a drone Dn for which a monitoring timing has arrived by determining whether a monitoring timing has arrived for each drone Dn based on the monitoring timing (reference) associated with each drone status identified by the status identification unit 331 and the current time in the monitoring priority master table. Note that the arrival of the monitoring timing means that the current time has become the start point of the monitoring timing.

[0049] The monitoring information display control unit 333 transmits display control data for displaying the monitoring information of the drone Dn identified by the monitoring-required drone identification unit 332 to the operator terminal Tm, thereby displaying the monitoring information of the drone Dn on the drone monitoring screen of the operator terminal Tm. For example, the simple information display area LA1 shown in FIG. 4 is transitioned to a selected state, and the monitoring information of the drone D1 is displayed in the drone basic information display area A1, the drone detailed information display area A2, the aircraft camera display area A3, and the base camera display area A4. As a result, when the monitoring-required timing arrives, the monitoring information of the identified drone Dn is automatically notified to the operator OPm via the operator terminal Tm. This prevents human error by the operator OPm when the operator OPm monitors multiple drones Dn during the same period (i.e., the period during which the monitoring task is performed). In addition, when the monitoring information display control unit 333 displays the takeoff decision button B on the drone monitoring screen, it may also display monitoring information including information about the drone Dn being monitored (e.g., the drone name) on the action request notification screen, along with information prompting the operator OPm to make a takeoff decision.

[0050] Furthermore, the monitoring information display control unit 333 may highlight a portion of the monitoring information of the drone Dn that requires particular attention from the operator OPm, based on the drone status of the drone Dn identified by the monitoring-required drone identification unit 332. For example, the monitoring information display control unit 333 may acquire highlighting information associated with the drone status identified by the status identification unit 331 from the monitoring priority master table and transmit display control data including the acquired highlighting information to the operator terminal Tm. This makes it possible to more accurately prevent human error by the operator OPm. For example, as shown in FIGS. 4 and 5, the monitoring information display control unit 333 may highlight an image including at least one of an aircraft image (e.g., aircraft camera image) and a base image (e.g., base camera image) according to the drone status of the drone Dn (i.e., the current drone status).

[0051] When a drone Dn is identified by the monitoring-requiring drone identification unit 332, the monitoring information display determination unit 334 determines whether the monitoring information of the identified drone Dn is already being displayed on the drone monitoring screen of the operator terminal Tm. If the monitoring information display determination unit 334 determines that the monitoring information of the drone Dn is not being displayed on the drone monitoring screen, the monitoring information display control unit 333 may transmit display control data for displaying the monitoring information of the identified drone Dn to the operator terminal Tm, thereby displaying the monitoring information of the drone Dn on the drone monitoring screen. This eliminates the need to transmit display control data for displaying the monitoring information of the drone Dn when it is determined that the monitoring information of the identified drone Dn is being displayed on the drone monitoring screen, thereby reducing the system load.

[0052] When the monitoring-required drone identification unit 332 identifies multiple drones Dn (i.e., when the monitoring-required timings overlap), the priority acquisition unit 335 acquires the monitoring priorities of each of the identified drones Dn based on the drone status (i.e., the current drone status) of each of the identified drones Dn. For example, the priority acquisition unit 335 acquires the monitoring priorities associated with the drone status (i.e., the current drone status) of each of the identified drones Dn in the monitoring priority master table. The monitoring-required drone identification unit 332 then compares the monitoring priorities acquired by the priority acquisition unit 335 and identifies the drone Dn with the highest monitoring priority from among the identified drones Dn. Monitoring information for the drone Dn finally identified in this way (the drone Dn with the highest monitoring priority) is displayed on the drone monitoring screen. This makes it possible to more accurately prevent human error by the operator OPm even when the monitoring-required timings of multiple drones Dn overlap (overlap).

[0053] Furthermore, the monitoring information display determination unit 334 may determine whether, among the multiple drones Dn managed by the operator OPm, monitoring information for another drone Dn (e.g., drone D1) that is currently in a monitoring-required timing (i.e., the monitoring-required timing has not yet ended) other than the one drone Dn (e.g., drone D2) identified by the monitoring-required drone identification unit 332 is already being displayed on the drone monitoring screen. Here, the other drone Dn that is currently in a monitoring-required timing is a drone Dn that was identified by the monitoring-required drone identification unit 332 before the one drone Dn. If it is determined that monitoring information for another drone Dn that is currently in a monitoring-required timing other than the one drone Dn is being displayed on the drone monitoring screen, the priority acquisition unit 335 acquires a monitoring priority (hereinafter referred to as a “first monitoring priority”) for the one drone Dn based on the drone status (i.e., the current drone status) of the one drone Dn, and acquires a monitoring priority (hereinafter referred to as a “second monitoring priority”) for the other drone Dn based on the drone status (i.e., the current drone status) of the other drone Dn.

[0054] Then, the priority determination unit 336 compares the first monitoring priority with the second monitoring priority acquired by the priority acquisition unit 335, and determines whether the first monitoring priority is higher than the second monitoring priority. When the priority determination unit 336 determines that the first monitoring priority is higher than the second monitoring priority, the monitoring information display control unit 333 transmits display control data to the operator terminal Tm for displaying the monitoring information of the identified drone Dn (i.e., switching to the display of the monitoring information), thereby switching from the display (screen) of the monitoring information of other drones Dn to the display (screen) of the monitoring information of the identified drone Dn. This makes it possible to more accurately prevent human error by the operator OPm even when the timings for monitoring multiple drones Dn overlap.

[0055] FIG. 9 is a conceptual diagram showing, in chronological order, the monitoring times T11 and T12 for drone D1 and the monitoring times T21, T22, and T23 for drone D2. The example in FIG. 9 indicates that the arrival of the monitoring time T23 for drone D2 overlaps with the monitoring time T12 for drone D1, for which monitoring information is being displayed. In this case, the monitoring time T12 for drone D1, for which monitoring information is being displayed, corresponds to the drone status “after takeoff determination” and is associated with a monitoring priority of “2” (an example of a second monitoring priority), while the monitoring time T23 for drone D2 corresponds to the drone status “arrived at destination” and is associated with a monitoring priority of “1” (an example of a first monitoring priority). Therefore, it is determined that the first monitoring priority is higher than the second monitoring priority, and the display of the monitoring information for drone D1 is switched to the display of the monitoring information for drone D2. In addition, if the monitoring priority of drone D1 and the monitoring priority of drone D2 are the same, it is preferable to display the monitoring information of drone D1, which has the earliest timing to monitor (i.e., no display switching).

[0056] 10 is a conceptual diagram showing, in time series, the monitoring time points T11 and T12 for drone D1, the monitoring time points T21, T22, and T23 for drone D2, and the monitoring time point T31 for drone D3. In the example of FIG. 10, when the monitoring time point T12 for drone D1 overlaps with the monitoring time point T23 for drone D2 for which monitoring information is displayed, the arrival of the monitoring time point T31 for drone D3 causes the monitoring time point T31 to overlap with the monitoring time point T12 for drone D1 and the monitoring time point T23 for drone D2. In this case, the monitoring time point T23 for drone D2 for which monitoring information is displayed corresponds to the drone status "arrived at destination" and is assigned a monitoring priority of "1" (an example of a second monitoring priority), while the monitoring time point T31 for drone D3 corresponds to the drone status "arrived at base" and is assigned a monitoring priority of "3" (an example of a first monitoring priority). Therefore, it is determined that the first monitoring priority is lower than the second monitoring priority, and the display of the monitoring information for drone D2 remains unchanged (i.e., the display is not switched).

[0057] The monitoring timing end determination unit 337 determines whether the monitoring timing for one drone Dn (e.g., drone D1) whose monitoring information is displayed on the drone monitoring screen has ended. Ending the monitoring timing means that the current time has reached the end of the monitoring timing. When the monitoring timing end determination unit 337 determines that the monitoring timing for one drone Dn has ended, the monitoring drone determination unit 338 determines whether there are any other drones Dn other than the one drone Dn that are currently in the monitoring timing among the multiple drones Dn managed by the operator OPm. When the monitoring drone determination unit 338 determines that there are other drones Dn currently in the monitoring timing, the monitoring information display control unit 333 switches the display of monitoring information for the one drone Dn whose monitoring timing has ended to the display of monitoring information for the other drones Dn currently in the monitoring timing. This allows the monitoring information for the other drones Dn that are still currently in the monitoring timing to be quickly displayed in response to the end of the monitoring timing for the one drone Dn.

[0058] Furthermore, if the monitoring-required drone determination unit 338 determines that there are multiple other drones Dn that are currently in the monitoring-required timing, the priority acquisition unit 335 acquires the monitoring priorities of each of the multiple other drones Dn based on their respective drone statuses (i.e., their current drone statuses). The monitoring-required drone identification unit 332 compares the monitoring priorities acquired by the priority acquisition unit 335 and identifies the other drone Dn with the highest monitoring priority from among the multiple other drones Dn. The monitoring information display control unit 333 then switches the display of monitoring information for the one drone Dn whose monitoring-required timing has ended to the display of monitoring information for the other drone Dn identified by the monitoring-required drone identification unit 332 (i.e., the other drone Dn with the highest monitoring priority). This allows the monitoring information of the drone Dn that should be most monitored among the multiple other drones Dn that are still currently in the monitoring-required timing to be quickly displayed in response to the end of the monitoring-required timing for the one drone Dn.

[0059] FIG. 11 is a conceptual diagram showing, in chronological order, the monitoring times T21, T22, and T23 for drone D2, the monitoring time T31 for drone D3, and the monitoring time T41 for drone D4. In the example of FIG. 11, after the monitoring time T23 for drone D2, for which monitoring information is displayed, ends, the monitoring time T31 for drone D3 overlaps with the monitoring time T41 for drone D4. In this case, the monitoring time T31 for drone D3 corresponds to the drone status "arrived at base" and is assigned a monitoring priority of "3," while the monitoring time T41 for drone D4 corresponds to the drone status "after takeoff determination" and is assigned a monitoring priority of "2." Therefore, drone D4, which has the higher monitoring priority, is identified, and the display of the monitoring information for drone D2 is switched to the display of the monitoring information for drone D4. In addition, if the monitoring priority of drone D3 and the monitoring priority of drone D4 are the same, it is recommended to display the monitoring information of drone D3, which has the earliest timing to be monitored.

[0060] The anomaly detection unit 339 detects an anomaly in the drone Dn by, for example, analyzing an image including at least one of an aircraft image and a base image. This allows for rapid detection of an anomaly in the drone Dn through analysis of the aircraft image or the base image. For example, the anomaly detection unit 339 detects an anomaly in the drone Dn when the brightness of the aircraft image or the base image is below a threshold (e.g., black (minimum brightness)). Alternatively, the anomaly detection unit 339 analyzes the base image (e.g., base camera video) to confirm the presence or absence of the aircraft, and detects an anomaly in the drone Dn when the aircraft cannot be detected from the base image (i.e., when the aircraft is not captured in the base image). The presence or absence of the aircraft may be confirmed based on a pre-registered drone exterior image (or characteristic points). The drone status of the drone Dn for which an anomaly is detected may be changed to "anomaly detected."

[0061] When the monitoring information for a drone Dn identified by the monitoring drone identification unit 332 is displayed on the drone monitoring screen, if the abnormality detection unit 339 detects an abnormality in a drone Dn other than the identified drone Dn among the multiple drones Dn managed by the operator OPm, the monitoring information display control unit 333 switches the display of the monitoring information for the drone Dn to the display of the monitoring information for the other drone Dn in which the abnormality was detected. This allows the monitoring information for the drone Dn in which the abnormality was detected to be displayed quickly and with priority over the other drones in response to the occurrence of an abnormality. Furthermore, even if the number of aircraft monitored by a single operator OPm becomes enormous in the future, the operator OPm will be able to provide strong support.

[0062] However, since constantly detecting abnormalities for all drones Dn would increase the system load, the abnormality detection unit 339 should be configured to detect abnormalities only for drones Dn for which the monitoring timing has arrived and is currently in the monitoring timing, and for which monitoring information is not displayed on the drone monitoring screen (for example, drone D1 shown in FIG. 9, or drones D1 and D3 shown in FIG. 10). In other words, abnormality detection should be activated only when a drone Dn that requires attention but cannot display monitoring information occurs. This can reduce the system load. Note that abnormalities in drones Dn may be detected based on automatic inspection result information received by the communication unit 31.

[0063] As described above, when the display of the monitoring information of drone Dn is switched, the monitoring information display control unit 333 may notify the operator OPm that the display (screen) of the monitoring information has been switched. This allows the operator OPm to quickly understand that the display of the monitoring information of drone Dn has been switched. For example, the monitoring information display control unit 333 sends a message indicating that the display of the monitoring information has been switched (for example, "Switching the screen to an aircraft requiring attention (drone D2)") to the operator terminal Tm, thereby causing the message to pop up on the drone monitoring screen.

[0064] Furthermore, when the display of the monitoring information for the drone Dn is switched, the monitoring information display control unit 333 may, together with the display switch, display (e.g., wipe) a portion of the monitoring information that was displayed on the drone monitoring screen before the display switch in a position on the drone monitoring screen that does not interfere with the display of the monitoring information after the display switch. This allows the portion of the monitoring information for the drone Dn that was not identified as a display target for the monitoring information to be continuously displayed to the operator OPm. For example, the monitoring information display control unit 333 transmits display control data for displaying the portion of the monitoring information that was displayed on the drone monitoring screen before the display switch to the operator terminal Tm, thereby displaying the portion of the monitoring information on the drone monitoring screen. Here, the portion of the monitoring information may be at least one of an aircraft image (e.g., aircraft camera image) and a base image (e.g., base camera image). This allows the operator OPm to view an image related to the drone Dn that was not identified as a display target for the monitoring information.

[0065] FIG. 12 is a diagram showing an example of a drone monitoring screen display in which the onboard camera video of drone D1 and the base camera video of drone D3 are displayed in a position that does not obstruct the display of the monitoring information of drone D2. In the example of FIG. 12, the monitoring information display control unit 333 selects one of the onboard camera video and the base camera video as part of the monitoring information based on the drone status (i.e., the current drone status) of each of drones D1 and D3, whose monitoring information was displayed on the drone monitoring screen before the display was switched, and displays the selected image in the wipe windows W1 and W2. This allows the operator Dn to view a more appropriate image based on the current drone status of drone Dn even after the display was switched. Note that if the drone status indicates that the drone has taken off or landed, a base image (e.g., the base camera image) may be selected. On the other hand, if the drone status indicates that the drone has arrived at its destination, a drone image (e.g., the onboard camera image) may be selected.

[0066] [ 2. Operation of the remote monitoring system S ] Next, the operation of the remote monitoring system S will be described with reference to Figs. 13 and 14. Fig. 13 is a flowchart showing an example of a monitoring information display process executed by the control unit 33 of the management server MS. Fig. 14 is a flowchart showing an example of an abnormality detection process in step S13 shown in Fig. 13. In the operation example described below, it is assumed that an operator OP1 of the operator terminal T1 is logged in, that multiple drones Dn managed by the logged-in operator OP1 are identified, and that a drone monitoring screen including a drone list showing the identified multiple drones Dn is displayed on the operator terminal T1. When the operator OP1 logs in in this way, the management server MS continuously recognizes (monitors) the display status of the drone monitoring screen on the operator terminal T1 (including the display content of the monitoring information).

[0067] The process shown in Fig. 13 starts, for example, when a drone monitoring screen is displayed on the operator terminal T1. When the process in Fig. 13 starts, the control unit 33 identifies the drone status of each of the multiple drones Dn managed by the operator OP1 by the status identification unit 331, for example, from the drone management database 322 (step S1).

[0068] Next, the control unit 33 determines whether or not there is a drone Dn among the multiple drones Dn managed by the operator OP1 for which the monitoring timing has arrived, based on the respective drone statuses identified in step S1 (step S2). If it is determined that there is a drone Dn for which the monitoring timing has arrived (step S2: YES), the process proceeds to step S3. On the other hand, if it is determined that there is no drone Dn for which the monitoring timing has arrived (step S2: NO), the process proceeds to step S9.

[0069] In step S3, the control unit 33 uses the monitoring-required drone identification unit 332 to identify one drone Dn for which the monitoring timing has arrived. If multiple drones Dn for which the monitoring timing has arrived have been identified, as described above, the priority acquisition unit 335 acquires (for example, stores in RAM) the monitoring priority of each of the identified drones Dn based on the drone status of each of the identified drones Dn. The monitoring-required drone identification unit 332 then compares the monitoring priorities acquired by the priority acquisition unit 335 and identifies one drone Dn with the highest monitoring priority from among the identified drones Dn.

[0070] Next, the control unit 33 determines whether the monitoring information of the drone Dn identified in step S3 is already displayed on the drone monitoring screen of the operator terminal T1 by the monitoring information display determination unit 334, for example, from the display status of the drone monitoring screen (step S4). If it is determined that the monitoring information of the identified drone Dn is not displayed on the drone monitoring screen (step S4: NO), the process proceeds to step S5. On the other hand, if it is determined that the monitoring information of the identified drone Dn is already displayed on the drone monitoring screen (step S4: YES), the process proceeds to step S9.

[0071] In step S5, the control unit 33 determines whether monitoring information of other drones Dn that are in monitoring timing other than the drone Dn identified in step S3 is already displayed on the drone monitoring screen of the operator terminal T1, for example, from the display status of the drone monitoring screen, using the monitoring information display determination unit 334. If it is determined that monitoring information of other drones Dn is not displayed on the drone monitoring screen (step S5: NO), the process proceeds to step S6. On the other hand, if it is determined that monitoring information of other drones Dn is already displayed on the drone monitoring screen (step S5: YES), the process proceeds to step S7.

[0072] In step S6, the control unit 33 causes the monitoring information display control unit 333 to display the monitoring information of the drone Dn identified in step S3 on the drone monitoring screen of the operator terminal T1. For example, the monitoring information display control unit 333 transmits display control data for displaying the monitoring information of the drone Dn identified in step S3 to the operator terminal T1 via the communication unit 31, and proceeds to step S9. At this time, the monitoring information display control unit 333 may highlight a portion of the monitoring information of the drone Dn that requires particular attention from the operator OP1, as described above, based on the drone status of the drone Dn identified in step S3.

[0073] In step S7, the control unit 33 acquires the monitoring priority of the drone Dn identified in step S3 based on the drone status of the drone Dn, acquires the monitoring priorities of other drones Dn that are in monitoring-required timings, and compares the acquired monitoring priorities with each other. Note that the monitoring priority of the drone Dn may be acquired before step S5 (for example, in step S1). Next, the control unit 33 determines, using the priority determination unit 336, whether the monitoring priority of the drone Dn identified in step S3 is higher than the monitoring priorities of other drones Dn that are in monitoring-required timings (step S8).

[0074] Then, if it is determined that the monitoring priority of the identified drone Dn is not high (low) (step S8: NO), the display of the monitoring information is not switched, and the process proceeds to step S9. On the other hand, if it is determined that the monitoring priority of the identified drone Dn is high (step S8: YES), the process proceeds to step S6, and the display of the monitoring information is switched by the process of step S6. At this time, the monitoring information display control unit 333 may, as described above, send a message indicating that the display of the monitoring information has been switched to the operator terminal T1, thereby causing the message to pop up on the drone monitoring screen.

[0075] In step S9, the control unit 33 determines whether the monitoring timing for the drone Dn, whose monitoring information is displayed on the drone monitoring screen, has ended using the monitoring timing end determination unit 337. If it is determined that the monitoring timing for the drone Dn has ended (step S9: YES), the process proceeds to step S10. On the other hand, if it is determined that the monitoring timing for the drone Dn has not ended (step S9: NO), the process proceeds to step S13. Although not shown, if no monitoring information is displayed on the drone monitoring screen (for example, immediately after the operator OP1 logs in) or if the drone Dn corresponding to the monitoring information displayed on the drone monitoring screen is not in the monitoring timing, the process may proceed to step S14.

[0076] In step S10, the control unit 33 determines whether or not there are other drones Dn in the monitoring timing other than the drones Dn for which the monitoring timing has ended, among the drones Dn managed by the operator OP1, using the monitoring drone determination unit 338. If it is determined that there are other drones Dn in the monitoring timing (step S10: YES), the process proceeds to step S11. On the other hand, if it is determined that there are no other drones Dn in the monitoring timing (step S10: NO), the display of monitoring information for the drones Dn for which the monitoring timing has ended continues, and the process proceeds to step S14.

[0077] In step S11, the control unit 33 identifies one drone Dn that is in a monitoring-required timing. If multiple drones Dn that are in a monitoring-required timing are identified, as described above, the priority acquisition unit 335 acquires the monitoring priority of each of the identified drones Dn based on the drone status of each of the identified drones Dn. The monitoring-required drone identification unit 332 then compares the monitoring priorities acquired by the priority acquisition unit 335 and identifies one drone Dn with the highest monitoring priority from among the identified drones Dn.

[0078] Next, similarly to step S6, the control unit 33 causes the monitoring information display control unit 333 to display the monitoring information of the drone Dn identified in step S11 on the drone monitoring screen of the operator terminal T1 (that is, the display of the monitoring information is switched). At this time, the monitoring information display control unit 333 may highlight a portion of the monitoring information of the drone Dn that requires particular attention from the operator OP1, as described above, based on the drone status of the drone Dn identified in step S11. Furthermore, as described above, the monitoring information display control unit 333 may pop up a message on the drone monitoring screen indicating that the display of the monitoring information has been switched.

[0079] In step S13, the control unit 33 executes an abnormality detection process. In the abnormality detection process, as shown in Fig. 14, the control unit 33 determines whether or not there is a drone Dn among the drones Dn managed by the operator OP1 that is in a monitoring required timing and whose monitoring information is not displayed on the drone monitoring screen (step S131). If it is determined that there is no drone Dn that is in a monitoring required timing and whose monitoring information is not displayed on the drone monitoring screen (step S131: NO), the process returns to the process shown in Fig. 13. On the other hand, if it is determined that there is a drone Dn that is in a monitoring required timing and whose monitoring information is not displayed on the drone monitoring screen (step S131: YES), the process proceeds to step S132.

[0080] In step S132, the control unit 33 identifies a drone Dn that is in a monitoring timing and whose monitoring information is not displayed on the drone monitoring screen as a target for abnormality detection. Next, the control unit 33 determines whether an abnormality in the drone Dn has been detected by having the abnormality detection unit 339 execute processing such as analyzing an image including at least one of an aircraft image and a base image related to the drone Dn identified in step S132 (step S133). If it is determined that an abnormality in the drone Dn has not been detected (step S133: NO), the process returns to the processing shown in FIG. 13. On the other hand, if it is determined that an abnormality in the drone Dn has been detected (step S133: YES), the process proceeds to step S134.

[0081] In step S134, similar to step S6, the control unit 33 causes the monitoring information display control unit 333 to display the monitoring information of the drone Dn in which an abnormality was detected in step S133 on the drone monitoring screen of the operator terminal T1 (that is, the display of the monitoring information is switched). At this time, the monitoring information display control unit 333 may highlight a portion of the monitoring information of the drone Dn that requires particular attention from the operator OP1, as described above, based on the drone status of the drone Dn in which an abnormality was detected. Furthermore, the monitoring information display control unit 333 may pop up a message on the drone monitoring screen indicating that the display of the monitoring information has been switched because an abnormality has been detected.

[0082] 13, in step S14, the control unit 33 determines whether or not it is time to update the monitoring information. For example, a timer may be set so that the update trigger occurs at predetermined time intervals. If it is determined that it is time to update the monitoring information (step S14: YES), the process returns to step S1, and the same process as above is performed. On the other hand, if it is determined that it is not time to update the monitoring information (step S14: NO), the process proceeds to step S15.

[0083] In step S15, the control unit 33 determines whether or not to end the monitoring information display process. For example, when the operator OP1 logs out or when a screen transition request is received from the operator terminal T1 in response to an instruction from the operator OP1 to transition to another screen, it is determined that the monitoring information display process is to be ended (step S15: YES), and the process shown in Fig. 13 ends. On the other hand, when it is determined that the monitoring information display process is not to be ended (step S15: NO), the process returns to step S9.

[0084] As described above, according to the above embodiment, the management server MS is configured to identify the drone status of each of the multiple drones Dn that the operator OPm is responsible for, identify the drones Dn that require monitoring based on the respective drone statuses, and display control data for displaying monitoring information for the identified drones Dn on the drone monitoring screen of the operator terminal Tm, thereby preventing human error by the operator OPm when monitoring multiple drones Dn over the same period.

[0085] The above embodiment is merely an embodiment of the present invention, and the present invention is not limited to the above embodiment. Various configurations and other changes may be made to the above embodiment without departing from the spirit of the present invention, and such changes are still within the technical scope of the present invention. In the above embodiment, instead of the management server MS, the operator terminal Tm (control unit 25) may be configured to acquire the drone status of each of the multiple drones Dn managed by the operator OPm from the management server MS in accordance with a monitoring application, identify a drone Dn among the multiple drones Dn for which a monitoring timing has arrived based on the respective drone statuses, and notify the operator OPm via the operator terminal Tm of monitoring information that prompts monitoring of the identified drone Dn and includes information about the drone Dn. In this case, the storage unit 23 of the operator terminal Tm may store the above-mentioned monitoring priority master table. In this case, the operator terminal Tm may execute the monitoring information display process shown in FIG. 13 while appropriately acquiring necessary information from the management server MS.

[0086] <Additional Notes> [1] The information processing device according to the present disclosure is characterized by comprising: a first identification means for identifying the status of each of a plurality of unmanned aerial vehicles under the operator's care; a second identification means for identifying, based on the status of each of the plurality of unmanned aerial vehicles identified by the first identification means, an unmanned aerial vehicle that has reached a monitoring timing that requires monitoring; and a processing means for notifying, via a terminal used by the operator, monitoring information that prompts the operator to monitor the unmanned aerial vehicle identified by the second identification means and that includes information about the unmanned aerial vehicle. This makes it possible to prevent human error by the operator when monitoring a plurality of unmanned aerial vehicles over the same period.

[0087] [2] The information processing device described in [1] above further comprises a first determination means for determining whether the monitoring information of the unmanned aerial vehicle identified by the second identification means is displayed on the screen of the terminal, and when the first determination means determines that the monitoring information of the unmanned aerial vehicle is not displayed on the screen, the processing means displays the monitoring information of the unmanned aerial vehicle identified by the second identification means on the screen, thereby reducing the system load.

[0088] [3] The information processing device according to [1] or [2] above further comprises: an acquisition means for acquiring a monitoring priority for each of the plurality of unmanned aerial vehicles based on the status of each of the plurality of unmanned aerial vehicles when the second identification means identifies the plurality of unmanned aerial vehicles; and a third identification means for comparing the monitoring priorities acquired by the acquisition means and identifying the unmanned aerial vehicle with the highest monitoring priority from among the plurality of unmanned aerial vehicles identified by the second identification means, wherein the processing means displays the monitoring information for the unmanned aerial vehicle identified by the third identification means on the screen of the terminal. This makes it possible to more accurately prevent human error caused by unmanned aerial vehicles even when the monitoring timings of multiple unmanned aerial vehicles overlap.

[0089] [4] In the information processing device described in [3] above, the processing means displays a portion of the monitoring information for unmanned aerial vehicles that have not been identified by the third identification means among the plurality of unmanned aerial vehicles identified by the second identification means on the screen at a position that does not obstruct the display of the monitoring information for unmanned aerial vehicles identified by the third identification means. This makes it possible to continuously show to the operator a portion of the monitoring information for unmanned aerial vehicles that have not been identified as targets for display of monitoring information.

[0090] [5] In the information processing device described in any one of [1] to [4] above, there is provided a second determination means for determining whether the monitoring information of other unmanned aerial vehicles among the plurality of unmanned aerial vehicles that are in the monitoring timing other than the one unmanned aerial vehicle identified by the second identification means is displayed on the screen of the terminal; an acquisition means for acquiring a first monitoring priority of the one unmanned aerial vehicle based on the status of the one unmanned aerial vehicle, and acquiring a second monitoring priority of the other unmanned aerial vehicles based on the status of the other unmanned aerial vehicles; a third determination means for, when the second determination means determines that the monitoring information of the other unmanned aerial vehicles is displayed on the screen, comparing the first monitoring priority with the second monitoring priority and determining whether the first monitoring priority is higher than the second monitoring priority; and, when the third determination means determines that the first monitoring priority is higher than the second monitoring priority, the processing means switches from displaying the monitoring information of the other unmanned aerial vehicles to displaying the monitoring information of the one unmanned aerial vehicle. This makes it possible to more accurately prevent human error by operators even when the monitoring timings for multiple unmanned aerial vehicles overlap.

[0091] [6] In the information processing device described in [5] above, the processing means, together with the switching of the display, displays a part of the monitoring information of the other unmanned aerial vehicle on the screen at a position that does not obstruct the display of the monitoring information of the first unmanned aerial vehicle. This makes it possible to continuously show to the operator the part of the unmanned aerial vehicle that was not identified as a target for display of monitoring information.

[0092] [7] The information processing device according to any one of [1] to [6] above, further comprising: a fourth determination means for determining whether the monitoring timing for one unmanned aerial vehicle whose monitoring information is displayed on the screen of the terminal has ended; a fifth determination means for determining, when the fourth determination means determines that the monitoring timing for the one unmanned aerial vehicle has ended, whether there is another unmanned aerial vehicle among the plurality of unmanned aerial vehicles that is currently in the monitoring timing other than the one unmanned aerial vehicle; and, when the fifth determination means determines that there is another unmanned aerial vehicle that is currently in the monitoring timing, the processing means switches the display of the monitoring information for the one unmanned aerial vehicle to the display of the monitoring information for the other unmanned aerial vehicle. This allows the monitoring information for the other unmanned aerial vehicle that is still currently in the monitoring timing to be quickly displayed in response to the end of the monitoring timing for the one unmanned aerial vehicle.

[0093] [8] The information processing device described in [7] above further comprises: an acquisition means for acquiring a monitoring priority for each of the other unmanned aerial vehicles based on the status of each of the other unmanned aerial vehicles when the fifth determination means determines that there are multiple other unmanned aerial vehicles that are currently in the monitoring required timing; and a third identification means for comparing the monitoring priorities acquired by the acquisition means and identifying the other unmanned aerial vehicle with the highest monitoring priority from among the multiple other unmanned aerial vehicles, wherein the processing means switches the display of the monitoring information for the first unmanned aerial vehicle to the display of the monitoring information for the other unmanned aerial vehicle identified by the third identification means. This makes it possible to quickly display the monitoring information for the unmanned aerial vehicle that should be most monitored among the multiple other unmanned aerial vehicles that are still currently in the monitoring required timing in response to the end of the monitoring required timing for the first unmanned aerial vehicle.

[0094] [9] The information processing device according to any one of [1] to [8] above further comprises a detection means for detecting an abnormality in the unmanned aerial vehicle, and when the monitoring information of one unmanned aerial vehicle identified by the second identification means is displayed on the screen of the terminal, if an abnormality is detected in another unmanned aerial vehicle other than the one unmanned aerial vehicle among the plurality of unmanned aerial vehicles, the display of the monitoring information of the one unmanned aerial vehicle is switched to the display of the monitoring information of the other unmanned aerial vehicle. This allows the monitoring information of the unmanned aerial vehicle to be displayed quickly and with priority over the other vehicles in response to the occurrence of an abnormality in the unmanned aerial vehicle in which an abnormality has been detected.

[0095]

[10] In the information processing device described in any one of [5] to [9] above, the processing means is characterized in that, when the display of the monitoring information of the unmanned aerial vehicle is switched, the processing means notifies the operator that the display of the monitoring information has been switched. This allows the operator to quickly understand that the display of the monitoring information of the unmanned aerial vehicle has been switched.

[0096]

[11] In the information processing device described in any one of [1] to

[10] above, the monitoring information is characterized in that it includes at least one of aircraft information acquired by the unmanned aerial vehicle and base information acquired by equipment at a base from which the unmanned aerial vehicle can take off and land.

[0097]

[12] In the information processing device described in

[11] above, the aircraft information includes an aircraft image captured by a camera equipped in the unmanned aerial vehicle, and the base information includes a base image captured by a camera equipped in the device, allowing an operator to check the appearance of the unmanned aerial vehicle or its surroundings at a glance.

[0098]

[13] In the information processing device described in

[12] above, the processing means displays at least one of the aircraft image and the base image on the screen of the terminal based on the status of the unmanned aerial vehicle. This makes it possible to show the operator an image that is more appropriate depending on the status of the unmanned aerial vehicle even after the display is switched.

[0099]

[14] In the information processing device described in [9] above, the aircraft information includes an aircraft image captured by a camera equipped on the unmanned aerial vehicle, the base information includes a base image captured by a camera equipped on the device, and the detection means detects an abnormality in the unmanned aerial vehicle by analyzing an image including at least one of the aircraft image and the base image. This makes it possible to more quickly detect an abnormality in the unmanned aerial vehicle.

[0100]

[15] In the information processing device described in any one of [2] to

[14] above, the processing means is characterized in that it highlights the part of the monitoring information that requires the operator's attention based on the status of the unmanned aerial vehicle identified by the second identification means. This makes it possible to more accurately prevent human error by the operator.

[0101]

[16] In the information processing device described in [4] or [6] above, the monitoring information includes either an image of the unmanned aerial vehicle taken by a camera equipped on the unmanned aerial vehicle or an image of a base taken by a camera equipped on equipment at a base from which the unmanned aerial vehicle can take off and land, and part of the monitoring information is the image. This makes it possible to show to the operator images related to unmanned aerial vehicles that were not identified as targets for display in the monitoring information.

[0102]

[17] In the information processing device described in

[16] above, the processing means selects one of the aircraft image and the base image as part of the monitoring information based on the status of the unmanned aerial vehicle identified by the second identification means. This makes it possible to show the operator an image more appropriate for the status of the unmanned aerial vehicle even after display switching.

[0103]

[18] The information display method of the present disclosure is an information display method executed by one or more computers, and is characterized by including the steps of: identifying the status of each of a plurality of unmanned aerial vehicles under the operator's responsibility; identifying an unmanned aerial vehicle among the plurality of unmanned aerial vehicles that has reached a monitoring timing that requires monitoring based on the identified status; and notifying the operator via a terminal used by the operator of monitoring information that prompts monitoring of the identified unmanned aerial vehicle and includes information about the unmanned aerial vehicle.

[0104]

[19] The program disclosed herein is characterized in that it causes a computer included in a terminal used by an operator to execute the steps of: obtaining the status of each of multiple unmanned aerial vehicles under the operator's responsibility from a predetermined server; identifying an unmanned aerial vehicle among the multiple unmanned aerial vehicles that has reached a monitoring timing that requires monitoring based on the status of each of the unmanned aerial vehicles; and notifying the operator via the terminal of monitoring information that prompts monitoring of the identified unmanned aerial vehicle and includes information about the unmanned aerial vehicle. [Explanation of symbols]

[0105] 11 Power supply section 12 Drive unit 13 Positioning unit 14 Communications Department 15 Sensor section 16 Memory section 17 Control Unit 21 Operation / display section 22 Communications Department 23 Memory section 24 Control Unit 31 Communications Department 32 Storage section 33 Control Unit 331 Status Identification Section 332 Drone surveillance specific section 333 Monitoring information display control section 334 Monitoring information display judgment unit 335 Priority acquisition unit 336 Priority judgment part 337 Monitoring timing end determination unit 338 Surveillance Drone Determination Department 339 Anomaly Detection Unit Dn Drone Tm Operator Terminal MS Management Server EM base equipment S Remote Monitoring System

Claims

1. a first identification means for identifying the status of each of a plurality of unmanned aerial vehicles under the operator's control; a second identification means for identifying an unmanned aerial vehicle among the plurality of unmanned aerial vehicles that has reached a monitoring timing that requires monitoring, based on each status identified by the first identification means; an acquisition means for acquiring a monitoring priority of the unmanned aerial vehicle based on the status of the unmanned aerial vehicle identified by the second identification means; a processing means for displaying on a screen of a terminal used by the operator monitoring information that prompts monitoring of the unmanned aerial vehicle with the highest monitoring priority among the plurality of unmanned aerial vehicles identified by the second identification means, the monitoring information including information about the unmanned aerial vehicle, and for displaying a portion of the monitoring information of the unmanned aerial vehicle with a lower monitoring priority in a position on the screen that does not interfere with the display of the monitoring information of the unmanned aerial vehicle with the highest monitoring priority; Equipped with The monitoring information includes one of an image of the unmanned aerial vehicle captured by a camera equipped on the unmanned aerial vehicle and an image of a base captured by a camera equipped on equipment at a base from which the unmanned aerial vehicle can take off and land, The information processing device is characterized in that the processing means selects one of the aircraft image and the base image as part of the monitoring information based on the status of the unmanned aerial vehicle with a low monitoring priority.

2. The information processing device described in claim 1, characterized in that when the monitoring information of the unmanned aircraft with a low monitoring priority is already displayed on the screen of the terminal, the processing means switches the display of the monitoring information of the unmanned aircraft with the low monitoring priority to the display of the monitoring information of the unmanned aircraft with the highest monitoring priority, and, together with the display switching, displays a portion of the monitoring information of the unmanned aircraft with the low monitoring priority in a position on the screen that does not interfere with the display of the monitoring information of the unmanned aircraft with the highest monitoring priority.

3. a first determination means for determining whether the monitoring timing for one unmanned aerial vehicle whose monitoring information is displayed on the screen of the terminal has ended; a second determination means for determining, when the first determination means determines that the monitoring timing for the one unmanned aerial vehicle has ended, whether or not there is another unmanned aerial vehicle among the plurality of unmanned aerial vehicles that is in the monitoring timing other than the one unmanned aerial vehicle; The information processing device described in claim 1, characterized in that when the second determination means determines that there is another unmanned aerial vehicle that is currently in the monitoring timing, the processing means switches from displaying the monitoring information of the one unmanned aerial vehicle to displaying the monitoring information of the other unmanned aerial vehicle.

4. Further comprising a detection means for detecting an abnormality of the unmanned aerial vehicle, The information processing device described in claim 1, characterized in that when the monitoring information of one unmanned aerial vehicle identified by the second identification means is displayed on the screen of the terminal, if an abnormality is detected in another unmanned aerial vehicle among the multiple unmanned aerial vehicles other than the one unmanned aerial vehicle, the display of the monitoring information of the one unmanned aerial vehicle is switched to the display of the monitoring information of the other unmanned aerial vehicle.

5. An information processing device as described in any one of claims 1 to 4, characterized in that when the display of the monitoring information of the unmanned aerial vehicle is switched, the processing means notifies the operator that the display of the monitoring information has been switched.

6. The information processing device according to claim 4, wherein the detection means detects an abnormality in the unmanned aerial vehicle by analyzing an image including at least one of the aircraft image and the base image.

7. An information processing device as described in any one of claims 1 to 4, characterized in that the processing means highlights parts of the monitoring information that require the operator's attention based on the status of the unmanned aerial vehicle identified by the second identification means.

8. 1. A method for displaying information executed by one or more computers, comprising: determining the status of each of a plurality of unmanned aerial vehicles under the operator's control; A step of identifying an unmanned aerial vehicle among the plurality of unmanned aerial vehicles that has reached a monitoring timing that requires monitoring based on each of the identified statuses; obtaining a monitoring priority for the identified unmanned aerial vehicle based on the status of the identified unmanned aerial vehicle; a processing step of displaying, on a screen of a terminal used by the operator, monitoring information that prompts monitoring of the unmanned aerial vehicle with the highest monitoring priority among the identified plurality of unmanned aerial vehicles, the information including the unmanned aerial vehicle, and displaying a portion of the monitoring information of the unmanned aerial vehicle with a lower monitoring priority in a position on the screen that does not interfere with the display of the monitoring information of the unmanned aerial vehicle with the highest monitoring priority; Including, The monitoring information includes one of an image of the unmanned aerial vehicle captured by a camera equipped on the unmanned aerial vehicle and an image of a base captured by a camera equipped on equipment at a base from which the unmanned aerial vehicle can take off and land, An information display method characterized in that in the processing step, one of the aircraft image and the base image is selected as part of the monitoring information based on the status of the unmanned aerial vehicle with a low monitoring priority.

9. The computer included in the terminal used by the operator Obtaining the status of each of the plurality of unmanned aerial vehicles managed by the operator from a predetermined server; A step of identifying an unmanned aerial vehicle among the plurality of unmanned aerial vehicles that has reached a monitoring timing that requires monitoring based on the status of each of the unmanned aerial vehicles; obtaining a monitoring priority for the identified unmanned aerial vehicle based on the status of the identified unmanned aerial vehicle; a processing step of displaying, on a screen of a terminal used by the operator, monitoring information that prompts monitoring of the unmanned aerial vehicle with the highest monitoring priority among the identified plurality of unmanned aerial vehicles, the information including the unmanned aerial vehicle, and displaying a portion of the monitoring information of the unmanned aerial vehicle with a lower monitoring priority in a position on the screen that does not interfere with the display of the monitoring information of the unmanned aerial vehicle with the highest monitoring priority; A program for executing The monitoring information includes one of an image of the unmanned aerial vehicle captured by a camera equipped on the unmanned aerial vehicle and an image of a base captured by a camera equipped on equipment at a base from which the unmanned aerial vehicle can take off and land, A program characterized in that in the processing step, one of the aircraft image and the base image is selected as part of the monitoring information based on the status of the unmanned aerial vehicle with a low monitoring priority.

Images