Flying robot control system and flying robot control method

Abstract

To provide a flying robot control system and a flying robot control method with which it is possible to appropriately limit the manual operation possible time using a handling device on a flying robot that autonomously flies.SOLUTION: A flying robot control system comprises a flying robot and a handling device, and includes: a mode setting unit which sets, as a flight mode for the flying robot, either an autonomous flight mode in which the flying robot autonomously flies and a manual flight mode it flies according to a manual operation using the handling device; and an acquisition unit for acquiring the state of a flight area including the current position of the flying robot or the surrounding of the current position. The mode setting unit sets the manual operation possible time by the manual flight mode on the basis of the state of the flight area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a flying robot control system including a flying robot and an operating device, and to a flying robot control method.

Background Art

[0002] In recent years, flying robots that fly autonomously and perform tasks such as returning, tracking, detecting, moving, and patrolling have been developed for guarding facilities such as train stations, commercial facilities, and power plants. In such flying robots, it is desired that the flying robot can be operated by manual operation using an operating device when it is necessary to focus on checking a specific location, or when the flying robot loses sight of a tracking target.

[0003] For example, Patent Document 1 discloses a semi-autonomously flyable flying robot that can fly autonomously and can be controlled by a user's operation via a wireless controller device or the like.

Prior Art Literature

Patent Literature

[0004]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] If manual operation is freely permitted for an autonomously flying flying robot, there is a possibility that the flying robot cannot execute a preset task.

[0006] An object of the present invention is to provide a flying robot control system and a flying robot control method capable of appropriately limiting the allowable time for manual operation using an operating device for an autonomously flying flying robot.

Means for Solving the Problem

[0007] To solve these problems, the present invention provides a flying robot control system having a flying robot and an operating device, comprising: a mode setting unit that sets either an autonomous flight mode in which the flying robot flies autonomously, or a manual flight mode in which the flying robot flies according to manual operation using the operating device as the flight mode of the flying robot; and an acquisition unit that acquires the state of the current position of the flying robot or the state of the flight area including the vicinity of the current position, wherein the mode setting unit sets the time during which manual operation is possible in manual flight mode based on the state of the flight area.

[0008] In this flying robot control system, the acquisition unit preferably acquires the status of the flight area, such as the occurrence of an anomaly in the flight area, whether or not the flight area is set as a monitoring area, or the current security status in the flight area.

[0009] In this flying robot control system, it is preferable that the mode setting unit makes the manual operation time when the flight area is in a state requiring manual monitoring, where manual monitoring is highly necessary, longer than the manual operation time when the flight area is not in a state requiring manual monitoring.

[0010] In this flying robot control system, in autonomous flight mode, the flying robot flies autonomously to perform a predetermined task, and the mode setting unit preferably sets the manual operation time based on the remaining time required to complete the task being performed by the flying robot when the state of the flight area is not in a state requiring manual monitoring, where manual monitoring is highly necessary, and sets the manual operation time regardless of the remaining time when the state of the flight area is in a state requiring manual monitoring.

[0011] In this flying robot control system, the mode setting unit preferably sets the manual operation time to a predetermined standard time when the flight area is not in a state requiring manual monitoring, and sets the manual operation time to a time longer than the standard time when the flight area is in a state requiring manual monitoring.

[0012] In this flying robot control system, it is preferable for the mode setting unit to notify the operator of the control device of the remaining manual operation time when the system is set to manual flight mode.

[0013] In this flying robot control system, it is preferable for the mode setting unit to restrict some manual operations when the manual operation time falls below a threshold in manual flight mode.

[0014] To solve these problems, the present invention provides a method for controlling a flying robot, which includes a computer setting either an autonomous flight mode in which the flying robot flies autonomously, or a manual flight mode in which the flying robot flies according to manual operation using an operating device, and acquiring the state of the flight area in which the flying robot is flying, and in setting, setting conditions regarding the time during which manual operation is possible in manual flight mode based on the state of the flight area. [Effects of the Invention]

[0015] The flying robot control system and flying robot control method according to the present invention make it possible to appropriately limit the time during which manual operation using an operating device is possible for an autonomously flying flying robot. [Brief explanation of the drawing]

[0016] [Figure 1] This is a schematic diagram illustrating monitoring area A1. [Figure 2] This diagram shows the overall system configuration of the flying robot control system 1. [Figure 3] This figure shows an example of the data structure of a management table. [Figure 4] A flowchart shows an example of the operation of the first setup process. [Figure 5] This flowchart shows an example of how the second configuration process works. [Modes for carrying out the invention]

[0017] Hereinafter, a flight robot control system according to an embodiment will be described with reference to the drawings.

[0018] FIG. 1 is a schematic diagram for explaining a monitoring area A1 of the flight robot control system 1 according to the embodiment. As shown in FIG. 1, the flight robot control system 1 includes a flight robot 10, an operating device 20, a management device 30, a server S, one or more security devices T, and the like. The flight robot control system 1 is a monitoring system that monitors and secures a monitoring area A1 set in a facility such as a train station, a commercial facility, or a power plant. The monitoring area A1 includes one or a plurality of monitored properties B such as platforms, stores, and power generation facilities.

[0019] The server S is arranged on a monitoring console or the like of a disaster prevention center C1 installed inside the monitoring area A1 or outside the monitoring area A1. The server S has a storage unit, manages various settings registered from the operating device 20 and / or the management device 30 to set the settings in the flight robot 10 and the security device T, and collects and manages monitoring results obtained by the flight robot 10 and the security device T.

[0020] The server S receives, from the flight robot 10 or the security device T, an abnormality notification signal indicating that an abnormality has occurred or has been resolved in the monitoring area A1 and / or an occurrence position of the abnormality, and stores the occurrence state of the abnormality at each position in the monitoring area A1 in the storage unit. The server S transmits, in accordance with a request from the flight robot 10, the operating device 20 and / or the management device 30, or spontaneously, abnormality occurrence state information indicating an occurrence state of an abnormality in each region within the monitoring area A1 to the flight robot 10, the operating device 20 and / or the management device 30.

[0021] Further, the server S receives, from the operating device 20 or the management device 30, a monitoring-required area setting signal for setting a monitoring-required area within the monitoring area A1. As the monitoring-required area, an area where an abnormality is likely to occur, an area to be monitored with emphasis, or the like is set by a controller who operates the operating device 20 or the management device 30. In the monitoring-required area setting signal, an area within the monitoring area A1 to be set as the monitoring-required area is specified. The server S stores, in a storage unit, the area set as the monitoring-required area within the monitoring area A1. The server S transmits, in accordance with a request from the flying robot 10, the operating device 20 and / or the management device 30, or spontaneously, monitoring-required area information indicating the area set as the monitoring-required area within the monitoring area A1 to the flying robot 10, the operating device 20 and / or the management device 30.

[0022] Further, the server S accepts, from an operation unit provided in the server S, setting of a security state in each area within the monitoring area A1 by an administrator of the server S. The setting of the security state includes the security state to be set and the area within the monitoring area A1 for which the security state is set. The server S stores the security state of each area within the monitoring area A1 in a storage unit, and transmits a security state setting signal for setting the security state to a corresponding security device T to set the security state. Further, when each security device T accepts, from an operation unit provided in the each security device T, setting of a security state by a user of the each security device T, the each security device T transmits, to the server S, security state information indicating the security state of a corresponding area within the monitoring area A1. The server S receives the security state information from each security device T, and stores the security state of the corresponding area within the monitoring area A1 in a storage unit. The server S transmits, in accordance with a request from the flying robot 10, the operating device 20 and / or the management device 30, or spontaneously, security state information indicating the security state of each area within the monitoring area A1 to the flying robot 10, the operating device 20 and / or the management device 30. Details of the security state will be described later.

[0023] Further, the server S stores a management table for managing the schedule of the flying robot 10. Details of the management table will be described later.

[0024] Security device T is installed on the monitored property B and has predetermined sensors and / or fixed cameras. The sensors include sensors for detecting human intrusion, such as magnetic sensors that detect the opening and closing of doors in each property or infrared sensors that detect the presence or absence of people, or sensors for detecting fire, such as heat sensors or smoke sensors. Based on the detection results of human intrusion or fire occurrence by the sensors or fixed cameras, security device T transmits an abnormality notification signal to server S.

[0025] Security device T has multiple security states. These security states include the alerted state and the de-alert state, among others. The alerted state is a security state in which the presence or absence of abnormalities is detected in monitoring area A1 (the monitored area), and an alert mode is set to send an alert when an abnormality is detected. The alerted state may include the unmanned alerted state and the manned alerted state. The unmanned alerted state is a state in which the alert mode is set to an unmanned alert mode that detects the presence or absence of abnormalities for all sensors. The manned alerted state is a state in which the alert mode is set to a manned alert mode that detects the presence or absence of abnormalities for specific sensors installed on windows or doors that can be used to enter the monitored area. The de-alert state is a state in which no alert mode is set and no abnormalities are detected. When security device T receives a security status setting signal from server S, it sets the specified security status. In addition, security device T accepts security status settings from users of security device T via an operation unit provided on security device T, sets the specified security status, and transmits the security status information to server S.

[0026] The flying robot 10 is a device that flies within a flyable area A2 set up within the monitoring area A1. The flyable area A2 is set up to include, for example, the monitored object B and the take-off and landing point (hangar, roboport) D. A patrol route A3 is set up in the flyable area A2 for the flying robot 10 to patrol. In the example shown in Figure 1, the patrol route A3 is set to start from the take-off and landing point D, pass over the monitored object B, and return to the take-off and landing point D. The start and end positions of the patrol route do not necessarily have to be the same. Also, the patrol route A3 may include multiple patrol routes, such as a first patrol route from the take-off and landing point D to a predetermined point, and a second patrol route from the predetermined point to the take-off and landing point D. If the flying robot 10 detects an anomaly such as the intrusion of a suspicious person by sensors installed on the monitored object B, it can autonomously fly toward the monitored object B to investigate the cause of the anomaly. The control device 20 is located on a monitoring console or similar in the disaster prevention center C1, which is installed within or around monitoring area A1. The control device 20 registers various settings with the flying robot 10 via the server S and controls the flying robot 10 according to the operations of the controller (operator) at the disaster prevention center C1. The control device 30 is installed on a monitoring desk or similar at security center C2, which is operated by the security company. The control device 30 registers various settings with the flying robot 10 via server S and controls the flying robot 10 according to the operations of the controller (operator) at security center C2.

[0027] Figure 2 shows the overall system configuration of the flying robot control system 1. As shown in Figure 2, the flying robot 10, the control device 20, the management device 30, the server S, and the security device T are interconnected via a first communication network N1, such as an intranet or the internet. The flying robot 10 and the control device 20 are connected to the first communication network N1 via a wireless communication network, such as a wireless LAN or a mobile phone network. The management device 30 is connected to the first communication network N1 via a second communication network N2, such as an intranet or the internet.

[0028] The flying robot 10 is a small, unmanned aerial vehicle capable of autonomous flight, such as a quadrotor or single-rotor small unmanned helicopter. The flying robot 10 has two flight modes: an autonomous flight mode in which the flying robot 10 flies autonomously, and a manual flight mode in which it flies according to manual operation using the control device 20. The flying robot 10 is, for example, a drone, a multicopter, or a UAV (Unmanned Aerial Vehicle). The flying robot 10 includes a position / attitude sensor 11, an imaging unit 12, a motor 13, a first communication unit 14, a first storage unit 15, and a first control unit 16.

[0029] The position and attitude sensor 11 acquires the current position and attitude of the flying robot 10. The position and attitude sensor 11 includes, for example, a receiver that receives radio waves (navigation signals) transmitted from navigation satellites (artificial satellites) such as GNSS (Global Navigation Satellite System), an accelerometer that measures acceleration, an electronic compass that measures direction, and a gyro sensor that measures angular velocity. The receiver receives navigation signals transmitted from multiple navigation satellites and outputs them to the first control unit 16. The accelerometer, electronic compass, and gyro sensor output measurement signals indicating the measured acceleration, direction, and angular velocity to the first control unit 16. The position and attitude sensor 11 may also acquire the current position of the flying robot 10 using other known sensors such as a laser scanner and a barometric pressure sensor. Furthermore, the position and attitude sensor 11 may also acquire the attitude of the flying robot 10 using other known sensors.

[0030] The imaging unit 12 includes a visible light camera. The visible light camera includes, for example, a photoelectric conversion element sensitive to visible light, such as a CCD element or a C-MOS element, an imaging optical system that forms an image on the photoelectric conversion element, and an A / D converter, and generates and outputs an image based on visible light. The imaging unit 12 sequentially generates image images at a predetermined frame period and outputs them to the first control unit 16. The imaging unit 12 may also include a thermal imaging camera for acquiring thermal images. The thermal imaging camera includes, for example, two-dimensionally arranged sensors that detect the radiant energy of two different wavelengths of electromagnetic radiation from an object, and an A / D converter that amplifies the electrical signal output from the sensors and performs analog-to-digital (A / D) conversion. The thermal imaging camera generates a thermal image based on a temperature value determined by the ratio of the two types of radiant energies and outputs it to the first control unit 16 at a predetermined frame period.

[0031] The motor 13 includes one or more (e.g., four) motors driven by a battery. A rotor (rotating blade, propeller) is connected to the rotation axis of each motor. The motor 13 receives a drive signal from the first control unit 16 and rotates according to the received drive signal to generate driving force and rotate each rotor. The flying robot 10 can generate acceleration in any direction by having one or more rotors rotate independently, and can adjust the movement and attitude of the aircraft.

[0032] The first communication unit 14 has, for example, an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals through a wireless communication line in accordance with a wireless communication protocol such as a wireless LAN, and connects to the first communication network N1 via an access point. Alternatively, the first communication unit 14 has, for example, a communication interface circuit compliant with the W-CDMA system or LTE system, and connects to the first communication network N1 via a communication network such as a base station and a mobile communication network. The first communication unit 14 outputs data received from the first communication network N1 to the first control unit 16 and transmits data input from the first control unit 16 to the first communication network N1.

[0033] The first storage unit 15 includes semiconductor memory such as ROM and RAM, a magnetic disk or optical disk drive such as a CD-ROM or DVD-ROM, and its recording medium. The first storage unit 15 stores computer programs and various data for controlling the flying robot 10, and inputs and outputs this information to and from the first control unit 16. The computer program may be installed in the first storage unit 15 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. The first storage unit 15 also stores position / attitude information 151 and route information 152 as data. The position / attitude information 151 indicates the current position and current attitude of the flying robot 100 detected based on the signals acquired by the position / attitude sensor 11. The route information 152 is information indicating the flight path that the flying robot 10 is scheduled to take, and is shown as a sequence of coordinates along the flight path. Confirmation points indicating the shooting positions for determining whether or not there is an abnormality may be set along the flight path. The route information 152 is set from the operating device 20 and / or the management device 30 via the server S. Alternatively, the first control unit 16 may search (calculate) a movement path from the current position to the target position based on the current position and current attitude stored in the position / attitude information 151, and the target position set by the operating device 20 and / or the management device 30, and store it in the first storage unit 15 as route information 152. Furthermore, the calculations for searching for the route information 152 may be performed not only by the first control unit 16 of the flying robot 10, but also by the second control unit 26 of the operating device 20, or the third control unit 36 ​​of the management device 30.

[0034] The first control unit 16 includes a processor such as a CPU or MPU, memory such as ROM or RAM, and peripheral circuits, and performs various signal processing for the flying robot 10. The first control unit 16 includes a flight control unit 161, etc., which is implemented as a functional module of a program that runs on the processor. A DSP, LSI, ASIC, FPGA, etc. may be used as the first control unit 16.

[0035] The flight control unit 161 calculates the current position and attitude of the flying robot 10 in a three-dimensional movement area (e.g., flight space) from the output of the position and attitude sensor 11 and stores it in the first storage unit 15 as position and attitude information 151. The flight control unit 161 obtains the latitude, longitude, and altitude from the navigation signal output from the position and attitude sensor 11 and converts them to a position in the coordinate system of the movement area using a pre-stored conversion rule to determine the current position. The flight control unit 161 also obtains the current attitude in the coordinate system of the movement area from the measurement signals of the acceleration sensor and gyro sensor output from the position and attitude sensor 11. The flight control unit 161 may also obtain the bearing in the coordinate system of the movement area from the measurement signal of the electronic compass output from the position and attitude sensor 11 and further calculate the current attitude using measurement signals from other sensors. Each time the flight control unit 161 calculates the current position and / or current attitude, it transmits the calculated current position and / or current attitude to the server S via the first communication unit 14, and then transmits it to the operating device 20 and / or management device 30 via the server S. Also, each time the imaging unit 12 generates an image, the flight control unit 161 transmits the generated image to the server S via the first communication unit 14, and then transmits it to the operating device 20 and / or management device 30 via the server S.

[0036] The flight control unit 161 receives control signals from the operating device 20 or management device 30 via the first communication unit 14, and drives the motors 13 to perform flight actions such as ascending, descending, changing direction (turning), moving forward, and hovering (staying stationary) according to the received control signals. The control signals specify the flight mode in which the flight robot 10 will operate. Either autonomous flight mode or manual flight mode can be set as the flight mode. If autonomous flight mode is set, the flight robot 10 autonomously performs predetermined actions such as calculating its own position, calculating the target position, calculating the path, flying along the path, attitude control, and reporting, without requiring operation by the operator using the operating device 20. That is, in autonomous flight mode, the flight robot 10 performs one of several tasks. When autonomous flight mode is set as the flight mode, the control signals specify the task to be performed by the flight robot 10. Tasks are classifications of processes to be performed by the flight robot 10 according to their purpose. Tasks are set according to instructions from control personnel at disaster prevention center C1 using the control device 20, instructions from control personnel at security center C2 using the management device 30, notification signals from security device T, or a task schedule pre-configured on server S. Tasks may include return tasks, tracking tasks, detection and movement tasks, designated location movement tasks, and patrol tasks. Tasks other than security-related tasks, such as inspection tasks, rescue tasks, delivery tasks, surveying tasks, and pesticide spraying tasks, may also be set.

[0037] The return task is a task in which, upon completion or interruption of other tasks, or in the event of an emergency such as an aircraft malfunction, the aircraft searches (calculates) a return route to takeoff / landing point D in accordance with the return instruction from air traffic control, and automatically lands at takeoff / landing point D. The location information of takeoff / landing point D is stored in advance in the first storage unit 15. In addition, when a return task (and the patrol task described later) is set, the control signal may further include the location information of takeoff / landing point D. The tracking task is a task in which the flying robot 10 flies to track a designated target (person, car, etc.) according to the target designation operation by a controller using the operating device 20 and / or management device 30, which display the captured images generated by the flying robot 10. When a tracking task is set, the control signal further includes identification information indicating the designated target. The detection and movement task involves flying (or waiting to fly) to the vicinity of the sensor or fixed camera that detected the intrusion of a person when the sensor or fixed camera of the security device T installed in monitoring area A1 detects the intrusion, and confirming the location and cause of the anomaly. When the detection and movement task is set, the control signal also includes the position information of the sensor or fixed camera that detected the intrusion. The designated location movement task is a task to fly to a location designated by a controller at the disaster prevention center C1 using the operating device 20, or to a location designated by a controller at the security center C2 using the management device 30. For example, the operating device 20 or the management device 30 displays a map, and the controller specifies the location on the map.

[0038] The patrol task involves performing a patrol flight along a pre-set flight path (patrol path), taking photographs at checkpoints set along the flight path, and determining whether or not there are any abnormalities at the checkpoints. The patrol task includes subtasks: a path movement / photography task and a landing task. The path movement / photography task involves autonomously moving along the flight path indicated in the path information 152 pre-stored in the first memory unit 15, and, if checkpoints are set along the flight path, hovering at the checkpoints to generate (take) captured images and determine whether or not there are any abnormalities. The landing task involves descending in altitude until landing at the takeoff / landing point D, and moving so as to be stored in the hangar. When a patrol task is set, the flight control unit 161 first executes a route movement and image capture task. If a checkpoint is set on the flight path, the flight control unit 161 detects a change area in the image captured at the checkpoint. If the size of the detected change area is within the range corresponding to the size of a person, the flight control unit 161 determines that a person has entered the area, and if no change area corresponding to the size of a person is detected, it determines that no person has entered the area. The flight control unit 161 may also compare an image captured at a checkpoint in the past with an image captured at the checkpoint currently taken, and determine whether or not there is an anomaly based on the change area in the image captured. For example, the flight control unit 161 determines that there is an anomaly if a predetermined vehicle that was captured in a past image is not present in the newly captured image. When the flight control unit 161 reaches the takeoff / landing point D, it executes a landing task. Based on the detection result of the occurrence of an anomaly, the flight control unit 161 transmits an anomaly notification signal to the server S via the first communication unit 14.

[0039] The operating device 20 is, for example, a tablet PC or a notebook PC. The operating device 20 includes a second operating unit 21, a second display unit 22, a second audio output unit 23, a second communication unit 24, a second storage unit 25, and a second control unit 26, etc.

[0040] The second operation unit 21 has input devices such as buttons, touch panels, and keyboards, and interface circuits that acquire signals from input devices, and accepts operations from the user and outputs signals corresponding to the accepted operations to the second control unit 26. The second display unit 22 is a liquid crystal display or an organic EL display, etc., and displays various information such as images and text according to instructions from the second control unit 26. The second audio output unit 23 is, for example, a speaker, and outputs audio according to instructions from the second control unit 26.

[0041] The second communication unit 24 has, for example, an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals via a wireless communication line in accordance with a wireless communication protocol such as a wireless LAN, and is connected to the first communication network N1 via an access point. Alternatively, the second communication unit 24 has, for example, a communication interface circuit compliant with the W-CDMA or LTE system, and is connected to the first communication network N1 via a communication network such as a base station and a mobile communication network. Alternatively, the second communication unit 24 has, for example, a wired communication interface circuit compliant with TCP / IP, and is connected to the first communication network N1. The second communication unit 24 outputs data received from the first communication network N1 to the second control unit 26 and transmits data input from the second control unit 26 to the first communication network N1.

[0042] The second storage unit 25 includes semiconductor memory such as ROM and RAM, a magnetic disk or optical disk drive such as a CD-ROM or DVD-ROM, and its recording medium. The second storage unit 25 stores computer programs and various data for controlling the operating device 20, and inputs and outputs this information to and from the second control unit 26. The computer program may be installed in the second storage unit 25 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. The second memory unit 25 also stores task information 251 and other data. The task information 251 is information about the task currently being performed by the flying robot 10 and the next task to be performed.

[0043] The second control unit 26 includes a processor such as a CPU or MPU, memory such as ROM or RAM, and peripheral circuits, and performs various signal processing for the operating device 20. The second control unit 26 includes a display control unit 261, a mode setting unit 262, an acquisition unit 263, and a notification unit 264, etc., which are implemented as functional modules of a program that runs on the processor. A DSP, LSI, ASIC, FPGA, etc. may be used as the second control unit 26.

[0044] The display control unit 261 periodically receives captured images transmitted from the flying robot 10 and the current position and / or attitude of the flying robot 10 from the server S via the second communication unit 24. The display control unit 261 stores the received captured images, current position and / or attitude in the second storage unit 25 and displays them on the second display unit 22. The display control unit 261 also transmits route information 152 to the server S via the second communication unit 24 and registers it with the flying robot 10 via the server S. Furthermore, if the flight mode of the flying robot 10 is set to manual operation mode, the display control unit 261 accepts manual operation of the flying robot 10 from the operator using the second operation unit 21. The display control unit 261 controls the flight of the flying robot 10 by transmitting operation signals corresponding to the received manual operation to the server S via the second communication unit 24 and to the flying robot 10 via the server S. The mode setting unit 262 sets the flight mode of the flying robot 10 to either an autonomous flight mode in which the flying robot flies autonomously, or a manual flight mode in which the flying robot flies according to manual operation using the control device 20. Furthermore, if the operator instructs the flying robot 10 to switch to manual flight mode while it is operating in autonomous flight mode, the mode setting unit 262 sets the time during which manual operation is possible in manual flight mode based on the state of the flight area including the flying robot 10's current position or the area surrounding the current position. The area surrounding the current position includes, for example, a range within a predetermined distance (e.g., 1 km) from the current position. Note that the area surrounding the current position may include the entire flightable area A2 or the entire monitoring area A1. The acquisition unit 263 acquires the status of the flight area of ​​the flying robot 10. The notification unit 264 notifies the operator of the flying robot 10 of a warning regarding the limitations on manual flight mode. Details of the process will be described later.

[0045] Furthermore, the second control unit 26 receives the setting of areas requiring monitoring in monitoring area A1 by the controller from the second operation unit 21, and transmits the area requiring monitoring setting signal to the server S via the second communication unit 24.

[0046] The management device 30 is, for example, a personal computer or a server. The management device 30 includes a third operation unit 31, a third display unit 32, a third audio output unit 33, a third communication unit 34, a third storage unit 35, and a third control unit 36, etc.

[0047] The third operation unit 31 has an input device such as a touch panel or keyboard and an interface circuit that acquires signals from the input device, accepts operations from the user, and outputs a signal corresponding to the accepted operation to the third control unit 36. The third display unit 32 is a liquid crystal display or an organic EL display, etc., and displays various information such as images and text according to instructions from the third control unit 36. The third audio output unit 33 is, for example, a speaker, and outputs audio according to instructions from the third control unit 36.

[0048] The third communication unit 34 has a communication interface circuit compliant with, for example, TCP / IP, and is connected to the second communication network N2. Alternatively, the third communication unit 34 has, for example, an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals via a wireless communication line in accordance with a wireless communication protocol such as wireless LAN, and is connected to the second communication network N2 via an access point. The third communication unit 34 outputs data received from the second communication network N2 to the third control unit 36 ​​and transmits data input from the third control unit 36 ​​to the second communication network N2.

[0049] The third storage unit 35 includes semiconductor memory such as ROM and RAM, a magnetic disk or optical disk drive such as a CD-ROM or DVD-ROM, and its recording medium. The third storage unit 35 stores computer programs and various data for controlling the management device 30, and inputs and outputs this information to and from the third control unit 36. The computer program may be installed in the third storage unit 35 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like.

[0050] The third control unit 36 ​​includes a processor such as a CPU or MPU, memory such as ROM or RAM, and peripheral circuits, and performs various signal processing for the management device 30. The third control unit 36 ​​includes a management unit 361, etc., which is implemented as a functional module of a program that runs on the processor. A DSP, LSI, ASIC, FPGA, etc. may be used as the third control unit 36.

[0051] The management unit 361 periodically receives captured images transmitted from the flying robot 10 and the current position and / or attitude of the flying robot 10 from the server S via the third communication unit 34. The management unit 361 stores the received captured images, current position and / or attitude in the third storage unit 35 and displays them on the third display unit 32. The management unit 361 also transmits route information 152 to the server S via the third communication unit 34 and registers it with the flying robot 10 via the server S.

[0052] Furthermore, the third control unit 36 ​​receives the setting of areas requiring monitoring in monitoring area A1 by the controller from the third operation unit 31, and transmits the area requiring monitoring setting signal to the server S via the third communication unit 34.

[0053] Figure 3 shows an example of the data structure of a management table. The management table contains the schedule (flight plan) of the flying robot 10. As shown in Figure 3, the management table has settings for one or more schedules, including the start time, start position, end position, and tasks. The start time is the time when the flying robot 10 begins the corresponding task in each schedule. The start and end positions are the starting and ending positions of the flying robot 10 when it performs a task in each schedule. A task is the name or identifier of a task that the flying robot 10 will perform in each schedule. Tasks set in each schedule include tasks that should be performed at predetermined times, such as patrol tasks or return tasks. Tasks set in each schedule may include inspection tasks, delivery tasks, surveying tasks, pesticide spraying tasks, etc.

[0054] As described above, the management table is stored in the storage unit of server S. Each piece of information stored in the management table is registered from the operating device 20 and / or the management device 30. The management table is transmitted from server S to the operating device 20 and may also be stored in the second storage unit 25 of the operating device 20.

[0055] Figure 4 is a flowchart showing an example of the operation of the first setting process by the control device 20. The first setting process is the process of setting the flight mode of the flying robot 10 when an instruction to change the flight mode of the flying robot 10 is received from the operator. This flowchart is executed mainly by the second control unit 26 in cooperation with each element of the control device 20, based on a program that is stored in the second storage unit 25 in advance.

[0056] First, the mode setting unit 262 waits until it receives an instruction from the operator to change the flight mode of the flying robot 10 using the second operation unit 21 (step S101). When an instruction to change the flight mode is received, the mode setting unit 262 determines whether the instructed changed flight mode is manual flight mode or autonomous flight mode (step S102). If the changed flight mode is autonomous flight mode, the mode setting unit 262 further accepts task specifications from the operator using the second operation unit 21. The mode setting unit 262 sends a first control signal to the server S via the second communication unit 24, and then sends it to the flying robot 10 via the server S. The first control signal is a signal requesting that the flight mode be set to autonomous flight mode, and the first control signal includes the mode accepted by the mode setting unit 262. As a result, the mode setting unit 262 sets autonomous flight mode as the flight mode of the flying robot 10, sets a task to be executed by the flying robot 10 from among multiple tasks (step S103), and returns to step S101.

[0057] On the other hand, if the changed flight mode is manual flight mode, the acquisition unit 263 acquires the status of the flight area of ​​the flying robot 10 and stores it in the second storage unit 25 (step S104). First, the acquisition unit 263 identifies the latest current position of the flying robot 10 from the current position of the flying robot 10 which is periodically received from the flying robot 10 via the server S, and then identifies the current flight area of ​​the flying robot 10 from the identified current position.

[0058] The acquisition unit 263 acquires the status of an anomaly in the flight area of ​​the flying robot 10 as the status of the flight area of ​​the flying robot 10. The acquisition unit 263 sends a first request signal to the server S via the second communication unit 24 to request the acquisition of anomaly status information. When the server S receives the first request signal, it sends the anomaly status information to the operating device 20. The acquisition unit 263 receives the anomaly status information from the server S via the second communication unit 24. The acquisition unit 263 identifies whether an anomaly has occurred in the flight area of ​​the flying robot 10, that is, whether an anomaly has occurred in the flight area of ​​the flying robot 10, based on the anomaly status at each location within the monitoring area A1 indicated in the received anomaly status information. Alternatively, the acquisition unit 263 may periodically receive anomaly status information that the server S voluntarily transmits and store it in the second storage unit 25, and identify the anomaly status in the flight area from the anomaly status information stored in the second storage unit 25.

[0059] The acquisition unit 263 may acquire whether or not the flight area of ​​the flying robot 10 is set as a monitoring area, as a status of the flying robot 10's flight area. The acquisition unit 263 sends a second request signal to the server S via the second communication unit 24 to request the acquisition of monitoring area information. When the server S receives the second request signal, it sends the monitoring area information to the operating device 20. The acquisition unit 263 receives the monitoring area information from the server S via the second communication unit 24. The acquisition unit 263 identifies whether or not the flight area of ​​the flying robot 10, particularly at least a part of the flight area, is set as a monitoring area, based on the area set as a monitoring area within monitoring area A1, as indicated in the received monitoring area information. The acquisition unit 263 may also periodically receive monitoring area information that the server S voluntarily transmits and store it in the second storage unit 25, and identify whether or not the flight area is set as a monitoring area from the monitoring area information stored in the second storage unit 25.

[0060] The acquisition unit 263 may also acquire the current security status in the flight area of ​​the flying robot 10 as the status of the flight area of ​​the flying robot 10. The acquisition unit 263 sends a third request signal to the server S via the second communication unit 24 to request the acquisition of security status information. When the server S receives the third request signal, it transmits the security status information to the operating device 20. The acquisition unit 263 receives the security status information from the server S via the second communication unit 24. The acquisition unit 263 identifies the current security status in the flight area of ​​the flying robot 10 from the security status in each area within the monitoring area A1 indicated in the received security status information. That is, the acquisition unit 263 identifies whether the flight area of ​​the flying robot 10, in particular at least a part of the flight area, is set to the alert set state. The acquisition unit 263 may also periodically receive security status information that the server S voluntarily transmits from the server S and store it in the second storage unit 25, and identify the current security status in the flight area from the security status information stored in the second storage unit 25.

[0061] Next, the mode setting unit 262 determines whether the state of the flight area of ​​the flying robot 10 acquired by the acquisition unit 263 is in a state requiring manual monitoring (step S105). A state requiring manual monitoring is a state in which the flying robot 10 needs to be operated (monitored) by an operator manually, or a state in which it is desirable for the flying robot 10 to be operated (monitored) by an operator manually. In other words, a manual monitoring state is a state in which the state of the flight area has a high need for manual monitoring.

[0062] For example, the mode setting unit 262 determines that the state of the flight area requires manual monitoring if an abnormality occurs in the flight area of ​​the flying robot 10, and determines that the state of the flight area does not require manual monitoring if no abnormality occurs in the flight area.

[0063] Alternatively, the mode setting unit 262 determines that the state of the flight area is in a state requiring manual monitoring if the flight area is set as a monitoring area, and determines that the state of the flight area is not in a state requiring manual monitoring if the flight area is not set as a monitoring area. In particular, the mode setting unit 262 determines that the state of the flight area is in a state requiring manual monitoring if at least a part of the flight area is set as a monitoring area, and determines that the state of the flight area is not in a state requiring manual monitoring if no area within the flight area is set as a monitoring area.

[0064] Alternatively, the mode setting unit 262 determines that the state of the flight area is in a state requiring manual monitoring if the flight area is set to a warning state, and determines that the state of the flight area is not in a state requiring manual monitoring if the flight area is not set to a warning state. In particular, the mode setting unit 262 determines that the state of the flight area is in a state requiring manual monitoring if at least a part of the flight area is set to a warning state, and determines that the state of the flight area is not in a state requiring manual monitoring if no area within the flight area is set to a warning state.

[0065] If the flight area of ​​the flying robot 10 is not in a state requiring manual monitoring, the mode setting unit 262 sets the manual operation time in manual flight mode to the first time (step S106). On the other hand, if the flight area of ​​the flying robot 10 is in a state requiring manual monitoring, the mode setting unit 262 sets the manual operation time in manual flight mode to the second time (step S107).

[0066] The second time is set to be longer than the first time. That is, the mode setting unit 262 makes the manual operation time when the state of the flight area of ​​the flying robot 10 is in a state requiring manual monitoring longer than the manual operation time when the state of the flight area of ​​the flying robot 10 is not in a state requiring manual monitoring. This allows the operator to operate the flying robot 10 manually for a longer period of time when it is necessary to operate the flying robot 10 manually, and to perform necessary tasks such as checking for abnormalities or taking action against abnormalities more reliably. On the other hand, when it is not necessary to operate the flying robot 10 manually, manual operation by the operator for a long period of time is limited, and the flying robot 10 can perform its assigned task more reliably.

[0067] The first time is set to a predetermined standard time (e.g., 5 minutes), and the second time may be set to a longer time than the standard time (e.g., 10 minutes). The standard time is set to the minimum, average, or maximum time required for the operator to temporarily check the situation around the flying robot 10, and is a time that does not interfere with the execution of tasks in autonomous flight mode. This allows the operator to more accurately grasp the situation around the flying robot 10 and reliably execute assigned tasks, even when there is not a high need to manually operate the flying robot 10. On the other hand, when it is necessary to manually operate the flying robot 10, the operator can operate the flying robot 10 manually for a long period of time and more reliably perform necessary tasks such as checking for abnormalities or taking action against abnormalities.

[0068] Furthermore, the first hour is set based on the remaining time required from the current time until the completion of the task currently being performed by the flying robot 10, while the second hour may be set regardless of the remaining time. In this case, the mode setting unit 262 first sends a task information acquisition request to the server S via the second communication unit 24 to request the acquisition of task information 251. The server S identifies the latest task set for the flying robot 10 as the task currently being executed by the flying robot 10, according to instructions from the operating device 20, instructions from the management device 30, notification signals from the security device T, or a schedule set in the management table. The server S also refers to the management table to identify the next task to be executed, its start time, and its start position. The server S sends task information 251, indicating the task currently being executed by the flying robot 10, the next task to be executed, its start time, and its start position, to the operating device 20. The mode setting unit 262 receives the task information 251 from the server S via the second communication unit 24 and stores it in the second storage unit 25. If the only device for setting tasks for the flying robot 10 is the operating device 20, the mode setting unit 262 may identify the task set in step S103 as the task currently being executed by the flying robot 10. Furthermore, if a management table is stored in the second storage unit 25, the mode setting unit 262 may refer to the management table to identify the next task to be executed, its start time, and its start position.

[0069] If the flight area status of the flying robot 10 is not in a state requiring manual monitoring, the mode setting unit 262 calculates the time required to move from the flying robot 10's current position to the end position of the task being executed as the remaining time. If the task being executed is a task whose schedule is set in the management table, such as a patrol task or a return task, the mode setting unit 262 refers to the management table stored in the server S or the second storage unit 25 to identify the end position of the task being executed. If the task being executed is a return task, the mode setting unit 262 may identify the flying robot 10's return position (takeoff and landing point D) as the end position of the task being executed. Also, if the task being executed is a specified location movement task, the mode setting unit 262 identifies the location specified by the air traffic controller as the end position of the task being executed. The mode setting unit 262 calculates the remaining time by dividing the distance from the flying robot 10's current position to the end position of the task being executed via the flight path (patrol path) by the flying robot 10's flight speed. The flight speed is, for example, the average flight speed of the flying robot 10. The flight speed may be predetermined for each flight section. Furthermore, if the task being executed is a tracking task or a detection and movement task, the task's end position (end time) cannot be determined. Therefore, the remaining time is set to a sufficiently long time. Also, if the task being executed is a task that performs photography at confirmation points, such as a patrol task, and confirmation points are included in the remaining flight path, the mode setting unit 262 may add the time required for photography to the remaining time. Also, if the task being executed is a task that lands at a return position, such as a return task, the mode setting unit 262 may add the time required for landing to the remaining time. Furthermore, the mode setting unit 262 calculates the time between the current time and the start time of the next task for the flying robot 10 as a grace period. If the end position of the currently executing task and the start position of the next task are different, the acquisition unit 263 may add to the remaining time the travel time obtained by dividing the distance from the end position of the currently executing task to the start position of the next task via the flight path by the flight speed of the flying robot 10. Alternatively, in that case, the mode setting unit 262 may subtract the travel time from the grace period. Also, if the next task involves the flying robot 10 taking off, the mode setting unit 262 may add the time required for takeoff to the grace period. The mode setting unit 262 then calculates the first time (manual operation time) based on the time obtained by subtracting the remaining time from the grace period. For example, the mode setting unit 262 calculates the time obtained by subtracting the remaining time from the grace period as the first time. The mode setting unit 262 may also calculate the first time by adding or subtracting a certain margin to the time obtained by subtracting the remaining time from the grace period. In other words, in this case, the mode setting unit 262 temporarily suspends the currently running task, resumes the task after the manual operation is completed, and sets the manual operation availability period so that the next task can be started according to schedule.

[0070] On the other hand, if the flight area of ​​the flying robot 10 is in a state requiring manual monitoring, the mode setting unit 262 calculates the time required for the flying robot 10 to move from its current position to the starting position of the next task as the return time. The mode setting unit 262 calculates the return time by dividing the straight-line distance from the flying robot 10's current position to the starting position of the next task by the flying speed of the flying robot 10. Furthermore, the mode setting unit 262 calculates the time between the current time and the start time of the next task for the flying robot 10 as a grace period. If the next task involves the flying robot 10 taking off, the mode setting unit 262 may add the time required for takeoff to the grace period. The mode setting unit 262 then calculates the second time (manual operation time) based on the time obtained by subtracting the return time from the grace period. For example, the mode setting unit 262 calculates the time obtained by subtracting the return time from the grace period as the second time. The mode setting unit 262 may also calculate the second time by adding or subtracting a certain margin to the time obtained by subtracting the return time from the grace period. In other words, in this case, the mode setting unit 262 completely stops the currently running task and sets a manual operation time so that the next task can be started on schedule after the manual operation is completed.

[0071] Thus, the mode setting unit 262 sets the manual operation time based on the remaining time when the flight area is not in a state requiring manual monitoring, and sets the manual operation time regardless of the remaining time when the flight area is in a state requiring manual monitoring. As a result, when the flying robot 10 does not need to be operated manually by an operator, the flying robot 10 can reliably perform the task currently being performed and the next task. On the other hand, when the flying robot 10 needs to be operated manually by an operator, the operator can interrupt the execution of the task currently being performed and operate the flying robot 10 manually, thereby more reliably performing necessary tasks such as checking for abnormalities or taking action against abnormalities.

[0072] Furthermore, the mode setting unit 262 sets the time during which manual operation is possible in manual flight mode based on the state of the flight area of ​​the flying robot 10. This allows the mode setting unit 262 to appropriately restrict the operator's manual operation of the flying robot 10 according to the state (situation) of the flight area.

[0073] Next, the mode setting unit 262 determines whether the manual operation time satisfies the change restriction conditions (step S108). The mode setting unit 262 determines that the manual operation time satisfies the change restriction conditions if the manual operation time is less than or equal to the first threshold, and determines that the manual operation time does not satisfy the change restriction conditions if the manual operation time is greater than the first threshold. The first threshold is an example of a threshold. The first threshold is set in advance to 0 or a time that is 0 plus a predetermined margin.

[0074] If the manual operation time limit conditions are met, the mode setting unit 262 decides to restrict the manual flight mode by limiting the change of flight mode from autonomous flight mode to manual flight mode (step S109).

[0075] Next, the notification unit 264 notifies the operator of the flying robot 10 of a warning regarding the restriction on manual flight mode (step S110), and returns to step S101. The notification unit 264 notifies the operator by displaying a warning on the second display unit 22 or by outputting it from the second audio output unit 23 that the change of flight mode from autonomous flight mode to manual flight mode is restricted. The notification unit 264 may also notify the operator of the reason why the restriction on manual flight mode has been imposed (manual operation available time). This allows the operator to understand that they cannot change to manual flight mode and the reason why. In this case, the mode setting unit 262 does not set manual flight mode for the flying robot 10 and does not perform the change of flight mode from autonomous flight mode to manual flight mode. As a result, the flying robot control system 1 can restrict manual operation of the flying robot 10 itself depending on whether the flying robot 10 can properly perform the next task.

[0076] On the other hand, if the task being executed and the manual operation time do not meet the change restriction conditions in step S109, the mode setting unit 262 determines whether the manual operation time meets the operation restriction conditions (step S111). The mode setting unit 262 determines that the manual operation time meets the operation restriction conditions if the manual operation time is less than or equal to the second threshold, and determines that the manual operation time does not meet the operation restriction conditions if the manual operation time is greater than the second threshold. The second threshold is set in advance to a value greater than the first threshold. If the manual operation time does not meet the operation restriction conditions, the mode setting unit 262 proceeds to step S115 without performing any special processing.

[0077] On the other hand, if the manual operation time satisfies the operation restriction conditions, the mode setting unit 262 decides to restrict the manual flight mode by limiting the horizontal movement, altitude change, or turning (direction change) of the flying robot 10 in manual flight mode (step S112). Furthermore, if the manual operation time satisfies the operation restriction conditions, the mode setting unit 262 may impose stricter operation restrictions on the flying robot 10 as the manual operation time decreases. For example, if the manual operation time is less than or equal to the third threshold, the mode setting unit 262 restricts all horizontal movement, altitude changes, and turns. The third threshold is set to a value greater than the first threshold and less than the second threshold. If the manual operation time is greater than the third threshold and less than or equal to the fourth threshold, the mode setting unit 262 restricts horizontal movement and altitude changes, and allows turns. The fourth threshold is set to a value greater than the third threshold and less than the second threshold. If the manual operation time is greater than the fourth threshold, the mode setting unit 262 restricts horizontal movement, and allows turns and altitude changes. As a result, the mode setting unit 262 restricts the flying robot 10 from moving away from its current position as the manual operation time decreases, and can more reliably enable the flying robot 10 to perform the next task on schedule.

[0078] Next, the notification unit 264 notifies the operator of the flying robot 10 of a warning regarding restrictions on manual flight mode (step S113). The notification unit 264 notifies the operator by displaying a warning on the second display unit 22 or by outputting it from the second audio output unit 23 that the flying robot 10's horizontal movement, altitude change, or turning in manual flight mode is restricted. The notification unit 264 may further notify the operator of the operation (horizontal movement, altitude change, or turning) that is restricted in manual flight mode and the reason for it. This allows the operator to understand that they cannot perform a certain operation in manual flight mode and the reason why.

[0079] Next, the mode setting unit 262 transmits a second control signal to the server S via the second communication unit 24, and then transmits it to the flying robot 10 via the server S. The second control signal is a signal requesting that the flight mode be set to manual flight mode. As a result, the mode setting unit 262 sets the flight mode of the flying robot 10 to manual flight mode and causes the flying robot 10 to temporarily suspend the task it is currently performing (step S114). The mode setting unit 262 also stores the current position and time of the flying robot 10 in the second storage unit 25 as the position and time of the flying robot 10 when the manual flight mode was set. Furthermore, if the manual operation time satisfies the operation restriction conditions, the mode setting unit 262 specifies in the second control signal that horizontal movement, altitude changes, or turns of the flying robot 10 should be restricted in manual flight mode. This allows the flying robot control system 1 to restrict some of the manual operations of the flying robot 10 depending on whether the flying robot 10 can properly perform the following tasks. On the other hand, if the manual operation time does not satisfy the operation restriction conditions, the mode setting unit 262 does not specify in the second control signal that horizontal movement, altitude changes, or turns of the flying robot 10 should be restricted in manual flight mode.

[0080] Next, the mode setting unit 262 notifies the operator of the flying robot 10 that the flight mode has been set (changed) to manual flight mode and the remaining manual operation time by displaying it on the second display unit 22, or by outputting it from the second audio output unit 23 (step S115), and then returns to step S101. That is, when the mode setting unit 262 is set to manual flight mode, it notifies the operator of the control device 20 of the remaining manual operation time. As a result, the operator can recognize that the flight mode of the flying robot 10 has been changed to manual flight mode and the remaining time that the flying robot 10 can be operated in manual flight mode, and the control device 20 can improve the operator's convenience.

[0081] Furthermore, the processing in steps S110, S113, and / or S115 may be omitted, and the control device 20 may not output a warning or manual operation time. Also, the processing in steps S108 to S113 may be omitted, and the control device 20 may output the manual operation time without imposing any restrictions on the manual flight mode.

[0082] Figure 5 is a flowchart showing an example of the operation of the second setting process by the control device 20. The second setting process is a process for changing the flight mode and task settings after the flight mode of the flying robot 10 has been set to manual flight mode. This flowchart is executed mainly by the second control unit 26 in cooperation with each element of the control device 20, based on a program that is stored in advance in the second storage unit 25. It is executed periodically when the flight mode of the flying robot 10 is set to manual flight mode.

[0083] First, the mode setting unit 262 updates the manual operation time (step S201). The mode setting unit 262 reads from the second storage unit 25 the time when the manual flight mode was set, which was stored in step S115 in Figure 4, and calculates the elapsed time from the read time to the current time. When the manual flight mode is set, the mode setting unit 262 updates the manual operation time by subtracting the elapsed time or a value obtained by adding or subtracting a margin from the elapsed time, which was calculated in step S108 in Figure 4.

[0084] Furthermore, if the mode setting unit 262 sets the manual operation time based on the remaining time or return time, it may modify the manual operation time based on the current position of the flying robot 10. If the manual operation time is set based on the remaining time, the mode setting unit 262 calculates the recovery time required to move from the current position of the flying robot 10 to the position of the flying robot 10 when manual flight mode was set. The mode setting unit 262 identifies the latest current position of the flying robot 10 from the current position of the flying robot 10 which is periodically received from the flying robot 10 via the server S. The mode setting unit 262 also reads the position of the flying robot 10 when manual flight mode was set, which was stored in step S114 of Figure 4, from the second storage unit 25. The mode setting unit 262 calculates the recovery time by dividing the distance between the current position of the flying robot 10 and the position of the flying robot 10 when manual flight mode was set by the flight speed of the flying robot 10. The mode setting unit 262 updates the manual operation time by subtracting the recovery time or a value obtained by adding or subtracting a margin from the manual operation time. On the other hand, if the manual operation time is set based on the return time, the mode setting unit 262 recalculates the time required for the flying robot 10 to move from its current position to the start position of the next task and updates the return time. The mode setting unit 262 also recalculates the time between the current time and the start time of the flying robot 10's next task and updates the grace period. Then, the mode setting unit 262 recalculates and updates the manual operation time based on the time obtained by subtracting the return time from the grace period. In this way, when manual flight mode is set, the mode setting unit 262 updates the manual operation time based on the current position of the flying robot 10. This allows the mode setting unit 262 to terminate manual flight mode in a way that ensures the robot can start on time for the next task if it moves away from the position where manual flight mode was set due to manual operation by the operator.

[0085] Next, the mode setting unit 262 notifies the operator of the flying robot 10 by displaying the updated manual operation time on the second display unit 22 or by outputting it from the second audio output unit 23 (step S202). This allows the operator to recognize the remaining time that the flying robot 10 can be operated in manual flight mode, and the operating device 20 can improve the operator's convenience.

[0086] Next, the mode setting unit 262 determines whether the updated manual operation time is less than or equal to the first threshold (step S203). If the manual operation time is greater than the first threshold, the mode setting unit 262 continues the manual flight mode without performing any special processing and returns to step S201. On the other hand, if the manual operation time falls below the first threshold, the mode setting unit 262 automatically switches from manual flight mode to autonomous flight mode and moves the flying robot 10 to the return position (step S204). The mode setting unit 262 sets the position of the flying robot 10 at the time the manual flight mode was set to the return position. However, if the mode setting unit 262 sets the manual operation time based on the return time, it sets the starting position of the next task to the return position. The mode setting unit 262 controls the flight of the flying robot 10 by sending an operation signal requesting to move to the return position to the server S via the second communication unit 24, and then sending it to the flying robot 10 via the server S.

[0087] After moving the flying robot 10, the mode setting unit 262 sets the task to be performed by the flying robot 10 (step S205). The mode setting unit 262 sets the task that was temporarily suspended in step S114 as the task to be executed by the flying robot 10. This causes the mode setting unit 262 to resume the task that was running immediately before the manual flight mode was set. The flying robot 10 resumes the temporarily suspended task, and after that task is completed, it executes the next task. However, if the mode setting unit 262 sets the manual operation time based on the return time, it sets the task following the task temporarily suspended in step S114 as the task to be executed by the flying robot 10. This causes the flying robot 10 to execute the next task without executing the temporarily suspended task. Thus, in manual flight mode, the mode setting unit 262 restricts some manual operations when the available manual operation time falls below a first threshold. This allows the flying robot 10 to properly perform pre-planned tasks.

[0088] Next, the mode setting unit 262 notifies the operator of the flying robot 10 that the flight mode has been set (changed) to autonomous flight mode by displaying it on the second display unit 22 or by outputting it from the second audio output unit 23 (step S206), and the series of steps ends. As a result, the operator can recognize that the flight mode of the flying robot 10 has been changed to autonomous flight mode, and the control device 20 can improve the operator's convenience.

[0089] As explained above, the flying robot control system 1 can appropriately control the flying robot 10 by setting its flight mode to manual flight mode when it wants to focus on checking a specific area or when the flying robot 10 loses sight of the target being tracked. On the other hand, the flying robot control system 1 sets the time during which manual operation is possible in manual flight mode based on the state of the flying robot 10's flight area. Therefore, the flying robot control system 1 can appropriately limit the time during which manual operation using the control device 20 is possible for the autonomously flying flying robot 10. In particular, when the flying robot control system 1 is used in so-called drone sharing, where the flying robot 10 is shared among multiple properties, users, or tasks, it can appropriately restrict the execution of manual flight mode so as not to interfere with the execution of the next task (other properties).

[0090] Although preferred embodiments have been described above, the embodiments are not limited to the examples described above. For example, if the mode setting unit 262 decides to impose a restriction on the manual flight mode in step S109 or S112 of Figure 4, it may send a signal to the management device 30 via the server S to request the removal of the restriction. When the mode setting unit 262 receives a signal from the management device 30 via the server S to permit the removal of the restriction, it removes the restriction. This allows the flight robot control system 1 to allow the operator to manually operate the flight robot 10 in the event of an emergency, even when the time to start the next task is approaching, enabling a flexible response to emergencies.

[0091] Furthermore, the restrictions on manual flight mode may include only one of the following: changing the flight mode from autonomous flight mode to manual flight mode, and certain operations of the flying robot 10 in manual flight mode (horizontal movement, altitude change, or turning). In addition, the restricted operations are not limited to horizontal movement, altitude change, or turning; the tilt of the aircraft, turning speed, and movement speed in manual operation may also be restricted.

[0092] Furthermore, the mode setting unit 262 may impose restrictions on the manual flight mode at any time, not just when the operator instructs a change in the flight mode of the flying robot 10. For example, the mode setting unit 262 may impose restrictions on the manual flight mode based on the latest task.

[0093] Furthermore, in the flying robot control system 1, the server S may be omitted. In that case, the flying robot 10, the operating device 20, the management device 30, and / or the security device T will send and receive information from each other without going through the server S.

[0094] Furthermore, in the flying robot control system 1, the management device 30 may impose restrictions on the manual flight mode instead of the operating device 20. In this case, the third storage unit 35 of the management device 30 stores the information stored in the second storage unit 25 of the operating device 20, and the third control unit 36 ​​of the management device 30 has the same functions as the second control unit 26 of the operating device 20 and executes the first setting process and the second setting process. The third control unit 36 ​​acquires various information specified by the operator of the management device 30 using the third operating unit 31, receives various information from the server S via the third communication unit 34, and transmits various signals to the flying robot 10 via the third communication unit 34 and the server S. In addition, if the third control unit 36 ​​imposes restrictions on the manual flight mode, it transmits a warning to the operating device 20 via the third communication unit 34 and the server S, notifying the operator of the operating device 20. Furthermore, when the flight mode is changed, the third control unit 36 ​​outputs a message to the control device 20 via the third communication unit 34 and the server S indicating that the flight mode has been changed and / or that manual operation is possible for a specified period, thereby notifying the operator of the control device 20. In this case as well, the flight robot control system 1 can appropriately limit the period during which manual operation using the control device 20 is possible for the autonomously flying flight robot 10.

[0095] Furthermore, in the flying robot control system 1, the flying robot 10 may impose restrictions on manual flight mode instead of the operating device 20. In this case, the first memory unit 15 of the flying robot 10 stores the information stored in the second memory unit 25 of the operating device 20, and the first control unit 16 of the flying robot 10 has the functions of the second control unit 26 of the operating device 20 and executes the first setting process and the second setting process. The first control unit 16 acquires various information by receiving it from the operating device 20, the management device 30 and / or the security device T via the first communication unit 14 and the server S, and sets it in the first memory unit 15. Also, if the first control unit 16 imposes restrictions on manual flight mode, it sends a warning to the operating device 20 via the first communication unit 14 and the server S, notifying the operator of the operating device 20. Furthermore, when the flight mode is changed, the first control unit 16 outputs a message to the control device 20 via the first communication unit 14 and the server S indicating that the flight mode has been changed and / or that manual operation is possible for a specified period of time, thereby notifying the operator of the control device 20. In this case as well, the flying robot control system 1 can appropriately limit the period of time during which manual operation using the control device 20 is possible for the autonomously flying flying robot 10.

[0096] An embodiment of the present invention, a flying robot control system and a flying robot control method, can contribute to solving social issues such as the declining labor force and long working hours. Furthermore, the flying robot control system and flying robot control method according to one embodiment of the present invention can contribute to achieving Goal 9 of the Sustainable Development Goals (SDGs) adopted by the United Nations, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation." [Explanation of symbols]

[0097] 1 Flying robot control system, 10 Flying robot, 20 Operating device, 30 Management device, 262 Mode setting unit, 263 Acquisition unit

Claims

1. A flying robot control system having a flying robot and an operating device, A mode setting unit sets one of the following as the flight mode of the flying robot: an autonomous flight mode in which the flying robot flies autonomously, and a manual flight mode in which the flying robot flies according to manual operation using the control device. The system includes an acquisition unit that acquires the current position of the flying robot or the state of the flight area including the vicinity of the current position, The mode setting unit sets the manual operation time in the manual flight mode based on the state of the flight area. A flying robot control system characterized by the following:

2. The flight robot control system according to claim 1, wherein the acquisition unit acquires, as the state of the flight area, the state of an abnormality occurring in the flight area, whether or not the flight area is set as a monitoring area, or the current security status in the flight area.

3. The flight robot control system according to claim 1 or 2, wherein the mode setting unit makes the manual operation time when the state of the flight area is in a state requiring manual monitoring, where manual monitoring is highly necessary, longer than the manual operation time when the state of the flight area is not in a state requiring manual monitoring.

4. In the autonomous flight mode, the flying robot flies autonomously to perform a predetermined task. The flying robot control system according to claim 1 or 2, wherein the mode setting unit sets the manual operation time based on the remaining time required to complete the task being performed by the flying robot when the state of the flight area is not a state requiring manual monitoring, and sets the manual operation time regardless of the remaining time when the state of the flight area is a state requiring manual monitoring.

5. The flight robot control system according to claim 1 or 2, wherein the mode setting unit sets the manual operation time to a predetermined standard time when the state of the flight area is not in a state requiring manual monitoring, and sets the manual operation time to a time longer than the standard time when the state of the flight area is in a state requiring manual monitoring.

6. The flying robot control system according to claim 1 or 2, wherein the mode setting unit, when set to the manual flight mode, notifies the operator of the control device of the manual operation time.

7. The flight robot control system according to claim 1 or 2, wherein the mode setting unit restricts some operations in the manual operation when the manual operation time falls below a threshold in the manual flight mode.

8. Computers The flight mode of the flying robot is set to either an autonomous flight mode in which the flying robot flies autonomously, or a manual flight mode in which the flying robot flies according to manual operation using an operating device. This includes obtaining the status of the flight area in which the aforementioned flying robot is flying, In the above setting, conditions regarding the time during which manual operation is possible in the manual flight mode are set based on the state of the flight area. A method for controlling a flying robot, characterized by the features described above.

Images