Aircraft control system, aircraft control device, aircraft control method, and program

Abstract

This improves convenience by eliminating the need to dispatch drone operators to various sites when performing tasks using drones. [Solution] In the CPU 11 of the aircraft control device 1 which constitutes the aircraft control system, the pilot information acquisition unit 33 acquires pilot information for remotely controlling a drone 3 located at a predetermined location using a pilot terminal 2, not through direct wireless communication but through indirect communication via a predetermined network, and the transmission control unit 37 transmits the acquired pilot information to the drone 3 via indirect communication.

Description

Technical Field

[0001] The present invention relates to a flight control system, a flight control device, a flight control method, and a program.

Background Art

[0002] By using drones for various inspection operations, the automation and labor-saving of inspection operations have been promoted, and related technologies have also been proposed (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Although the automation and labor-saving of inspection operations using drones have advanced, a human backup system has been adopted in case of any trouble. For example, when a drone equipped with a camera is used to image an inspection point, it can be switched from automatic operation to manual operation in case of unexpected behavior of the drone. Furthermore, manual operation is performed by an operator dispatched to the inspection site within the range where radio waves can reach. However, since the operator of the drone is a person with a predetermined qualification and a higher operation skill, the number of people who can be dispatched to the inspection site is limited. Therefore, one operator often takes charge of a plurality of inspection sites, and the moving time of the operator moving between inspection sites has become an issue.

[0005] The present invention has been made in view of such a situation, and an object thereof is to improve convenience such that it is not necessary to dispatch an operator of a drone to the site when performing various operations using the drone.

Means for Solving the Problems

[0006] To achieve the above objective, one aspect of the present invention is: A means for acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, by indirect communication via a predetermined network rather than direct wireless communication, A transmission control means that causes the acquired control information to be transmitted to the aircraft via the indirect communication, This is an aircraft control system that has [a certain feature].

[0007] An aircraft control device, aircraft control method, and program corresponding to the aircraft control system according to one aspect of the present invention are also provided as an aircraft control device, aircraft control method, and program corresponding to the aircraft control system according to one aspect of the present invention. [Effects of the Invention]

[0008] According to the present invention, when performing various tasks using drones, it is possible to improve convenience by eliminating the need to dispatch a drone operator to the site. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall configuration of an aircraft control system according to one embodiment of the present invention. [Figure 2] This block diagram shows an example of the hardware configuration of the aircraft control device that makes up the aircraft control system shown in Figure 1. [Figure 3] This block diagram shows an example of the hardware configuration of the drone that makes up the flight control system shown in Figure 1. [Figure 4] Figure 1 is a functional block diagram showing an example of the functional configuration of the aircraft control system, pilot terminal, and drone that make up the aircraft control system. [Figure 5] This flowchart shows an example of the processing flow of an aircraft control system. [Figure 6]This flowchart follows the flowchart shown in Figure 5. [Figure 7] This figure shows a specific example of a service to which the aircraft control system shown in Figure 1 is applied. [Figure 8] This diagram shows specific examples of when field terminals are used. [Modes for carrying out the invention]

[0010] This embodiment will be described below with reference to the drawings. <Aircraft control system S> Figure 1 shows an example of the overall configuration of an aircraft control system S according to one embodiment of the present invention. The aircraft control system S is a system that enables the remote control of the drone 3 as an aircraft. In this invention, "remote control" refers to control via indirect communication over a predetermined communication line, which is a network N. In other words, "remote control" in this invention differs from conventional control using direct communication with radio waves between a transmitter (control terminal) and a receiver (drone). Examples of "indirect communication" via network N include lines provided by communication carriers (telecommunications operators) such as mobile phone lines (e.g., 4G / LTE or 5G), and communication via the Internet using satellite phone lines (e.g., Iridium or Starlink). The communication carrier (telecommunications operator) is not particularly limited.

[0011] The aircraft control system S is configured such that an aircraft control device 1, a control terminal 2, and a drone 3 located at a predetermined location are connected via a network N. In this embodiment, the "predetermined location" in the present invention is assumed to be a site where various inspection work is performed (hereinafter referred to as the "inspection site"). Furthermore, the network N is assumed to be a communication line using "indirect communication" to realize the "remote control" described above.

[0012] [Aircraft control device 1] The aircraft control device 1, which constitutes the aircraft control system S, is an information processing device that acts as a server managing the entire aircraft control system S. The aircraft control device 1 is a device that can execute predetermined application programs that make the aircraft control system S available for use. The aircraft control device 1 can transmit various types of information to the pilot terminal 2, the drone 3, and external sources. In addition, the aircraft control device 1 can acquire various types of information transmitted from the pilot terminal 2, the drone 3, and external sources, and perform various processing operations on it.

[0013] For example, the aircraft control device 1 acquires information about the pilot operating the drone 3 (hereinafter referred to as "pilot information") transmitted from the control terminal 2. The pilot information includes information that can uniquely identify the pilot. The aircraft control device 1 also acquires information about the drone 3 (hereinafter referred to as "aircraft information") transmitted from the drone 3. The aircraft information includes information that can uniquely identify the drone 3. The aircraft information also includes information about the assembly status of the drone 3.

[0014] As will be described in detail later, the drone 3 of this embodiment is an assembled type. The drone 3 includes a plurality of sensors that react when correctly assembled. Information indicating that the sensor has reacted is included in the aircraft information and transmitted to the flight control device 1 and the control terminal 2. Specifically, for example, the drone 3 has an arm portion for connecting four propellers, and it is assumed that the main body (central) portion of the drone 3 and the arm portion can be fitted and removed. In this case, the main body portion and the arm portion of the drone 3 can be fitted only in a predetermined orientation, and it is assumed that the arms have fitting mechanisms of different shapes so that the arms correspond one-to-one. And the drone 3 has a sensor that reacts when the arm portion is properly fitted (inserted to a certain depth). Also, for example, the drone 3 includes a sensor that reacts when the wiring of the drone 3 is connected to a propeller motor or the like. Thus, information of a plurality of sensors that react when correctly assembled, or information indicating that all sensors have reacted, is included in the aircraft information. Thereby, even if the on-site person in charge H lacks expertise, it is communicated to the operator P that the correct setting (assembly, etc.) has been completed.

[0015] Based on the acquired operator information and aircraft information, the flight control device 1 performs an authentication process on whether the operator can operate the drone 3. The result of the authentication process is transmitted from the flight control device 1 to the control terminal 2 and the drone 3. The flight control device 1 that has performed the authentication process allowing the operator to operate the drone 3 acquires control information (hereinafter referred to as "operation information") for remotely operating the drone 3 transmitted from the control terminal 2. Then, the flight control device 1 transmits the acquired operation information to the drone 3.

[0016] The control information includes, for example, information regarding an instruction to start flight (hereinafter referred to as "flight start instruction information") and information regarding an instruction to end flight (hereinafter referred to as "flight end instruction information"). Among these, the flight start instruction information includes, for example, an instruction to start flight, conditions for starting flight, and the like. Also, the flight end instruction information includes, for example, an instruction to end flight, conditions for ending flight, and the like.

[0017] Further, the control information includes information regarding the route along which the drone 3 flies (hereinafter referred to as "flight route information") and information regarding an instruction to interrupt flight (hereinafter referred to as "flight interruption instruction information"). Among these, the flight route information includes, in addition to the flight route, information such as flight range, flight altitude, flight speed, and the like. Also, the flight interruption instruction information includes, for example, an instruction to interrupt flight, information on conditions for interrupting flight, and the like.

[0018] When the flight control device 1 receives sensing information (hereinafter referred to as "sensing information") regarding the result of sensing from the drone 3, it acquires the sensing information. Note that the content of the sensing information will be described later. The flight control device 1 transmits the acquired sensing information to the control terminal 2.

[0019] When the flight control device 1 receives alert information for notifying an abnormal situation from the drone 3, it acquires the alert information. Note that the content of the alert information will be described later. The flight control device 1 transmits the acquired alert information to the control terminal 2. Note that the details of the configuration and processing of the flight control device 1 will be described later.

[0020] 〔Control Terminal 2〕 The control terminal 2, which constitutes the aircraft control system S, is a terminal for remotely controlling the drone 3. The control terminal 2 consists of an information processing device such as a smartphone, tablet, or personal computer operated by the pilot. The control terminal 2 is capable of executing a predetermined application program that enables the aircraft control system S to be used. The control terminal 2 receives various types of input information and transmits it to the aircraft control device 1 or to the outside. In addition, the control terminal 2 is capable of performing various processes based on various types of information transmitted from the aircraft control device 1 and the outside, as well as various types of information input by the pilot.

[0021] For example, the control terminal 2 receives pilot information and transmits it to the aircraft control unit 1. The control terminal 2 also obtains the results of the authentication process transmitted from the aircraft control unit 1 and displays them on a display or the like. Furthermore, the control terminal 2 receives pilot information and transmits that pilot information to the aircraft control unit 1. As described above, the pilot information includes information such as flight start instruction information, flight end instruction information, flight route information, and flight interruption instruction information. The control terminal 2 also obtains sensing information transmitted from the aircraft control unit 1 and displays it on a display or the like. Details of the configuration and processing of the control terminal 2 will be described later.

[0022] [Drone 3] The drone 3, which constitutes the aircraft control system S, is a small, unmanned aircraft equipped with communication functions, information processing functions, and various sensing functions. The drone 3 is capable of executing a predetermined application program that enables the aircraft control system S to be used. The drone 3 receives various input information and transmits it to the aircraft control device 1. The drone 3 is also capable of performing various processes based on various information transmitted from the aircraft control device 1 and various information input by the pilot to the control terminal 2.

[0023] For example, drone 3 transmits its own aircraft information to the flight control device 1. This aircraft information may be transmitted automatically by drone 3, or it may be transmitted triggered by auxiliary operations performed by a field representative (not the pilot) dispatched to the inspection site. Drone 3 also obtains the authentication result from the flight control device 1 regarding whether the pilot is authorized to control drone 3, and displays it on a display or similar device.

[0024] Furthermore, Drone 3 acquires control information transmitted from the aircraft control unit 1 and controls its own flight based on that information. Drone 3 also performs various sensing operations while flying over the inspection site and transmits the results of these sensing operations as sensing information to the aircraft control unit 1. Details of the configuration and processing of Drone 3 will be described later.

[0025] The above-described processing performed by each of the components constituting the aircraft control system S—the aircraft control device 1, the control terminal 2, and the drone 3—is merely an example. Furthermore, as long as the aircraft control system S as a whole has the functionality to implement the above-described processing, some or all of the functions for implementing the above-described processing may be shared or performed collaboratively within the aircraft control system S.

[0026] For example, some or all of the functions of the aircraft control device 1 may be assigned to other devices within the aircraft control system S (such as the pilot terminal 2 or the drone 3). Alternatively, some or all of the functions of other devices within the aircraft control system S may be assigned to the aircraft control device 1. Furthermore, some or all of the functions of the aircraft control device 1 may be transferred to a server or other device not shown in the diagram. This facilitates the overall processing of the aircraft control system S and allows for complementary processing.

[0027] <Hardware Configuration> [Hardware configuration of the aircraft control device 1] Figure 2 is a block diagram showing an example of the hardware configuration of the aircraft control device 1 that constitutes the aircraft control system S in Figure 1. The aircraft control device 1 comprises a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.

[0028] The CPU 11 executes various processes according to the program stored in the ROM 12 or the program loaded from the storage unit 18 into the RAM 13. The RAM 13 also stores data necessary for the CPU 11 to execute various processes. The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14.

[0029] The input / output interface 15 is connected to an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20. The output unit 16 consists of a display, speaker, etc., and outputs various information as images, sounds, etc. The input unit 17 consists of a keyboard, mouse, touch panel, etc., and accepts input of various information. The storage unit 18 consists of a hard disk, DRAM (Dynamic Random Access Memory), etc., and stores various data. The communication unit 19 communicates indirectly with other devices via the aforementioned network N, which is configured as the Internet, etc.

[0030] The drive 20 is appropriately equipped with removable media 21, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. Programs read from the removable media 21 by the drive 20 are installed in the storage unit 18 as needed. The removable media 21 can also store various data stored in the storage unit 18, just like the storage unit 18.

[0031] [Hardware configuration of control terminal 2] The control terminal 2 has a hardware configuration similar to that of the aircraft control device 1 shown in Figure 2. That is, the control terminal 2 has a CPU, ROM, RAM, bus, input / output interface, output unit, input unit, storage unit, communication unit, drive, and removable media, corresponding to the CPU 11, ROM 12, RAM 13, bus 14, input / output interface 15, output unit 16, input unit 17, storage unit 18, communication unit 19, drive 20, and removable media 21 in Figure 2.

[0032] [Hardware configuration of Drone 3] Figure 3 is a block diagram showing an example of the hardware configuration of the drone 3 that constitutes the aircraft control system S in Figure 1. Drone 3 has a hardware configuration similar to that of the aircraft control device 1 shown in Figure 2 above. Specifically, Drone 3 has a CPU 51, ROM 52, RAM 53, bus 54, input / output interface 55, output unit 56, input unit 57, storage unit 58, communication unit 59, drive 60, and removable media 61, corresponding to the CPU 11, ROM 12, RAM 13, bus 14, input / output interface 15, output unit 16, input unit 17, storage unit 18, communication unit 19, drive 20, and removable media 21 in Figure 2.

[0033] In addition to these hardware components, the drone 3 also includes a flight unit 62 consisting of propellers and motors that rotate the propellers, and a sensing unit 63 consisting of various sensors. The various sensors mounted on the sensing unit 63 include, for example, an angular velocity (gyro) sensor, an acceleration sensor, a vision sensor, an ultrasonic sensor, a magnetic compass sensor, a GPS (Global Positioning System) sensor, a temperature sensor, a humidity sensor, and an altitude sensor.

[0034] Among the various sensors mounted on the sensing unit 63, the vision sensors consist of a 2D vision sensor and the like that senses data from images captured by the camera. The vision sensors of the sensing unit 63 perform tasks such as sensing data from images of inspection points and sensing for flight control of the drone 3.

[0035] <Functional Configuration> Figure 4 is a functional block diagram showing an example of the functional configuration of the aircraft control device 1, pilot terminal 2, and drone 3 that constitute the aircraft control system S in Figure 1. [Functional configuration of the aircraft control device 1] In the CPU 11 of the aircraft control device 1, when it is in operation, the pilot information acquisition unit 31 functions as a means for acquiring pilot information, the aircraft information acquisition unit 32 functions as a means for acquiring aircraft information, and the pilot information acquisition unit 33 functions as a means for acquiring pilot information. In addition, the CPU 11 functions as a sensing information acquisition unit 34, an information management unit 35 for managing various types of information, an authentication unit 36 ​​as an authentication means, and a transmission control unit 37 as a transmission control means.

[0036] Furthermore, the memory unit 18 of the aircraft control device 1 is equipped with various databases. For example, it is equipped with a pilot DB 41 for storing pilot information, an aircraft DB 42 for storing aircraft information, a pilot DB 43 for storing piloting information, and a sensing DB 44 for storing sensing information.

[0037] The pilot information acquisition unit 31 acquires pilot information transmitted from the control terminal 2. The aircraft information acquisition unit 32 acquires aircraft information transmitted from the drone 3. The pilot information acquisition unit 33 acquires pilot information transmitted from the pilot terminal 2. The sensing information acquisition unit 34 acquires sensing information transmitted from the drone 3.

[0038] The information management unit 35 stores and manages various types of information in the various databases of the storage unit 18. For example, the information management unit 35 stores and manages pilot information in the pilot DB 41. The information management unit 35 also stores and manages aircraft information in the aircraft DB 42. The information management unit 35 also stores and manages piloting information in the pilot DB 43. The information management unit 35 also stores and manages sensing information in the sensing DB 44.

[0039] The authentication unit 36 ​​performs authentication processing to determine whether the pilot is authorized to operate the drone 3, based on pilot information and aircraft information stored in various databases. For example, if the authentication unit 36 ​​has already associated information that uniquely identifies the pilot (as pilot information) with information that uniquely identifies the aircraft (as aircraft information), it performs authentication processing to authorize operation.

[0040] The transmission control unit 37 causes the communication unit 19 to transmit various types of information. For example, the transmission control unit 37 transmits the authentication result regarding whether the operator is authorized to control the drone 3 to the control terminal 2 and the drone 3. The transmission control unit 37 also transmits the control information acquired from the control terminal 2 to the drone 3. Furthermore, the transmission control unit 37 transmits the sensing information acquired from the drone 3 to the control terminal 2.

[0041] [Functional configuration of control terminal 2] In the CPU of the control terminal 2, the information acquisition unit 71, the output control unit 72, and the transmission control unit 73 function during operation.

[0042] The information acquisition unit 71 acquires various types of information. For example, the information acquisition unit 71 acquires various types of information transmitted from the aircraft control device 1 to the control terminal 2. For example, the information acquisition unit 71 acquires the results of the authentication process transmitted from the aircraft control device 1. The information acquisition unit 71 also acquires sensing information transmitted from the aircraft control device 1.

[0043] The output control unit 72 causes various types of information to be output to the output unit. For example, the output control unit 72 causes the results of the authentication process to be output to the output unit. The results of the authentication process may be displayed on the display as image information such as characters or figures, or output from the speaker as audio information. The output control unit 72 also causes sensing information and alert information to be output to the output unit. The sensing results and alerts may be displayed on the display as image information, or output from the speaker as audio information. The output control unit 72 can also output information about the weather at the inspection site to the output unit. Information about the weather at the inspection site may be calculated from the sensing information.

[0044] The transmission control unit 73 causes the communication unit to transmit various types of information. For example, the transmission control unit 73 causes the input pilot information to be transmitted to the aircraft control unit 1. The transmission control unit 73 also causes the input pilot information to be transmitted to the aircraft control unit 1.

[0045] [Functional Configuration of Drone 3] In the CPU 51 of drone 3, the information acquisition unit 81, the information management unit 82, the flight control unit 83, and the transmission control unit 84 all function during operation.

[0046] The information acquisition unit 81 acquires various types of information. For example, the information acquisition unit 81 acquires control information transmitted from the aircraft control device 1. The information acquisition unit 81 also acquires sensing results from the sensing unit 63 (see Figure 3) as sensing information. Sensing information includes, for example, image data obtained from the vision sensor and sensing results from other sensors. Information calculated from the sensing results is also included in the sensing information. Examples of information calculated from the sensing results include the attitude angles of the drone 3 (inclination angle, roll angle (x-axis rotation), pitch angle (y-axis rotation), yaw angle (z-axis rotation)).

[0047] The information management unit 82 manages various types of information. For example, the information management unit 82 stores and manages sensing information acquired by the information acquisition unit 81 in the database of the storage unit 58 (see Figure 3). The sensing information managed by the information management unit 82 can be stored on external storage media such as SD memory cards and USB memory. Therefore, operators and field personnel can use the sensing information stored on the external storage media.

[0048] The flight control unit 83 controls the flight of the drone 3. For example, the flight control unit 83 controls the flight of the drone 3 based on the sensing results from the sensing unit 63 and the piloting information acquired by the information acquisition unit 81. Specifically, the flight control unit 83 controls the drone 3 to start flying based on flight start instruction information as piloting information. The flight control unit 83 also controls the drone 3 to end flying based on flight end instruction information as piloting information. Furthermore, the flight control unit 83 controls the drone 3 to interrupt flying based on flight interruption instruction information as piloting information. Interruption of the drone 3's flight may occur, for example, due to a malfunction in the drone 3 or an anomaly in the space in which the drone 3 is flying (such as a sudden anomaly in weather).

[0049] The transmission control unit 84 causes the communication unit 59 to transmit various types of information. For example, the transmission control unit 84 causes the aircraft information to be transmitted to the aircraft control device 1. The transmission control unit 84 also causes the sensing information to be transmitted to the aircraft control device 1. Furthermore, the transmission control unit 84 causes the alert information to be transmitted to the aircraft control device 1.

[0050] <Processing flow of the aircraft control device 1> Figure 5 is a flowchart showing an example of the processing flow of the aircraft control device 1. Figure 6 is a flowchart that follows the flowchart shown in Figure 5. As shown in Figures 5 and 6, when pilot information is transmitted from the control terminal 2 (YES in step S1), the aircraft control device 1 acquires the transmitted pilot information (step S2) and proceeds to the decision process in step S3. Conversely, if pilot information is not transmitted from the control terminal 2 (NO in step S1), the aircraft control device 1 proceeds to the decision process in step S3.

[0051] If aircraft information is transmitted from drone 3 (YES in step S3), the aircraft control device 1 acquires the transmitted aircraft information (step S4) and proceeds to the process in step S5. On the other hand, if aircraft information is not transmitted from drone 3 (NO in step S3), the aircraft control device 1 returns to the decision process in step S1.

[0052] The aircraft control unit 1 authenticates whether the pilot is authorized to operate the drone 3 based on the pilot information acquired in step S2 and the aircraft information acquired in step S4 (step S5). If the pilot information and aircraft information correspond and authentication is possible (YES in step S6), the aircraft control unit 1 performs an authentication process that allows the pilot to operate the drone 3 (step S7). The aircraft control unit 1 then notifies the pilot of this (step S8). Specifically, the aircraft control unit 1 transmits information to the control terminal 2 indicating that the pilot is authorized to operate the drone 3.

[0053] In response to this, if authentication is not possible because the pilot information and aircraft information do not correspond (NO in step S6), the pilot is prevented from controlling drone 3 (step S9), and the pilot is notified of this (step S10). Specifically, the aircraft control device 1 transmits information to the control terminal 2 indicating that the pilot is prevented from controlling drone 3. Then, the aircraft control device 1 terminates the process (END).

[0054] When the aircraft control device 1 receives control information from the control terminal 2 (YES in step S11), it acquires the transmitted control information (step S12) and transmits that control information to the drone 3 (step S13). Then, the aircraft control device 1 proceeds to the decision process in step S14. On the other hand, if no control information has been transmitted from the control terminal 2 (NO in step S11), the aircraft control device 1 repeats the decision process in step S11.

[0055] When sensing information is transmitted from the drone 3 (YES in step S14), the aircraft control device 1 acquires the transmitted sensing information (step S15) and transmits that sensing information to the control terminal 2 (step S16). Then, the aircraft control device 1 returns to the decision process in step S11. On the other hand, if no sensing information is transmitted from the drone 3 (NO in step S14), the aircraft control device 1 repeats the decision process in step S14.

[0056] <Specific example> [Specific examples of services to which the aircraft control system S is applied] Figure 7 shows a specific example of a service to which the aircraft control system S shown in Figure 1 is applied (hereinafter referred to as "this service"). Figure 7 shows the service provider that provides this service, the business operator that performs inspection work using this service (hereinafter referred to as the "inspection business operator"), and the inspection site where the inspection business operator carries out the inspection work. The service provider manages the aircraft control device 1. The inspection business operator is shown to have a pilot P who remotely controls the drone 3 by operating the control terminal 2, and a field staff member H who performs auxiliary work related to the flight of the drone 3 that is stationed at the inspection site. Field staff member H is an employee of the inspection business operator (or a locally hired part-time employee) and is not required to have the skills to operate the drone 3. Field staff member H performs tasks such as the following:

[0057] In other words, when field personnel H arrives at the inspection site, they begin preparation work. For example, field personnel H sets up environmental measuring equipment (e.g., anemometer, thermometer, hygrometer, etc.), checks the surrounding conditions (e.g., obstacles, access restrictions, presence of third parties, etc.), takes out the remote control terminal 5 for emergency stop instructions (hereinafter referred to as the "emergency stop remote"), and checks the power supply. Field personnel H also takes out the drone 3 from the transport box in which it is stored and prepares it to be ready for flight. Specifically, field personnel H unfolds the folding parts (propellers, arms, antenna, etc.), attaches the onboard equipment (camera, gimbal, etc.), checks the appearance (no scratches, dirt, deformation, etc.), and checks for unusual noises (no sounds of anything coming loose, nothing rattling around, etc.).

[0058] Furthermore, on-site personnel H will activate drone 3. Specifically, on-site personnel H will connect the battery, check that the power LED lights up, check the startup sound (buzzer, melody), and check the communication status of emergency stop remote control 5. On-site personnel H will also contact pilot P. Specifically, they will inform pilot P about the surrounding conditions (for example, wind speed, temperature and humidity from environmental measuring equipment, presence of obstacles, whether there are people nearby, etc.), confirm that there are no problems with drone 3, and confirm that the battery connection is complete.

[0059] When pilot P initiates remote control of drone 3, they perform tasks such as the following: pilot P starts up control terminal 2, checks the connection status of network N, connects to network N, starts the application program, and enters pilot information. In addition, pilot P receives the authentication result (including aircraft information), checks the sensing information, transmits the control information (flight route information), and notifies the on-site person in charge H that flight preparations are complete.

[0060] Next, the following coordination work takes place between the on-site staff member H and the pilot P. Specifically, on-site staff member H informs pilot P that the drone 3 is ready to begin flight. Specifically, on-site staff member H reconfirms the surrounding conditions, contacts pilot P (to inform them that flight can begin), and visually tracks drone 3 while holding the emergency stop remote control 5. Then, pilot P begins remote control while paying attention to the alert information. Specifically, pilot P transmits control information (flight start instruction information) and monitors sensing information (camera images, sensing results).

[0061] Next, on-site personnel H confirms that drone 3 has started flying. Specifically, on-site personnel H visually confirms that drone 3 has started flying and contacts pilot P to confirm the start of flight. Upon receiving the confirmation of the start of flight, pilot P remotely controls drone 3 based on the sensing information obtained from drone 3. That is, the sensing information includes data from surrounding image data obtained from the vision sensor, as well as data for flight control obtained from the angular velocity (gyro) sensor, acceleration sensor, vision sensor, ultrasonic sensor, magnetic compass sensor, GPS sensor, etc. Therefore, the pilot can remotely control the flight of drone 3, including controlling landing and flight control to avoid danger.

[0062] If an abnormal situation occurs with drone 3 during inspection work, the field staff member H and the pilot P will each perform the following actions to respond to the abnormality. Specifically, field staff member H will check for abnormalities in the surrounding environment (e.g., strong winds, gusts, approaching third parties, aircraft behavior), and if there is a problem, will perform an emergency stop operation using the emergency stop remote control 5 and contact the pilot. Pilot P will also check the sensing information from drone 3 (image data, altitude, battery level, communication status, GPS sensitivity, GPS data, etc.), and if there is a problem, will transmit pilot information (flight interruption instruction information) to field staff member H. As a result, drone 3 will perform flight control (emergency stop operation) according to the content of the flight interruption instruction information, for example, by hovering or automatically landing in place, or returning to the takeoff point and hovering or automatically landing. In cases of particularly high urgency, a parachute may be deployed from drone 3.

[0063] Furthermore, the drone 3 is equipped with LEDs, which can be used to indicate the communication status with the aircraft control device 1 and the control terminal 2. For example, the drone 3 can control the LEDs in a predetermined manner, such as by changing the LED color or by combining on, blinking, and off states, depending on the communication status. This allows the field operator H to easily understand the communication status simply by keeping the drone 3 in their line of sight, without having to check the control terminal 2. If the field operator H sees the LEDs and communication with the aircraft control device 1 and the control terminal 2 is lost, they can perform an emergency stop operation.

[0064] Furthermore, if communication is lost or an emergency stop operation is performed, Drone 3 will hover in place as an emergency stop operation, and if a predetermined time elapses without instructions from the pilot P or on-site personnel H, or if the battery level falls below a predetermined value, it can automatically land (lower its altitude) in place.

[0065] The parachute can be deployed based on instructions from the aircraft control device 1 or the control terminal 2, but it can also be deployed without instructions from the aircraft control device 1 or the control terminal 2. For example, a parachute deployment device (not shown) is mounted on the drone 3 and deploys the parachute by detecting the tilt of the aircraft. Specifically, for example, the parachute deployment device deploys the parachute when the aircraft is tilted at an angle of 45° or more. This allows the parachute to be deployed even when the pilot P or the on-site personnel H cannot intervene or when there is a sudden change in the situation that prevents them from intervening.

[0066] Furthermore, the parachute deployment device can transmit information to the drone 3 indicating that the parachute has been deployed once it has deployed. Upon receiving this information, the drone 3 will automatically stop its propellers (especially the propellers) if the aircraft (particularly the propellers) is controllable. This allows the drone 3 to continue recording its own flight log (data for flight control and information recording the flight path, etc.) while landing relatively safely using the parachute.

[0067] When the flight of drone 3 is to be terminated upon completion of the inspection work, the following coordination work will be carried out between the on-site personnel H and the pilot P. On-site personnel H will perform a check before the end of the flight. Specifically, on-site personnel H will check the surrounding conditions (for example, whether there are any third parties present) and inform pilot P that landing is possible. Pilot P will then perform the operation to terminate the flight of drone 3. Specifically, pilot P will send operation information (flight termination instruction information) with the following content.

[0068] The transmitted pilot information (flight termination instruction information) includes, for example, instructions to return to the takeoff point and perform an automatic landing, and instructions to terminate the flight (for example, stopping or ending the rotation of the propellers). In addition, pilot P will contact on-site personnel H to inform them that the flight is being terminated, to instruct them to retrieve sensing data from drone 3 (for example, captured image data), to provide flight logs, and to create a flight log. Furthermore, when drone 3 or the terminal used by on-site worker H (for example, on-site terminal 4 described later using Figure 8) receives the flight termination instruction information, the instruction is announced from drone 3 or the terminal used by on-site worker H. This automates notification to on-site worker H and prevents human error (forgetting to notify).

[0069] Afterward, on-site staff member H performs the tasks necessary to end the flight of drone 3. Specifically, on-site staff member H receives notification from pilot P that the flight has ended. Then, using an external storage medium, H retrieves sensing information (e.g., image data) and flight logs from the drone 3. H also removes the battery from drone 3. On-site staff member H then performs a post-flight check. Specifically, H checks the appearance of the drone 3 (e.g., for scratches, dirt, deformation, etc.), checks for any looseness in the aircraft, and stores the aircraft (folding the propellers, arms, and antenna, removing the camera and gimbal, packing it into the shipping box, etc.). On-site staff member H also performs the dismantling work. Specifically, H stores the emergency stop remote control 5, stores the environmental measurement equipment, loads peripheral equipment, reports the completion of the work to the pilot, and dismantles (e.g., shipping the equipment to the next inspection site, traveling by car, etc.). Drone 3 may automatically transmit sensing information and aircraft information to the aircraft control device 1 and the pilot terminal 2, etc., upon receiving flight termination instruction information. This automates data retrieval and prevents human error (forgetting to send a notification).

[0070] Furthermore, when the terminal used by field worker H (for example, field terminal 4, described later using Figure 8) receives flight termination instruction information, the terminal used by field worker H automatically outputs a shipping label for sending the packaged drone 3 to the next inspection location or service provider, etc. If there are no abnormalities in the sensing information or aircraft information of Drone 3, and there is a next location for use of Drone 3 (inspection location, etc.), the address for the next location for use of Drone 3 will be printed on the shipping label. If there are abnormalities in the sensing information or aircraft information, the address of a location managed by the service provider (for example, the office or warehouse of pilot P) will be printed on the shipping label. In other words, in this service, Drone 3 is the property of the service provider, and it is assumed that the user of the service will be the field worker H. In this case, Drone 3 needs to be shipped as appropriate to other inspection locations requested by the service provider. With the system described above, human error can be avoided even if field worker H lacks specialized knowledge, and Drone 3 can be shipped to the next inspection location, promoting smooth operations.

[0071] According to this service, even if the personnel dispatched to the inspection site where drone 3 is used for inspection work are all individuals who do not possess the skills to operate drone 3 (for example, on-site personnel H in Figure 7), the inspection work can be carried out smoothly.

[0072] <Advantageous effects of this embodiment> According to the above-described embodiment, when performing various tasks using drones, it becomes unnecessary to dispatch a drone operator to the site.

[0073] <Other> Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and any modifications, improvements, etc., that can achieve the objectives of the present invention are included in the present invention. For example, the specific example of the service shown in Figure 7 is merely one example for achieving the objectives of the present invention and is not particularly limited.

[0074] In other words, in the example in Figure 7, the field staff member H prepares and cleans up the drone 3 at the inspection site and performs tasks such as emergency stopping of the drone 3 in the event of an abnormal situation, but is not limited to these tasks. For example, the field staff member may be provided with a notebook-type information processing device (hereinafter referred to as the "field terminal") connected to the network N (see Figure 1), so that various types of information are exchanged between the aircraft control device 1 and the drone 3 via the field terminal. The following explanation will be given with reference to Figure 8.

[0075] Figure 8 shows a specific example of when field terminal 4 is used. If various types of information are exchanged between the aircraft control device 1 and the drone 3 via the field terminal 4, the drone 3 and the field terminal 4 may be connected via direct communication using radio waves. This allows the service to be used even if the drone 3 is unable to perform indirect communication via the network N (for example, if it cannot connect to the internet), as long as the field terminal can connect to the network N. Here, direct communication using radio waves refers to, for example, wireless communication in the 920MHz band, wireless communication in the 2.4GHz band, Wi-Fi, Bluetooth, etc.

[0076] Furthermore, while the above-described embodiment involves flight control of a drone (for example, Drone 3 in Figure 1), the aircraft is not limited to a drone. Any aircraft having communication functions, information processing functions, and sensing functions is acceptable.

[0077] Furthermore, in the embodiments described above, the "predetermined location" in the present invention is an inspection site where various inspection tasks are performed. However, the "predetermined location" in the present invention is not limited to an inspection site. It can be any location where the aircraft can perform sensing such as imaging while in flight.

[0078] Furthermore, the series of processes described above can be executed by hardware or by software. In other words, the functional configuration described above is merely illustrative and not particularly limiting. That is, it is sufficient for the information processing system to have the functionality to execute the series of processes described above as a whole, and the functional blocks used to realize this functionality are not particularly limited to the examples given above.

[0079] Furthermore, the location of the functional blocks is not particularly limited and can be arbitrary. For example, the functional blocks of the aircraft control device 1 may be transferred to other devices, or the functional blocks of other devices may be transferred to a server, etc. Also, a single functional block may consist of hardware alone, software alone, or a combination of both.

[0080] When a series of processes are executed by software, the programs that make up that software are installed on a computer or other device from a network or storage medium. The computer may be a computer built into dedicated hardware. Alternatively, the computer may be a computer capable of performing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0081] Such recording media containing programs may consist not only of removable media (not shown) distributed separately from the main device to provide programs to users, but also of recording media provided to users in a state where they are pre-installed in the main device. Since programs can be distributed via a network, the recording media may be installed on or accessible from a computer connected to or capable of connecting to a network.

[0082] In this specification, the step of describing a program to be recorded on a recording medium includes not only processes that are performed chronologically in that order, but also processes that are not necessarily performed chronologically, but are executed in parallel or individually. Furthermore, in this specification, the term "system" refers to an overall system composed of multiple devices, means, etc.

[0083] In other words, the aircraft control system to which the present invention is applied can take various forms having the following configurations. (1) That is, the aircraft control system S to which the present invention is applied includes a control information acquisition means (for example, a control information acquisition unit 33 in Figure 4) that acquires control information for remotely controlling an aircraft (for example, a drone 3 in Figure 1) located at a predetermined location using a predetermined control terminal (for example, a control terminal 2 in Figure 1) via indirect communication over a predetermined network (for example, network N in Figure 1) rather than direct wireless communication, A transmission control means (for example, a transmission control unit 37 in Figure 4) that transmits the acquired control information to the aircraft via indirect communication, This is an aircraft control system that has [a certain feature].

[0084] (2) The transmission control means can also transmit, as control information, flight start instruction information relating to an instruction to start a flight and flight end instruction information relating to an instruction to end a flight to the aircraft.

[0085] (3) The transmission control means may also transmit flight route information relating to the flight route to be flown as the control information to the aircraft.

[0086] (4) The transmission control means may also transmit flight interruption instruction information relating to an instruction to interrupt flight as the control information to the aircraft.

[0087] (5) The system further includes sensing information acquisition means (for example, sensing information acquisition unit 34 in Figure 4) that acquires sensing information relating to the results of sensing by the aircraft flying based on the pilot information from the aircraft via indirect communication, The transmission control means can further cause the acquired sensing information to be transmitted to the control terminal via indirect communication.

[0088] (6) In addition, a pilot information acquisition means (for example, the pilot information acquisition unit 31 in Figure 4) that acquires pilot information relating to the pilot performing the operation from the control terminal via indirect communication, An aircraft information acquisition means (for example, the aircraft information acquisition unit 32 in Figure 4) acquires aircraft information relating to the aircraft from the aircraft via the indirect communication, An authentication means (for example, the authentication unit 36 ​​in Figure 4) that authenticates whether the pilot is authorized to control the aircraft based on the acquired pilot information and aircraft information, Furthermore, The transmission control means can further transmit the authentication result as authentication information to the control terminal and the aircraft.

[0089] (7) Furthermore, the flying object is connected by direct communication to a predetermined information processing device located near the predetermined location. The information processing device can be connected to the predetermined network via the indirect communication method.

[0090] Furthermore, the aircraft control device to which the present invention is applied can take various forms having the following configurations. (8) That is, the aircraft control device 1 to which the present invention is applied includes a means for acquiring control information for controlling an aircraft located at a predetermined location by remote control from a predetermined control terminal, by indirect communication via a predetermined network, not by direct wireless communication, A transmission control means that causes the acquired pilot information to be transmitted to the aircraft via the indirect communication, This is an aircraft control system that has [a certain feature].

[0091] Furthermore, the aircraft control method to which the present invention is applied can take various forms having the following configurations. (9) That is, the aircraft control method to which the present invention is applied includes the step of acquiring control information for controlling an aircraft positioned at a predetermined location by remote control from a predetermined control terminal, through indirect communication via a predetermined network, not through direct wireless communication, The steps include transmitting the acquired control information to the aircraft via the indirect communication, This is an aircraft control method that includes [the following].

[0092] Furthermore, the program to which the present invention is applied can take various forms having the following configurations. (10) That is, the program to which the present invention is applied is a computer-controlled aircraft control device 1, The steps include: acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, not through direct wireless communication, but through indirect communication via a predetermined network; The steps include transmitting the acquired control information to the aircraft via the indirect communication, This is a program that performs information processing that includes [specific data / information]. [Explanation of Symbols]

[0093] 1: Aircraft control device, 2: Pilot terminal, 3: Drone, 4: Field terminal, 11: CPU, 16: Output unit, 17: Input unit, 18: Memory unit, 19: Communication unit, 31: Pilot information acquisition unit, 32: Aircraft information acquisition unit, 33: Pilot information acquisition unit, 34: Sensing information acquisition unit, 35: Information management unit, 36: Authentication unit, 37: Transmission control unit, 51: CPU, 56: Output unit, 57: Input unit, 58: Memory unit, 59: Communication unit, 62: Flight unit, 63: Sensing unit, 71: Information acquisition unit, 72: Output control unit, 73: Transmission control unit, 81: Information acquisition unit, 82: Information management unit, 83: Flight control unit, 84: Transmission control unit, S: Aircraft control system, N: Network

Claims

1. A means for acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, by indirect communication via a predetermined network rather than direct wireless communication, A transmission control means that causes the acquired pilot information to be transmitted to the aircraft via the indirect communication, An aircraft control system having

2. The transmission control means causes the aircraft to transmit, as pilot information, flight start instruction information relating to an instruction to start flight and flight end instruction information relating to an instruction to end flight. The aircraft control system according to claim 1.

3. The transmission control means further transmits flight route information relating to the flight route to be flown as the control information to the aircraft. The aircraft control system according to claim 1.

4. The transmission control means further transmits flight interruption instruction information relating to an instruction to interrupt flight as pilot information to the aircraft. The aircraft control system according to claim 1.

5. The system further includes sensing information acquisition means for acquiring sensing information relating to the results of sensing by the aircraft flying based on the aforementioned piloting information, from the aircraft via the aforementioned indirect communication. The transmission control means further causes the acquired sensing information to be transmitted to the control terminal via indirect communication. The aircraft control system according to claim 1.

6. A pilot information acquisition means that acquires pilot information relating to the pilot performing the aforementioned operation from the control terminal via the aforementioned indirect communication, An aircraft information acquisition means for acquiring aircraft information relating to the aircraft from the aircraft via the indirect communication, An authentication means that authenticates whether the pilot is authorized to control the aircraft based on the acquired pilot information and aircraft information, Furthermore, The transmission control means further transmits the authentication result as authentication information to the control terminal and the aircraft. The aircraft control system according to claim 1.

7. The flying object is connected to a predetermined information processing device located near the predetermined location via direct communication. The information processing device is connected to the predetermined network by the indirect communication. The aircraft control system according to claim 1.

8. A means for acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, by indirect communication via a predetermined network rather than direct wireless communication, A transmission control means that causes the acquired pilot information to be transmitted to the aircraft via the indirect communication, A flight control device having

9. The steps include: acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, not through direct wireless communication, but through indirect communication via a predetermined network; The steps include transmitting the acquired control information to the aircraft via the indirect communication, A method for controlling an aircraft, including the control of an aircraft.

10. On the computer, The steps include: acquiring control information for remotely controlling an aircraft positioned at a predetermined location from a predetermined control terminal, not through direct wireless communication, but through indirect communication via a predetermined network; The steps include transmitting the acquired control information to the aircraft via the indirect communication, A program that performs information processing, including the processing mentioned above.

Images