Flight control system, flight control method, and program

Abstract

To make it possible to safely operate a flying body.SOLUTION: A system that controls flight of a flying body includes a control unit that controls the flying body according to an operation instruction sent from a user. In a first operation mode, the control unit makes it possible to concurrently receive plural operation instructions. In a second operation mode, the control unit makes it possible to receive only one operation instruction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a flight control system, a flight control method, and a program. [Background technology]

[0002] Various services using aircraft are provided (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-15697 Summary of the Invention [Problem to be solved by the invention]

[0004] Normally, operations are performed while visually observing the aircraft, but when operating an aircraft from a remote location, sudden operations can cause it to become uncontrollable.

[0005] The present invention has been made in view of the above background, and has as its object to enable safe operation of an aircraft. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is a system for controlling the flight of an aircraft, which includes a control unit that controls the aircraft in response to operation instructions from a user, and in a first operation mode, the control unit is capable of accepting multiple operation instructions simultaneously, and in a second operation mode, the control unit is capable of accepting only one operation instruction.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings. [Effects of the Invention]

[0008] According to the present invention, it is possible to operate a flying object safely. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an overview of a system 1 according to an embodiment of the present disclosure. [Figure 2] 2 is a diagram illustrating an example of the hardware configuration of the control device 10 and the remote device 30. FIG. [Figure 3] A block diagram showing an example of the functional configuration of the unmanned aerial vehicle 20 according to this embodiment. [Figure 4] 2 is a block diagram showing the functional configuration of a control unit 11 in a control device 10. FIG. [Figure 5] FIG. 2 is a diagram illustrating the operation of the present system 1. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] <System Overview> FIG. 1 is a diagram illustrating an overview of a system 1 according to an embodiment of the present disclosure. As illustrated, the system 1 includes a control device 10 and a remote device 30. The system 1 according to this embodiment can be used, for example, for construction management and inspection of a building that is the subject of photography by an unmanned aerial vehicle 20. In this system 1, a user can control the unmanned aerial vehicle 20 using the control device 10. In this embodiment, the unmanned aerial vehicle 20 can be indirectly controlled by issuing instructions to the control device 10 from the remote device 30 located at a remote location.

[0012] The control device 10 and the remote device 30 may be implemented by a small tablet-like computer. In other embodiments, the control device 10 and the remote device 30 may be implemented by a portable information processing terminal such as a smartphone or game console, or by a stationary information processing terminal such as a personal computer. The control device 10 and the remote device 30 may also be implemented by multiple pieces of hardware, with functions distributed among them.

[0013] <Control device 10 and remote device 30> 2 is a diagram showing an example of the hardware configuration of the control device 10 and the remote device 30. As shown in the figure, the control device 10 and the remote device 30 include a control unit 11 and a touch panel 12, which is an example of a display unit.

[0014] The processor 11a is an arithmetic unit that controls the operation of the control unit 11, controls the transmission and reception of data between each element, and performs processes necessary for executing programs, etc. In this embodiment, the processor 11a is, for example, a CPU (Central Processing Unit), and performs various processes by executing programs stored in the storage 11c (described later) and expanded in the memory 11b.

[0015] The memory 11b includes a main storage device that is configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary storage device that is configured with a nonvolatile storage device such as a flash memory, an HDD (Hard Disc Drive), etc. This memory 11b is used as a working area for the processor 11a, and also stores a BIOS (Basic Input / Output System) that is executed when the control unit 11 starts up, various setting information, etc.

[0016] The storage 11c stores programs, information used for various processes, etc. For example, when a user operates an unmanned aerial vehicle for capturing image information of the roof 101 via the information processing terminal 10, the storage 11c may store a program for controlling the flight of the unmanned aerial vehicle.

[0017] The transmitter / receiver 11d connects the controller 11 to a network such as the Internet, and may include a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

[0018] The input / output unit 11e is an interface to which an input / output device is connected, and in this embodiment, the touch panel 12 is connected.

[0019] The bus 11f transmits, for example, address signals, data signals, and various control signals between the connected processor 11a, memory 11b, storage 11c, transmission / reception unit 11d, and input / output unit 11e.

[0020] The touch panel 12 is an example of a display unit and includes a display surface on which acquired videos and images are displayed. In this embodiment, the display surface accepts information input by touching the display surface, and is implemented using various technologies such as a resistive film system or a capacitive system.

[0021] The display surface of the touch panel 12 may display, for example, images captured by the unmanned aerial vehicle 20. The display surface may also display buttons, objects, etc. for controlling the flight of the unmanned aerial vehicle 20 and for controlling the imaging device. The user may also input operational instructions via the touch panel 12 to the images, buttons, etc. displayed on the display surface. While such operational instructions will be described in detail later, operations related to inputting such operational instructions include, for example, touch (tap) operations on buttons, objects, etc., and tap operations, slide operations, swipe operations, etc. for determining the flight path of the unmanned aerial vehicle 20.

[0022] <Unmanned Aerial Vehicle 20> Fig. 3 is a block diagram showing an example of the functional configuration of the unmanned aerial vehicle 20 according to this embodiment. As shown in Fig. 3, the unmanned aerial vehicle 20 according to one embodiment includes a main body 21, a transceiver 22, a flight controller 23, a battery 24, an ESC 25, a motor 26, a propeller 27, and a camera 28. The unmanned aerial vehicle 20 is an example of an aerial vehicle. The type of the aerial vehicle is not particularly limited, and may be, for example, a multi-rotor drone as shown in Fig. 3.

[0023] The flight controller 23 may have one or more processors 23A, such as programmable processors (e.g., central processing units (CPUs)).

[0024] Flight controller 23 has and has access to memory 23B, which stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps.

[0025] The memory 23B may include a separable medium such as an SD card or random access memory (RAM) or an external storage device. Data acquired from the sensors 23C may be directly transmitted to and stored in the memory 23B. For example, still image and video data captured by the camera 28 is recorded in the built-in memory or an external memory.

[0026] Flight controller 23 includes a control module configured to control the state of unmanned aerial vehicle 20. For example, the control module controls the propulsion mechanism (e.g., motor 26) of unmanned aerial vehicle 20 via ESC (Electric Speed ​​Controller) 25 to adjust the spatial position, speed, and / or acceleration of unmanned aerial vehicle 20, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). The control module can control one or more of camera 28, sensors 23C, etc.

[0027] The flight controller 23 can communicate with a transceiver 22 configured to transmit and / or receive data from one or more external devices (e.g., a terminal such as the information processing terminal 10, a display device, or other remote control). For example, the transceiver 22 can utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, radio, WiFi, a point-to-point (P2P) network, a telecommunications network, cloud communications, etc.

[0028] The transceiver unit 22 can transmit and / or receive one or more of the following: data acquired by the camera 28 or sensors 23C, processing results generated by the flight controller 23, predetermined control data, user commands from the information processing terminal 10 or a remote controller, etc.

[0029] The sensors 23C according to this embodiment may include an inertial sensor (acceleration sensor, gyro sensor), a GPS sensor, a proximity sensor (e.g., lidar), or a vision / image sensor (e.g., camera).

[0030] The battery 24 may be a known battery such as a lithium polymer battery. The power that drives the unmanned aerial vehicle 20 is not limited to the electric power supplied from the battery 24, but may be powered by an internal combustion engine, for example.

[0031] Camera 28 is an example of an imaging device. The type of camera 28 is not particularly limited, and may be, for example, a normal digital camera, a spherical camera, an infrared camera, a thermography image sensor, or the like. Camera 28 may be connected to main body 21 by a gimbal (not shown) or the like so as to be independently displaceable.

[0032] <Control Unit 10> Figure 4 is a block diagram showing the functional configuration of the control unit 11 in the control device 10. As shown in Figure 4, the control unit 11 of the control device 10 includes an image acquisition unit 111, a display control unit 112, an operation instruction acquisition unit 113, a flight control information generation unit 114, a flight control information transmission unit 115, a status acquisition unit 116, a status display unit 117, and a screen operation sharing unit 118. Each of these functional units can be realized by the processor 11a of the remote device 30 reading a program stored in the storage 11c into the memory 11b and executing it.

[0033] The image acquisition unit 111 acquires images captured by the unmanned aerial vehicle 20. The image acquisition unit 111 acquires images captured by the camera 28 mounted on the unmanned aerial vehicle 20. The acquired images may be moving images obtained in real time, or still images captured at any timing. Information about the acquired images is output to the display control unit 112.

[0034] The display control unit 112 displays the acquired image on the touch panel 12. The display control unit 112 also has a function of displaying information such as buttons, objects, and text in the image to provide information to the user of the present system 1 and to acquire operation instructions based on operations by the user.

[0035] The operation instruction acquisition unit 113 acquires operation instructions generated based on operations on the image displayed on the touch panel 12. The operation instructions include instructions to specify the orientation of the unmanned aerial vehicle 20's attitude (direction of rotation around the pitch axis, roll axis, and yaw axis), the direction of movement of the unmanned aerial vehicle 20 (front / back, left / right, up / down directions), the flight speed of the unmanned aerial vehicle 20, shooting with the camera 28, zooming the camera 28, the orientation (angle) of the camera 28, switching the camera type, automatic return, etc. The operation instruction acquisition unit 113 can also accept multiple operation instructions simultaneously, for example, to increase flight speed while moving forward.

[0036] The flight control information generation unit 114 generates information (flight control information) for controlling the flight of the unmanned aerial vehicle 20 based on the operation instructions. The flight control information is control information for controlling the operation of the unmanned aerial vehicle 20 in accordance with the operation instructions described above. The flight control information can include control information corresponding to multiple operation instructions.

[0037] The flight control information transmission unit 115 transmits the flight control information generated by the flight control information generation unit 114 to the unmanned aerial vehicle 20. The unmanned aerial vehicle 20 can operate based on the flight control information.

[0038] The status acquisition unit 116 acquires the status of the unmanned aerial vehicle 20. The status acquisition unit 116 can acquire, for example, measurement values ​​from sensors equipped in the unmanned aerial vehicle 20 as status information. The status acquisition unit 116 can acquire, as status information, various types of information observable in the unmanned aerial vehicle 20, such as the presence or absence of obstacles around the unmanned aerial vehicle 20, the position information of the unmanned aerial vehicle 20, the movement speed of the unmanned aerial vehicle 20, and the altitude of the unmanned aerial vehicle 20.

[0039] The status display unit 117 displays the status of the unmanned aerial vehicle 20 on the touch panel 12. The status display unit 117 can update the status displayed on the touch panel 12 each time the status acquisition unit 116 acquires status information from the unmanned aerial vehicle 20.

[0040] The screen operation sharing unit 118 enables the remote device 30 to display and operate the screen of the control device 10. The screen operation sharing unit 118 can be realized by, for example, VNC (Virtual Network Computing) using the RFB (Remote Frame Buffer) protocol. Note that a method other than VNC may also be adopted.

[0041] The control device 10 has two operating modes. The first operating mode is a normal operating mode in which a user directly operates the control device 10 to control the unmanned aerial vehicle 20, and the second operating mode is a remote operation mode in which the control device 10 is remotely operated from a remote device 30.

[0042] In the second operating mode, the flight control information generator 114 generates flight control information by adding restrictions to the operation instructions. The flight control information generator 114 generates flight control information corresponding to only one operation instruction. This makes it possible to restrict the operation of the unmanned aerial vehicle 20 that is in a remote location and cannot be seen.

[0043] For example, the flight control information generation unit 114 can be configured to include only operation instructions for a specific direction in the flight control information. For example, when operation instructions for the forward and right directions are input, the flight control information generation unit 114 can generate flight control information that includes only the forward direction or the right direction. In this case, a priority of directions may be set in advance, and flight control information that includes only the highest priority direction among multiple input directions may be generated. For example, when priority is set to parallel movement (forward / backward / left / right), the flight control information generation unit 114 can generate flight control information that includes only the forward direction when forward and upward directions are input. Furthermore, when input information for a diagonally forward right direction is input when only forward / backward / left / right directions are set in advance, the flight control information generation unit 114 can generate flight control information that includes only the forward direction.

[0044] Furthermore, for example, the flight control information generation unit 114 can include only predetermined directions of movement (e.g., up, down, forward, backward) and only orientation (rotational movement) in the flight control information. The flight control information generation unit 114 can ignore operation instructions in directions other than the predetermined directions (e.g., up, down, forward, backward). In this case, the flight control information generation unit 114 can generate flight control information such that orientation and movement direction are not changed simultaneously.

[0045] Furthermore, for example, when the flight control information generation unit 114 receives an operation instruction to change the flight speed, it can ignore the operation instruction and include a predetermined flight speed in the flight control information. The predetermined flight speed can be, for example, a speed slower than the flight speed of the unmanned aerial vehicle 20 during normal flight. The flight speed of the unmanned aerial vehicle 20 during normal flight may be set in advance, or the average speed of the unmanned aerial vehicle 20 may be calculated. The flight control information generation unit 114 can generate flight control information so that the speed does not change.

[0046] <Operation> FIG. 5 is a diagram illustrating the operation of the present system 1.

[0047] The control device 10 receives an operation instruction from the user (S301). The operation instruction may be received directly from the touch panel 12 of the control device 10 or may be received from the remote device 30 by sharing a screen operation.

[0048] When the control device 10 receives one operation instruction (S302::NO), it can create flight control information including the received operation instruction regardless of the operation mode (S303). Note that in step S302, in the second operation mode, a predetermined operation (e.g., a change in flight speed) may be ignored.

[0049] If the control device 10 has received multiple operation instructions (S302: YES), and is in the second operation mode (S304: YES), it creates flight control information including only one operation instruction (S305).

[0050] As described above, according to the system of this embodiment, the user of the remote device 30 can operate the unmanned aerial vehicle 20 even if it is an unmanned aerial vehicle 20 that cannot be seen by sight, while looking at images taken by the unmanned aerial vehicle 20 and referring to various sensor values ​​measured by the unmanned aerial vehicle 20. In the system of this embodiment, in the remote control mode (second operation mode), flight control information is created by adding restrictions to operation instructions, so that it is possible to prevent, for example, excessive speed, collisions in directions where the obstacle sensors are not effective, unclear behavior due to complex operation, or loss of understanding of the surrounding environment of the remote device 30.

[0051] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0052] For example, the processing by each functional unit of the pilot control device 10 described above may be performed by any functional unit. Also, a different functional unit may be added that performs part of the processing by each functional unit described above. Also, the functional units of the pilot control device 10 may be distributed across multiple computers.

[0053] <Disclosures> The present disclosure also includes the following configurations. [Item 1] A system for controlling the flight of an aircraft, a control unit that controls the flying object in response to an operation instruction from a user; In the first operation mode, the control unit is capable of simultaneously accepting a plurality of the operation instructions, In the second operation mode, the control unit is capable of accepting only one of the operation instructions; A flight control system characterized by: [Item 2] Item 1: A flight control system according to item 1, In the second operation mode, the control unit maintains a constant flight speed of the aircraft; A flight control system characterized by: [Item 3] Item 2: The flight control system according to item 2, The flight speed that is maintained constant is slower than the flight speed of the flying object in the first operation mode; A flight control system characterized by: [Item 4] Item 1: A flight control system according to item 1, a status receiving unit that receives a status of the aircraft from the aircraft; a status display unit that displays the status; A flight control system comprising: [Item 5] 1. A method for controlling flight of an air vehicle, comprising: the computer executes a step of controlling the flying object in response to an operation instruction from a user; In the controlling step, the computer In the first operation mode, a plurality of the operation instructions can be simultaneously received; In the second operation mode, only one of the operation instructions can be accepted; A flight control method comprising: [Item 6] A program for controlling the flight of an aircraft, causing a computer to execute a step of controlling the flying object in response to an operation instruction from a user; In the controlling step, the computer In the first operation mode, a plurality of the operation instructions can be simultaneously accepted; In the second operation mode, only one of the operation instructions is acceptable; A program characterized by. [Explanation of symbols]

[0054] 1. User terminal 2 Management Server

Claims

1. A system for controlling the flight of an aircraft, a control unit that controls the flying object in response to an operation instruction from a user; In the first operation mode, the control unit is capable of simultaneously accepting a plurality of the operation instructions, In the second operation mode, the control unit is capable of accepting only one of the operation instructions; A flight control system characterized by:

2. 2. The flight control system of claim 1, In the second operation mode, the control unit maintains a constant flight speed of the aircraft; A flight control system characterized by:

3. 3. The flight control system of claim 2, the flight speed that is maintained constant is slower than the flight speed of the flying object in the first operation mode; A flight control system characterized by:

4. 2. The flight control system of claim 1, a status receiving unit that receives a status of the aircraft from the aircraft; a status display unit that displays the status; A flight control system comprising:

5. 1. A method for controlling flight of an air vehicle, comprising: the computer executes a step of controlling the flying object in response to an operation instruction from a user; In the controlling step, the computer In the first operation mode, a plurality of the operation instructions can be simultaneously accepted; In the second operation mode, only one of the operation instructions can be accepted; A flight control method comprising:

6. A program for controlling the flight of an aircraft, causing a computer to execute a step of controlling the flying object in response to an operation instruction from a user; In the controlling step, the computer In the first operation mode, a plurality of the operation instructions can be simultaneously accepted; In the second operation mode, only one of the operation instructions is acceptable; A program characterized by.

Images