Steering assitance system

Abstract

Provided is a steering assistance system for assisting the steering of an unmanned aircraft by means of an operation device. An operation device 1 comprises: a device state-information transmission unit 14 that transmits position information and orientation information of the operation device to an unmanned aircraft; and an operation information transmission unit 15 that transmits operation information from the operator for the unmanned aircraft to the unmanned aircraft. An unmanned aircraft 2 comprises: a coordinate system management unit 24 that, on the basis of the position information and the orientation information of the operation device and the position information on the unmanned aircraft, calculates an operator coordinate system based on the viewpoint of the operator, and converts the operation information from the operator into operation information for the operator coordinate system; and a flight control unit 25 that controls flight of the unmanned aircraft in accordance with the operation information from the operator which has been converted for the operator coordinate system.

Description

Pilot Assistance System

[0001] The present disclosure relates to piloting assistance systems.

[0002] Various piloting support functions, such as automatic landing and obstacle detection functions, have been developed for UAVs (Unmanned Aerial Vehicles) (Non-Patent Documents 1 and 2). However, the basic operation system is based on a fixed, structurally determined forward direction. Therefore, it is difficult to separate the direction of the structural forward direction from the operation of the aircraft.

[0003] Even multi-rotor UAVs, the most common type of UAV, have a fixed forward orientation, even though the design of the aircraft is structurally meaningless. For example, a four-rotor aircraft has a shape that can be considered roughly 90° rotationally symmetrical, but the aircraft has a set orientation for forward, backward, left, and right. Therefore, it must be controlled based on that orientation, and advanced training is required to understand the aircraft's behavior when rotating.

[0004] Christopher Doer and three others, "Radar Based Autonomous Precision Takeoff and Landing System for VTOLs in GNSS Denied Environments," 2020 International Conference on Unmanned Aircraft Systems (ICUAS), IEEE, September 1-4, 2020, pp. 922-931. "Improved sensor capabilities, dual camera system, and omnidirectional obstacle detection deliver the highest quality video, making this the ultimate camera drone," DJI JAPAN Co., Ltd., PRTIMES, [online], [searched September 21, 2022], Internet<URL : https: / / prtimes.jp / main / html / rd / p / 000000236.000015765.html>

[0005] This section describes the current issues with the remote control of multi-rotor UAVs. Compared to fixed-wing aircraft, where the main wings are fixed to the aircraft body, the shape of the multi-rotor UAV makes it difficult to visually distinguish between front, back, left, and right. Therefore, when the distance from the pilot is great, it is extremely difficult to determine the direction of the aircraft, and the only way to do so is to estimate it from the UAV's behavior. Unless the pilot is an experienced pilot, it is impossible to accurately determine the aircraft's direction while piloting.

[0006] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a technology that enables intuitive control of a UAV.

[0007] One aspect of the piloting assistance system disclosed herein is a piloting assistance system that assists in piloting an unmanned aircraft using an operating device, wherein the operating device comprises a first transmitter that transmits position information and orientation information of the operating device to the unmanned aircraft, and a second transmitter that transmits pilot operation information for the unmanned aircraft to the unmanned aircraft, and the unmanned aircraft comprises a conversion unit that calculates a pilot coordinate system based on the pilot's viewpoint based on the position information and orientation information of the operating device and the position information of the unmanned aircraft, and converts the pilot operation information into operation information in the pilot coordinate system, and a control unit that controls the flight of the unmanned aircraft using the pilot operation information converted into the pilot coordinate system.

[0008] The present disclosure provides technology that allows intuitive control of a UAV.

[0009] Fig. 1 is a diagram showing the configuration of a piloting assistance system, Fig. 2 is a diagram showing the operation flow of an operating device, and Fig. 3 is a diagram showing the operation flow of a multi-rotor UAV.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0011] [Summary] The present disclosure relates to a radio-assisted control technology for UAVs (unmanned aerial vehicles, drones), and aims to make it easier for pilots to intuitively grasp the flight status of a UAV.

[0012] UAVs fly for a variety of purposes, including photography, inspection, pesticide spraying, and transporting goods. These flight purposes require maneuvering the drone to approach or avoid fixed targets on the ground. Regardless of the flight purpose, the ground, buildings, etc. are the targets for takeoff and landing.

[0013] At this time, the pilot's interest is in the relative position and movement of the aircraft to the target object, such as the ground. Therefore, if the pilot's operation can directly control the movement in a coordinate system based on the ground, it will be intuitively easy to understand and the burden on the pilot will be reduced.

[0014] On the other hand, piloting is usually done by specifically manipulating the controller to indicate how you want the drone to move relative to the ground or a target on the ground. However, the orientation of the drone is important in piloting, and the pilot must determine how the drone will behave relative to the ground based on the drone's controls.

[0015] However, inexperienced pilots often do not understand the relationship between operations and behavior in the first place. Not being able to grasp the direction of the aircraft can lead to incorrect operations. In extreme cases, they may operate the aircraft in the exact opposite direction of their intended direction.

[0016] Therefore, in the present disclosure, the movement of the UAV is converted into movement in the pilot's coordinate system based on the pilot's orientation. This allows the pilot to control the aircraft based on the orientation as seen by the pilot, without having to constantly keep track of the aircraft's orientation. As a result, it becomes easier for the pilot to intuitively understand the flight status of the UAV.

[0017] Hereinafter, a coordinate system fixed with the pilot as the origin is assumed. This coordinate system will be hereinafter referred to as the pilot coordinate system. Furthermore, a coordinate system fixed with respect to a specific target will be similarly referred to as the target coordinate system, and in particular, a coordinate system fixed on the Earth that can be acquired by a GNSS (Global Navigation Satellite System) sensor will be referred to as the GNSS coordinate system.

[0018] [Simple Configuration Example] The simplest configuration will be described.

[0019] The pilot coordinate system can be calculated based on the control device. The position and movement of the pilot (control device) are detected by a sensor equipped on the control device as the movement of the reference point of the pilot coordinate system. The detected position information and movement information of the pilot are constantly notified to the UAV.

[0020] In a UAV, the current position, inclination, and motion state of the aircraft are acquired. Coordinate transformation is performed based on the position information and motion information of the reference point in the pilot's coordinate system notified by the control device. This coordinate transformation converts the aircraft's position information, etc., into position and motion in the pilot's coordinate system. In addition, the pilot's control operations from the control device are considered to be control operations in the pilot's coordinate system, and the pilot's control operations are also subjected to coordinate transformation. Flight control commands based on the pilot's control operations after coordinate transformation are then output to the flight mechanism that manages the actual power output and steering amount.

[0021] [Piloting Assistance System] Figure 1 is a diagram showing the configuration of a piloting assistance system according to this embodiment. The piloting assistance system is a system that assists piloting of an unmanned aircraft using an operation device, and includes an operation device 1 that a pilot uses to wirelessly operate a multi-rotor UAV 2, and the multi-rotor UAV 2 whose flight is controlled by operating the operation device 1.

[0022] (Operation Device) The operation device 1 includes a GNSS sensor 11, an acceleration sensor 12, a device status management unit 13, a device status information transmission unit 14, and an operation information transmission unit 15, in addition to existing mechanisms such as a control lever and an antenna.

[0023] The GNSS sensor 11 is a sensor that measures the position of the operation device 1 .

[0024] The acceleration sensor 12 is a sensor that detects the movement of the controller device 1. The movement includes accelerations in three orthogonal directions, the direction of movement of the controller device 1 determined from the magnitude of each acceleration, and the orientation of the controller device 1.

[0025] The device state management unit 13 has a function of managing the position information and movement information (acceleration information, movement direction information, orientation information, etc.) of the controller device 1 as the device state.

[0026] The device status information transmission unit (first transmission unit) 14 has a function of wirelessly transmitting device status information (position information, movement information) of the control device 1 to the multi-rotor UAV 2.

[0027] The operation information transmitter (second transmitter) 15 has the function of wirelessly transmitting the pilot's operation information (turning information, etc.) for the multi-rotor UAV 2 to the multi-rotor UAV 2.

[0028] (Multi-rotor UAV) The multi-rotor UAV 2 includes a flight mechanism unit 21 , a flight state management unit 22 , an operation unit 23 , a coordinate system management unit 24 , and a flight control unit 25 .

[0029] The flight mechanism unit 21 includes an engine, a motor, a rudder, and the like mounted on the aircraft.

[0030] The flight status management unit 22 has a sensor information management function that acquires and manages various sensor information from various sensors built into the aircraft, and a flight status management function that manages the aircraft's flight status (position, inclination, movement status, etc.) based on the various sensor information.

[0031] The various sensors include a GNSS sensor, an acceleration sensor, an IMU (Inertial Measurement Unit) sensor, a motor sensor, a base temperature sensor, a wind speed sensor, a LiDAR (Light Detection and Ranging) sensor, and the like, which are built into the aircraft.

[0032] The operation unit 23 has a device status information receiving function that receives device status information (position information, movement information) of the operation device 1 from the operation device 1, and an operation information receiving function that receives operator operation information (turning information, etc.) from the operation device 1.

[0033] The coordinate system management unit (conversion unit) 24 has a function of calculating a pilot coordinate system using device status information (position information, movement information) of the controller device 1. Specifically, the coordinate system management unit 24 calculates a pilot coordinate system based on the orientation of the controller device 1, i.e., the pilot's viewpoint, from the positional relationship between the controller device 1 and the player's aircraft and the orientation of the controller device 1.

[0034] The coordinate system management unit (conversion unit) 24 regards the pilot's control operations as control operations in the pilot coordinate system, and has the function of converting the pilot's control information (turning information, etc.) and the aircraft's flight status information (position, tilt, motion status, etc.) into information in the pilot coordinate system.

[0035] The flight control unit 25 has a function to control the flight of the multi-rotor UAV 2 using the pilot's operation information and the aircraft's flight status information converted into the pilot's coordinate system. Specifically, the flight control unit 25 generates flight control commands based on the pilot's control operations after the coordinate system conversion, and outputs the flight control commands to the flight mechanism unit 21, which is responsible for motor and engine output control and rudder control.

[0036] [Operation of the Operation Device] FIG. 2 is a diagram showing the operation flow of the operation device 1. As shown in FIG.

[0037] The control device 1 detects the position and movement of the control device 1, and transmits position information and movement information (orientation information, etc.) of the control device 1 to the multi-rotor UAV 2 each time the position and movement of the control device 1 are detected (step S101).

[0038] When the operator operates the operation device 1, the operation device 1 transmits the operator's operation information to the multi-rotor UAV 2 each time the operator operates the operation device 1 (step S102).

[0039] [Operation of Multi-Rotor UAV] FIG. 3 is a diagram showing the operation flow of the multi-rotor UAV 2.

[0040] The operation unit 23 receives the position information and movement information (orientation information, etc.) of the operation device 1 transmitted from the operation device 1 (step S201).

[0041] The flight state management unit 22 acquires various sensor information from various sensors built into the aircraft, and manages the current flight state of the aircraft based on the acquired various sensor information (step S202).

[0042] The operation unit 23 receives the pilot's operation information transmitted from the operation device 1, adds the current flight state information managed in step S202 to the operation information, and transmits the operation information and the current flight state information to the coordinate system management unit 24 (step S203).

[0043] The coordinate system management unit 24 calculates and sets a pilot coordinate system based on the orientation of the control device 1, i.e., the pilot's viewpoint, from the positional relationship between the control device 1 and the aircraft itself and orientation information of the control device 1, and converts the movement of the multi-rotor UAV 2 into movement in the pilot coordinate system (step S204).

[0044] For example, the pilot's operation information transmitted from the control device 1 is interpreted as an operation command from the pilot's viewpoint, converted into operation information in the pilot's coordinate system, and incorporated into the airplane coordinate system as the orientation and movement of the pilot's coordinate system. Current flight status information is also converted into information in the pilot's coordinate system.

[0045] The flight control unit 25 generates a flight control command in the pilot coordinate system corresponding to the pilot's operation information based on the pilot's operation information after the coordinate system conversion and the current flight state information, and outputs the flight control command to the flight mechanism unit 21 (step S205). As a result, the multi-rotor UAV 2 is controlled based on the direction seen by the pilot.

[0046] The operation unit 23 notifies the operation device 1 of the current flight state (step S206).

[0047] [Effects] According to this embodiment, the multi-rotor UAV 2 calculates a pilot coordinate system based on the pilot's viewpoint based on the position information and movement information (orientation information, etc.) of the control device 1 and the position information of the multi-rotor UAV 2, converts the pilot's operation information into operation information in the pilot coordinate system, and controls the flight of the multi-rotor UAV 2 using the pilot's operation information converted into the pilot coordinate system, so the pilot can control the aircraft based on the orientation as seen by the pilot without having to keep track of the orientation of the aircraft. As a result, it is easier for the pilot to intuitively understand the flight status of the UAV.

[0048] [Others] In this embodiment, a multi-rotor UAV has been described as an example, but the present invention can be applied to various UAVs, such as fixed-wing aircraft.

[0049] REFERENCE SIGNS LIST 1 Operation device 11 GNSS sensor 12 Acceleration sensor 13 Device status management unit 14 Device status information transmission unit 15 Operation information transmission unit 2 Multi-rotor UAV 21 Flight mechanism unit 22 Flight status management unit 23 Operation unit 24 Coordinate system management unit 25 Flight control unit

Claims

1. A piloting assistance system that assists in piloting an unmanned aircraft using an operating device, the operating device comprising: a first transmitting unit that transmits position information and orientation information of the operating device to the unmanned aircraft; and a second transmitting unit that transmits pilot operation information for the unmanned aircraft to the unmanned aircraft, the unmanned aircraft comprising: a conversion unit that calculates a pilot coordinate system based on the pilot's viewpoint based on the position information and orientation information of the operating device and the position information of the unmanned aircraft and converts the pilot's operation information into operation information in the pilot coordinate system; and a control unit that controls the flight of the unmanned aircraft using the pilot's operation information converted into the pilot coordinate system.

2. The piloting assistance system according to claim 1, wherein the unmanned aircraft is a multi-rotor unmanned aircraft.

Images