Unmanned aerial vehicle control system and control device

Abstract

To provide technology that enables an unmanned aerial vehicle to reliably follow a vehicle even when the distance between the unmanned aerial vehicle and the vehicle is large, on a course with many obstacles and a complex shape. [Solution] The unmanned aerial vehicle control system controls the unmanned aerial vehicle and the unmanned aerial vehicle to fly in accordance with the target vehicle. When predetermined conditions are met, the unmanned aerial vehicle control system controls the unmanned aerial vehicle based on a trajectory-following mode and sets a route that reproduces the driving trajectory of the target vehicle as the flight route of the unmanned aerial vehicle in the trajectory-following mode. The predetermined conditions include at least the distance between the unmanned aerial vehicle and the target vehicle being greater than or equal to a threshold.

Description

[[TECHNICAL FIELD]]

[0001] The present disclosure relates to an unmanned aerial vehicle control system that controls an unmanned aerial vehicle following a vehicle, and to an in-vehicle control device that controls an unmanned aerial vehicle. [[BACKGROUND ART]]

[0002] Patent Document 1 discloses a system for controlling a drone to photograph a traveling vehicle from various angles. This system is characterized by predicting the future traveling position of the vehicle and calculating a flight path for the drone that passes through a preset relative position with respect to the vehicle in accordance with each situation. [[PRIOR ART DOCUMENTS]] [[PATENT DOCUMENTS]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2021-110692 [[SUMMARY OF THE INVENTION]] [[PROBLEM TO BE SOLVED BY THE INVENTION]]

[0004] In specific environments (for example, forests with many obstacles such as trees and many curves), the control method based on the future traveling position of the vehicle as in Patent Document 1 may not function properly in some cases. In particular, when the distance between the drone and the vehicle is large, flying the drone straight toward the position of the vehicle is the shortest route for following the vehicle; however, if there are many obstacles on the route, it will involve complicated avoidance behavior to avoid those obstacles.

[0005] One object of the present disclosure is to provide a technology that allows an unmanned aerial vehicle to reliably follow a vehicle even when the distance between the unmanned aerial vehicle and the vehicle is large on a course with many obstacles and a complex shape. [[MEANS FOR SOLVING THE PROBLEM]]

[0006] A first aspect relates to an unmanned aerial vehicle control system. The unmanned aerial vehicle control system comprises an unmanned aerial vehicle and one or more processors that control the unmanned aerial vehicle to fly in accordance with a target vehicle. One or more processors, When certain conditions are met, the unmanned aerial vehicle is controlled based on the trajectory tracking mode. In trajectory-following mode, the flight path of the unmanned aerial vehicle is set to a route that replicates the driving trajectory of the target vehicle. It is configured in this way. The specified conditions include, at a minimum, that the distance between the unmanned aerial vehicle and the target vehicle is greater than or equal to a threshold.

[0007] The second aspect relates to the control devices installed in the vehicle. The control system comprises one or more processors that control the unmanned aerial vehicle to fly in accordance with the vehicle. One or more processors, When certain conditions are met, the unmanned aerial vehicle is controlled based on the trajectory tracking mode. In trajectory-following mode, the flight path of the unmanned aerial vehicle is configured to reproduce the vehicle's travel trajectory. The prescribed conditions include, at a minimum, that the distance between the unmanned aerial vehicle and the vehicle is greater than or equal to a threshold. [Effects of the Invention]

[0008] According to the technology disclosed herein, when certain conditions are met, the unmanned aerial vehicle (UAV) is controlled according to a trajectory-following mode. Specifically, in trajectory-following mode, the UAV is controlled to fly along a route that reproduces the trajectory of the target vehicle (i.e., to trace the trajectory). Since it is unlikely that there will be obstacles for the UAV to fly in an area that the target vehicle has already passed through, a trajectory-following mode that controls the UAV to fly along the trajectory of the target vehicle is effective in courses with many obstacles and complex shapes. [Brief explanation of the drawing]

[0009] [Figure 1]This is a schematic diagram of a drone control system. [Figure 2] This is a schematic diagram of the trajectory tracking mode. [Figure 3] This is a schematic diagram illustrating an example of switching to trajectory tracking mode in response to a communication failure. [Figure 4] This flowchart shows a series of processes, including the mode switching process. [Figure 5] This is a block diagram showing an example configuration of a drone control system. [Modes for carrying out the invention]

[0010] Embodiments of this disclosure will be described with reference to the attached drawings. This embodiment focuses on a drone 20 as a typical example of an unmanned aerial vehicle. Accordingly, the unmanned aerial vehicle control system will be described as the drone control system 1. The concept of an unmanned aerial vehicle includes not only drones but also unmanned airplanes and unmanned helicopters. The drone 20 is configured to be capable of autonomous flight.

[0011] 1. Drone control system Figure 1 is a schematic diagram of the drone control system 1. The drone 20 flies in pursuit of the target vehicle 10. When the distance between the target vehicle 10 and the drone 20 increases and the drone 20 attempts to catch up to the target vehicle 10, the shortest theoretical route is to fly straight towards the current position of the target vehicle 10, as shown in route R1. However, with such control, if the target vehicle 10 is traveling in an environment such as a mountain road with many obstacles such as trees and rocks and many curves, the risk of the drone 20 colliding with obstacles increases. Alternatively, the drone 20 would frequently need to perform evasive maneuvers to avoid obstacles, resulting in complex flight behavior. Therefore, such control may actually increase the time it takes for the drone 20 to catch up to the target vehicle 10. Furthermore, in order to reliably execute such complex behaviors, a stable communication environment is also necessary to prevent control delays.

[0012] In such cases, route R2, which directly tracks the trajectory of the target vehicle 10, is a more reliable method. Since there are basically no obstacles along the path taken by the target vehicle 10, unnecessary evasive maneuvers are suppressed. This control mode, which directly tracks the trajectory of the target vehicle 10, is called the "trajectory tracking mode." The drone control system 1 can reliably track the target vehicle 10 even in environments with many obstacles and curves by switching between the trajectory tracking mode and a different normal mode when controlling the drone 20. The normal mode is the conventional control mode, and a typical example is a mode that controls the drone to fly at a specific relative position to the target vehicle 10 (for example, 5m behind and 2m above).

[0013] 2. Tracking mode Figure 2 is a schematic diagram of the trajectory-following mode. In trajectory-following mode, the drone control system 1 sets a flight route for the drone 20 that reproduces the driving trajectory 15 of the target vehicle 10. More specifically, the drone control system 1 acquires the vehicle position information VPO of the target vehicle 10 and sets a flight route based on the acquired vehicle position information VPO. Flight route generation may be performed by the target vehicle 10, or by the drone 20 that has acquired the vehicle position information VPO. It may also be performed by an external server that can communicate with the target vehicle 10 or the drone 20. The flight route is absolute or relative coordinate information that indicates the path the drone 20 will fly. In trajectory-following mode, the flight route flies so as to faithfully trace the driving trajectory 15 of the target vehicle 10. Regarding the position of the target vehicle 10, following both the horizontal and vertical positions is suitable as a route to avoid obstacles. The position of the target vehicle 10 is, for example, the center position of the target vehicle 10.

[0014] The drone control system 1 calculates a flight route using vehicle position information VPO. These pieces of information are associated with times. That is, as shown in the drawing, the drone control system 1 acquires the position (travel trajectory 15) of the target vehicle 10 at each time. The drone control system 1 reproduces the travel trajectory 15 as a flight route in the trajectory following mode.

[0015] The drone control system 1 may use drone position information DPO indicating the position of the drone 20 when calculating the flight route. In this case, the drone control system 1 refers to the current position of the drone 20 to calculate how far the drone is to be moved from the current position. The drone position information DPO is acquired by the drone 20 using a satellite system or the like, and may be shared via communication with the target vehicle 10 or an external server in the same manner as the vehicle position information VPO. Alternatively, the drone 20 may receive flight route information FR, which is information related to a flight route calculated using the vehicle position information VPO or the drone position information DPO.

[0016] 2-1. Conditions for transitioning to trajectory following mode The drone control system 1 executes "mode switching processing" for switching between a normal mode and a trajectory following mode. In principle, the drone control system 1 switches the control mode from the normal mode to the trajectory following mode when a distance d between the target vehicle 10 and the drone 20 is equal to or greater than a threshold value. A long distance d means that obstacles or curves that hinder the drone from catching up in a straight line are likely to appear between the target vehicle 10 and the drone 20. Therefore, switching to the trajectory following mode, which sets a flight route with fewer obstacles when the distance d increases, is reasonable. A method for measuring the distance d will be described in Section 3 together with the configurations of the target vehicle 10 and the drone 20.

[0017] The condition for transitioning to the trajectory following mode may include that a failure occurs in communication for the drone 20 to acquire vehicle position information VPO. The communication failure referred to herein includes communication interruption or communication delay. Since the occurrence of a communication failure means that tracking using real-time vehicle position information VPO or drone position information DPO is difficult, the distance d tends to increase as a result. That is, it can be said that a communication failure is a precursor to an increase in the distance d. In such a case, it is reasonable to transition to the trajectory following mode in advance and adopt a more reliable flight route. Figure 3 is a schematic diagram showing an example of transitioning to the trajectory following mode in response to a communication failure. Since the drone 20 cannot acquire real-time information while a communication failure is occurring, it is preferable that the drone 20 retains flight route information FR and vehicle position information VPO in the storage device 252 at least temporarily in a normal state (a state where communication is possible). In other words, the drone control system 1 can control the drone 20 even while the communication failure is occurring by using each piece of information acquired by the drone 20 before the communication failure occurs.

[0018] 2-2. Flow of mode switching process Figure 4 is a flowchart showing a series of processes including the mode switching process.

[0019] In step S10, the drone control system 1 controls the drone 20 in the normal mode. Thereafter, the process proceeds to step S20.

[0020] In step S20, the drone control system 1 determines whether a predetermined condition is satisfied. As described above, the predetermined condition includes that the distance d between the target vehicle 10 and the drone 20 becomes equal to or greater than a threshold value, and that a communication failure occurs. If the predetermined condition is satisfied (step S20; Yes), the process proceeds to step S30. If the predetermined condition is not satisfied (step S20; No), the process returns to step S10.

[0021] In step S30, the drone control system 1 controls the drone 20 in trajectory-following mode. Then, the process returns to step S20. In other words, step S20 means that if the predetermined conditions are no longer met even after transitioning to trajectory-following mode, the system will transition to normal mode according to S10. In this way, normal mode and trajectory-following mode can be used interchangeably depending on the situation.

[0022] 3. Example Configuration Figure 5 is a block diagram showing an example configuration of the drone control system 1.

[0023] 3-1. Target Vehicles The target vehicle 10 is equipped with a communication device 110, a sensor group 120, a running device 130, an output device 140, and a control device 150.

[0024] The communication device 110 transmits and receives information necessary for mode switching processing by communicating with external devices. For example, the communication device 110 exchanges information with the drone 20 via wireless communication. Various wireless communication standards can be used in the drone control system 1. Therefore, the optimal communication standard can be selected according to the situation from the perspective of communication distance, communication speed, power consumption, cost, etc. The communication device 110 also receives radio waves from satellites and base stations for self-positioning. It also receives signals from satellites and base stations for self-position measurement. For example, the communication device 110 uses the Global Navigation Satellite System (GNSS) for self-position measurement. The GNSS used in the drone control system 1 may be a relative positioning method in addition to the general standalone positioning method. One of the relative positioning measurement methods is known as Real Time Kinematic (RTK) positioning. RTK positioning involves receiving signals with two receivers: a base station whose location is known and a mobile station to be positioned. The base station's location information is then wirelessly transmitted to the mobile station. The mobile station corrects for errors based on the base station's location information, thereby improving accuracy and obtaining more precise position information than single-point positioning. While single-point positioning has a measurement error of several meters, RTK positioning can reduce the measurement error to several centimeters. In trajectory tracking mode, it is desirable for the drone 20 to accurately follow the driving trajectory 15 of the target vehicle 10; therefore, RTK positioning is preferred for positioning in trajectory tracking mode.

[0025] The sensor group 120 includes recognition sensors, vehicle condition sensors, etc. The recognition sensors recognize (detect) the surrounding conditions of the target vehicle 10. Examples of recognition sensors include on-board cameras, LIDAR (Light Detection and Ranging), radar, etc. The vehicle condition sensors detect the state of the target vehicle 10. The vehicle condition sensors include speed sensors, acceleration sensors, yaw rate sensors, steering angle sensors, etc.

[0026] The running gear 130 includes a steering gear, a drive gear, and a braking gear. The steering gear steers the wheels. For example, the steering gear includes an electric power steering (EPS) system. The drive gear is a power source that generates driving force. Examples of drive gears include an engine, an electric motor, an in-wheel motor, etc. The braking gear generates braking force.

[0027] The output device 140 outputs various information. For example, the output device 140 includes a display device. The display device presents various information to the driver of the target vehicle 10 by displaying the information. As another example, the output device 140 may include a speaker. When the drone control system 1 transitions to trajectory-following mode, it may provide a notification to the driver via the output device 140 to inform them of this. This allows the driver to know that the target vehicle 10 and the drone 20 are a considerable distance apart, and to take action such as stopping or slowing down the target vehicle 10 until the drone 20 is close enough to the target vehicle 10.

[0028] The control device 150 includes one or more processors 151 (hereinafter simply referred to as "processor 151") and a storage device 152. The processors 151 perform mode switching processing. Examples of processors 151 include general-purpose processors, application-specific processors, CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), integrated circuits, conventional circuits, and / or combinations thereof. The processors 151 may also be referred to as circuitry or processing circuitry.

[0029] The storage device 152 stores various types of information. Examples of storage devices 152 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc. The functions of the control device 150 are realized through the cooperation of the processor 151 and the storage device 152. The storage device 152 records the drone control program PROG. The functions of the drone control system 1 are realized when the processor 151 executes the drone control program PROG. The drone control program PROG may be recorded on a computer-readable recording medium. The storage device 152 also records vehicle position information VPO.

[0030] 3-2. Drones The drone 20 includes a communication device 210, a sensor group 220, a flight device 230, and a control device 250.

[0031] The communication device 210 transmits and receives information necessary for mode switching by communicating with external devices. For example, the communication device 210 exchanges information with the target vehicle 10 via wireless communication. The communication device 210 also receives radio waves from satellites and base stations for self-positioning. The communication device 210 uses GNSS for self-positioning. Similar to the target vehicle 10, the positioning of the drone 20 in trajectory tracking mode is preferably done by RTK positioning.

[0032] The sensor group 220 includes attitude control sensors, distance measuring sensors, geomagnetic sensors, etc. Attitude control sensors are necessary for controlling the attitude of the drone 20 and include angular velocity sensors and acceleration sensors. Distance measuring sensors include vision sensors built into the camera, ultrasonic sensors, LIDAR, etc., and are used to measure the distance d to the target vehicle 10. In other words, the drone control system 1 determines whether the conditions for mode switching processing are met based on the distance d detected by the distance measuring sensors. Distance measuring sensors are also used to measure the distance between the drone 20 and the ground surface and are used for altitude control of the drone 20. The geomagnetic sensor detects the direction in which the drone 20 is facing.

[0033] The flight device 230 is a power source (e.g., an electric motor) that generates lift. The flight device 230 is connected to a propeller, and lift is generated by rotating the propeller. By attaching a flight device 230 to each propeller, it becomes possible for each propeller to exhibit different rotational behavior. This enables fine attitude control and flight control, such as hovering, turning, ascending / descending, and lateral movement.

[0034] The control device 250 includes one or more processors 251 (hereinafter simply referred to as "processor 251") and a storage device 252. The processors 251 perform mode switching processing. Examples of processors 251 include general-purpose processors, application-specific processors, CPUs, GPUs, ASICs, FPGAs, integrated circuits, conventional circuits, and / or combinations thereof. The processors 251 may also be referred to as circuitry or processing circuitry.

[0035] The storage device 252 stores various types of information. Examples of storage devices 252 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc. The functions of the control device 250 are realized through the cooperation of the processor 251 and the storage device 252. The storage device 252 includes drone position information DPO and flight route information FR.

[0036] 3-3. Others The target vehicle 10 and the drone 20 may communicate via an external server. Specifically, the communication device 110 and the communication device 210 may communicate with each other via an external server. Furthermore, some or all of the processing related to the drone control system 1 may be executed by the external server. For example, the external server may obtain vehicle position information VPO from the target vehicle 10 and drone position information DPO from the drone 20, and then create a flight route. The external server may further transmit the created flight route as flight route information FR to the drone 20. [Explanation of Symbols]

[0037] 1: Drone control system, 10: Target vehicle, 15: Driving trajectory, 20: Drone, 110: Communication device, 120: Sensor group, 130: Driving device, 140: Output device, 150: Control device, 210: Communication device, 220: Sensor group, 230: Flight device, 250: Control device, DPO: Drone position information, FR: Flight route information, VPO: Vehicle position information

Claims

1. Unmanned aerial vehicles and One or more processors that control the aforementioned unmanned aerial vehicle to fly in pursuit of a target vehicle. Equipped with, The one or more processors described above are: When predetermined conditions are met, the unmanned aerial vehicle is controlled based on the trajectory tracking mode. In the trajectory-following mode, the flight route of the unmanned aerial vehicle is set to a route that reproduces the driving trajectory of the target vehicle. It is configured in such a way, The aforementioned predetermined conditions include, at a minimum, that the distance between the unmanned aerial vehicle and the target vehicle is greater than or equal to a threshold. Unmanned aerial vehicle control system.

2. An unmanned aerial vehicle control system according to claim 1, The aforementioned unmanned aerial vehicle is equipped with a communication device and a memory device, The communication device acquires vehicle location information indicating the location of the target vehicle via communication, The storage device holds the acquired vehicle location information at least temporarily, The aforementioned predetermined conditions further include the occurrence of a failure in the communication for acquiring the vehicle location information, In the trajectory tracking mode, the one or more processors are configured to reproduce the driving trajectory of the target vehicle based on the vehicle position information held in the storage device. Unmanned aerial vehicle control system.

3. An unmanned aerial vehicle control system according to claim 1 or 2, In the trajectory-following mode, the one or more processors notify the driver of the target vehicle that the unmanned aerial vehicle is being controlled based on the trajectory-following mode. It is configured to Unmanned aerial vehicle control system.

4. A control device installed in a vehicle, The system includes one or more processors that control an unmanned aerial vehicle to fly in accordance with the vehicle, The one or more processors described above are: When predetermined conditions are met, the unmanned aerial vehicle is controlled based on the trajectory tracking mode. In the aforementioned trajectory-following mode, the flight route of the unmanned aerial vehicle is set to a route that reproduces the vehicle's travel trajectory. It is configured in such a way, The aforementioned predetermined conditions include, at a minimum, that the distance between the unmanned aerial vehicle and the vehicle is greater than or equal to a threshold. Control device.

Images