Quad-rotor unmanned aerial vehicle perception and control method and system under complex visual condition

Abstract

The invention discloses a four-rotor unmanned aerial vehicle perception and control method and system under a complex visual condition, and belongs to the technical field of unmanned aerial vehicle control. According to the invention, by combining the KCF algorithm and the YOLO algorithm, more accurate and more robust target tracking is realized, and the defects of insufficient anti-interference performance of the traditional KCF algorithm in a complex environment and the limitation of the YOLO algorithm are overcome, so that the real-time sensing ability of the unmanned aerial vehicle to the target is improved; a robust PID control algorithm is used as a control part, accurate control over the speed and the position of the unmanned aerial vehicle is achieved, the stability and the accuracy of flight control are optimized, meanwhile, the calculation complexity of the system is reduced, and the real-time performance and the applicability of the system are improved; the method is not only suitable for the field of unmanned aerial vehicles, but also can be expanded to other application scenes requiring target tracking and accurate control.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to a method and system for perceiving and controlling a quad-rotor unmanned aerial vehicle under complex visual conditions. Background Art

[0002] Currently, drone technology is widely used in various fields, but there are still some challenges in its autonomous perception and control system. Traditional drone control systems mainly rely on inertial navigation systems and global positioning systems (GPS), which can provide position and attitude information, but their accuracy and robustness are limited in complex environments, such as urban areas or dense woodlands.

[0003] In response to these challenges, some solutions have emerged in the existing technology. Among them, the KCF (Kernelized Correlation Filters) algorithm is widely used in target tracking of UAVs, which can achieve real-time tracking of moving objects through visual information. On the other hand, the YOLO (You Only Look Once) algorithm is an efficient target detection algorithm that can quickly and accurately detect multiple targets in an image. In addition, PID controller, as a classic control algorithm, has been widely used in the flight control of UAVs. Although the MPC (Model Predictive Control) advanced control algorithm for UAVs can provide precise flight control, it has certain difficulties in practical applications due to its high computational complexity and high requirements for embedded devices.

[0004] The above-mentioned existing technologies have the following problems: (1) The accuracy of traditional control systems is limited: Traditional UAV control systems mainly rely on inertial navigation and global positioning systems (GPS), but in complex environments, such as urban areas or dense woodlands, the accuracy and robustness are limited. (2) The KCF algorithm has poor anti-interference ability: Although the KCF algorithm is used for target tracking, it has poor anti-interference ability in complex environments and is easily affected by changes in lighting and background interference, resulting in insufficient target tracking accuracy. (3) The MPC algorithm has high computational complexity: The traditional MPC algorithm can provide precise flight control, but its computational complexity is high and the requirements for onboard computers are high, which poses certain difficulties in practical applications.

[0005] In order to solve the above problems, the present invention proposes a method and system for perceiving and controlling a quadrotor drone under complex visual conditions. Summary of the invention

[0006] The technical problem to be solved by the present invention is: how to solve the problems existing in the above-mentioned prior art, and provide a method for perceiving and controlling a quad-rotor drone under complex visual conditions.

[0007] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0008] S1: Target detection and tracking preparation

[0009] Set the type of tracking target according to detection needs;

[0010] S2: Target Tracking

[0011] Use KCF-Y algorithm to perform environment perception and target tracking to obtain target location information;

[0012] S3: Control information generation

[0013] When the target is identified and tracked, the target position information is transmitted to the control system to obtain the position of the center of the target frame in the target position information on the screen and the target frame size information. A robust PID controller is used to generate the speed and position control instructions of the UAV according to the position of the center of the target frame in the target position information on the screen and the deviation of the target frame size relative to the center point of the field of view, so as to achieve stable tracking of the target.

[0014] Furthermore, in step S2, the KCF-Y algorithm is a fusion algorithm of the KCF algorithm and the YOLOv8s algorithm, and the formula is as follows:

[0015] TrackedObject=KCF(YOLO(IFrame),SFrame)

[0016] Among them, IFrame represents the initial frame, SFrame represents the subsequent frame, YOLO represents the processing of the initial frame by the YOLOv8s algorithm, and KCF represents the processing of the subsequent frame by the KCF algorithm.

[0017] Furthermore, in step S2, the specific processing process is as follows:

[0018] S21: In the first frame of detection, the target is identified using the YOLOv8s algorithm, and the identified target frame is passed to the KCF algorithm for target tracking;

[0019] S22: Starting from the second frame, tracking is performed using the KCF algorithm to obtain target position information.

[0020] Furthermore, in step S22, the target position information includes the position of the center of the target frame on the screen and the size information of the target frame.

[0021] Furthermore, in step S3, in order to buffer the robust PID control, a buffer area is set, that is, the UAV will not generate speed and position control instructions when the target is within the range set by the center point of the UAV. The specific expression is as follows:

[0022] e(t)=Ptarget(t)-Pcenter

[0023] |e(t)|≤ε

[0024] Among them, Ptarget(t) represents the target center point position, Pcenter is the center position of the camera field of view, and ε is the pre-defined error limit.

[0025] Furthermore, in step S3, the robust PID controller expression is as follows:

[0026]

[0027] Among them, u(t) is the output signal of the controller, that is, the required control quantity; e(t) is the error signal of the system, defined as the difference between the expected value and the actual value; K p , K i , K d are proportional, integral and differential gains, which control the response speed, steady-state error and anti-interference ability of the robust PID controller respectively; F robust is a robustness enhancement term; τ is a time variable that reflects the cumulative effect of deviation over time.

[0028] Furthermore, the method for sensing and controlling a quadrotor drone under complex visual conditions further includes the following steps:

[0029] S4: Control feedback and adjustment

[0030] The relative position and speed between the UAV and the target are monitored according to the actual flight situation to obtain feedback information, and the above information is used to evaluate the effect of the robust PID controller and identify any deviations or unstable flight behaviors.

[0031] The present invention also provides a perception and control system for a quad-rotor drone under complex visual conditions, which uses the above method to control a quad-rotor drone, including a visual system and a control system;

[0032] The visual system is used to perceive the environment and track targets using the KCF-Y algorithm, capture targets of interest and obtain their location information;

[0033] A control system is used to generate corresponding speed control instructions and position control instructions using a robust PID controller.

[0034] Compared with the prior art, the present invention has the following advantages: the perception and control method of the quad-rotor unmanned aerial vehicle under complex visual conditions combines the KCF algorithm with the YOLO algorithm to achieve more accurate and robust target tracking, overcomes the insufficient anti-interference ability of the traditional KCF algorithm in complex environments and the limitations of the YOLO algorithm, thereby improving the real-time perception capability of the unmanned aerial vehicle to the target; the robust PID control algorithm is applied as the control part to achieve precise control of the attitude and position of the unmanned aerial vehicle, optimize the stability and accuracy of the flight control, and at the same time reduce the computational complexity of the system, and improve the real-time and applicability of the system; it is not only applicable to the field of unmanned aerial vehicles, but also can be extended to other application scenarios that require target tracking and precise control, such as intelligent monitoring, automatic driving, etc., expanding the application scope and potential market of the technology; through the innovative solution of the visual part, the real-time perception capability of the unmanned aerial vehicle to the target is improved, thereby realizing the rapid tracking and identification of moving objects in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of the structure of the quad-rotor drone perception and control system under complex visual conditions in an embodiment of the present invention;

[0036] Figure 2 It is a pseudo code example diagram of the KCF-Y algorithm in an embodiment of the present invention;

[0037] Figure 3 It is a workflow diagram of the perception and control system of a quadrotor drone under complex visual conditions in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0039] like Figure 1 As shown, this embodiment provides a technical solution: a four-rotor drone perception and control system under complex visual conditions, including a visual system and a control system, the visual system is used to identify environmental information, capture targets of interest and obtain their position information, and the control system is used to generate corresponding speed control instructions and position control instructions.

[0040] In this embodiment, the visual perception system adopts the KCF-Y algorithm, which is a fusion algorithm of the KCF algorithm and the YOLOv8s algorithm. The algorithm principle is as follows: Figure 2 The pseudo code shown is given, which can be expressed as follows using the formula:

[0041] TrackedObject=KCF(YOLO(IFrame),SFrame) (5-1)

[0042] Among them, IFrame represents the initial frame, SFrame represents the subsequent frame, YOLO represents the processing of the initial frame with the YOLOv8s algorithm, and KCF represents the processing of the subsequent frame with the KCF algorithm. This algorithm is responsible for environmental perception, target detection, tracking and re-detection in the perception and control system of quadrotor drones. The fused algorithm has greatly improved the detection speed, detection accuracy and anti-interference performance compared with the original algorithm. Target tracking can achieve specific target tracking, thanks to the recognition of the YOLO algorithm. Without any training and modification, the drone can be set to recognize 80 different types of targets. If there are specific detection requirements, custom target training can be performed, and the recognition accuracy can reach above 0.9, and the training time is very short. The KCF-Y algorithm can be performed without pre-training. This is due to the circulant matrix in the principle of the algorithm. Samples of the target area can be automatically generated for learning. The algorithm formula is shown in (5-2):

[0043]

[0044] The speed of the KCF algorithm is due to the use of fast Fourier transform, as shown in formula (5-3):

[0045] F(x)=FFT(x) (5-3)

[0046] Among them, F(x) represents the representation of x in the frequency domain, and FFT represents the fast Fourier transform processing, which accelerates the operation of the KCF algorithm so that it can quickly train and update the classifier.

[0047] In the control system, in order to buffer the robust PID control, a buffer area is set, that is, the drone will not generate speed and position control instructions when the target is within a certain range of the drone center point. This can effectively reduce the oscillation of the drone when it approaches the desired position, as shown in formulas (5-4) and (5-5):

[0048] e(t)=Ptarget(t)-Pcenter (5-4)

[0049] |e(t)|≤ε (5-5)

[0050] Among them, Ptarget(t) represents the target center point position, Pcenter is the center position of the camera field of view, and ε is the pre-defined error limit.

[0051] According to the target position information obtained by the UAV, the display coordinates of the target are obtained through coordinate conversion, and the corresponding position and speed control instructions are generated in combination with the robust PID controller. The robust PID controller can be shown by formula (5-6):

[0052]

[0053] Among them, u(t) is the output signal of the controller, that is, the required control quantity; e(t) is the error signal of the system, which is usually defined as the difference between the expected value and the actual value; K p , K i , K d are proportional, integral and differential gains, which control the response speed, steady-state error and anti-interference ability of the robust PID controller respectively; F robust It is a robustness enhancement term, which can be selected and designed according to specific robustness design principles; τ represents an integral variable, which represents a time variable, reflects the cumulative effect of deviation over time, is an independent variable different from t, and represents the time in the integration process. The integral term in the expression is the accumulation of the error e(τ) at all times before the current time t.

[0054] The entire quadcopter perception and control system uses the ROS communication system to transmit information between different nodes. First, the onboard camera obtains the video stream, which contains frame-by-frame image information. The visual system processes the image information of each frame and identifies the target of interest through the recognition and tracking algorithm (KCF-Y algorithm) for tracking. At this time, the visual system has obtained the environmental information and target position, and then encapsulates the message through MAVROS and passes it to the control system in the form of topic communication. Combined with the robust PID controller, control instructions are generated and the information is passed to the flight control system. The flight control system performs speed and position control instructions. Finally, the next decision is made based on the control results and environmental information.

[0055] It should be noted that most of the product is automatic, and only one part needs to be set manually, which is the name of the target of interest. Due to the different needs of different users, the first step in using the system is to set the target of interest. The target name can be any one in the COCO dataset.

[0056] The working process of the entire quadrotor drone perception and control system can be summarized as follows:

[0057] (1) Target detection and tracking preparation stage

[0058] First, you need to set the type of the target to be tracked in the preparation stage. For example, "person" can be any of the 60 types in the COCO dataset, and these 60 types are given with official training weights. If the target is a specific object, you can also build your own dataset, and train the dataset to generate the corresponding weights to identify the specific target.

[0059] (2) Target tracking stage

[0060] The KCF-Y algorithm is used in the target tracking stage. This algorithm is used for environmental perception (environmental perception is when the target is blocked or lost, the algorithm needs to re-detect the objects in the field of view to rediscover the target and continue tracking) and target tracking. In the first frame of the detection, the recognition function of the YOLOv8s algorithm is used to identify the target, and the identified target frame is passed to KCF for target tracking. From the second frame, the KCF algorithm starts tracking. During this process, YOLOv8s is used to correct the target every 50 frames to prevent the target from being lost. At the same time, the target can also be re-identified. At this stage, the location information of the target can be obtained, providing necessary information for the subsequent control part.

[0061] (3) Control information generation stage

[0062] When the target is identified and tracked, the target frame information is passed to the control system to obtain the position of the center of the target frame on the screen and the size of the target frame. A robust PID controller is used to generate the position and speed control instructions of the drone based on the position of the center of the target frame on the screen and the deviation of the target frame size relative to the center of the field of view. The robust PID algorithm is used to compare the position difference between the center of the target frame and the center of the field of view to calculate the speed and direction that need to be adjusted to ensure that the center of the target frame is within a certain range of the center of the field of view, thereby achieving stable tracking of the target.

[0063] (4) Control feedback and adjustment stage

[0064] During the control feedback and adjustment phase, the system monitors the relative position and speed between the drone and the target according to the actual flight situation, obtains feedback information, and uses this information to evaluate the effect of the PID control algorithm and identify any deviations or unstable flight behaviors. Subsequently, the next control instruction is corrected based on the feedback information to optimize the control effect and improve the stability of the system. By periodically adjusting parameters and testing, the drone's flight movements and target tracking remain stable.

[0065] In summary, the perception and control method of the quad-rotor drone under complex visual conditions in the above-mentioned embodiment, by combining the KCF algorithm with the YOLO algorithm, realizes more accurate and robust target tracking, overcomes the insufficient anti-interference of the traditional KCF algorithm in complex environments and the limitations of the YOLO algorithm, thereby improving the real-time perception ability of the drone to the target; the robust PID control algorithm is applied as the control part to realize the precise control of the attitude and position of the drone (attitude control is included in the process of speed and position control, that is, in order to achieve the desired speed command and position command, the command must be achieved through the change of attitude to achieve the control target), optimizes the stability and accuracy of flight control, and reduces the computational complexity of the system, and improves the real-time and applicability of the system; it is not only applicable to the field of drones, but also can be extended to other application scenarios that require target tracking and precise control, such as intelligent monitoring, automatic driving, etc., expanding the application scope and potential market of the technology; through the innovative solution of the visual part, the real-time perception ability of the drone to the target is improved, thereby realizing the rapid tracking and identification of moving objects in a dynamic environment.

[0066] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for perceiving and controlling a quadrotor drone under complex visual conditions, characterized in that: The following steps are involved: S1: Target detection and tracking preparation Set the type of tracking target according to detection needs; S2: Target Tracking Use KCF-Y algorithm to perform environment perception and target tracking to obtain target location information; S3: Control information generation When the target is identified and tracked, the target position information is transmitted to the control system to obtain the position of the center of the target frame in the target position information on the screen and the target frame size information. A robust PID controller is used to generate the speed and position control instructions of the UAV according to the position of the center of the target frame in the target position information on the screen and the deviation of the target frame size relative to the center point of the field of view, so as to achieve stable tracking of the target.

2. According to the method for sensing and controlling a quadrotor drone under complex visual conditions as described in claim 1, it is characterized in that: In step S2, the KCF-Y algorithm is a fusion algorithm of the KCF algorithm and the YOLOv8s algorithm, and the formula is as follows: TrackedObject=KCF(YOLO(IFrame),SFrame) Among them, IFrame represents the initial frame, SFrame represents the subsequent frame, YOLO represents the processing of the initial frame by the YOLOv8s algorithm, and KCF represents the processing of the subsequent frame by the KCF algorithm.

3. The method for sensing and controlling a quadrotor drone under complex visual conditions according to claim 2 is characterized in that: In step S2, the specific processing process is as follows: S21: In the first frame of detection, the target is identified using the YOLOv8s algorithm, and the identified target frame is passed to the KCF algorithm for target tracking; S22: Starting from the second frame, tracking is performed using the KCF algorithm to obtain target position information.

4. The method for sensing and controlling a quadrotor drone under complex visual conditions according to claim 3 is characterized in that: In the step S22, the target position information includes the position of the center of the target frame on the screen and the size information of the target frame.

5. The method for sensing and controlling a quadrotor drone under complex visual conditions according to claim 4, characterized in that: In step S3, in order to buffer the robust PID control, a buffer area is set, that is, the UAV will not generate speed and position control instructions when the target is within the range set by the center point of the UAV. The specific expression is as follows: e(t)=Ptarget(t)-Pcenter |e(t)|≤ε Among them, Ptarget(t) represents the target center point position, Pcenter is the center position of the camera field of view, and ε is the pre-defined error limit.

6. The method for sensing and controlling a quadrotor drone under complex visual conditions according to claim 5 is characterized in that: In step S3, the robust PID controller expression is as follows: Among them, u(t) is the output signal of the controller, that is, the required control quantity; e(t) is the error signal of the system, defined as the difference between the expected value and the actual value; K p , K i , K d are proportional, integral and differential gains, which control the response speed, steady-state error and anti-interference ability of the robust PID controller respectively; F robust is a robustness enhancement term; τ is a time variable that reflects the cumulative effect of deviation over time.

7. The method for sensing and controlling a quadrotor drone under complex visual conditions according to claim 6, characterized in that: The method for sensing and controlling a quadrotor drone under complex visual conditions further comprises the following steps: S4: Control feedback and adjustment The relative position and speed between the UAV and the target are monitored according to the actual flight situation to obtain feedback information, and the above information is used to evaluate the effect of the robust PID controller and identify any deviations or unstable flight behaviors.

8. A perception and control system for a quad-rotor drone under complex visual conditions, using the method described in any one of claims 1 to 7 to control the quad-rotor drone, including a visual system and a control system; The visual system is used to perceive the environment and track targets using the KCF-Y algorithm, capture targets of interest and obtain their location information; A control system is used to generate corresponding speed control instructions and position control instructions using a robust PID controller.

Images