A UAV visual tracking method using machine vision

Abstract

The invention discloses a visual tracking method for an unmanned aerial vehicle using machine vision, comprising the following steps: step a, a ground operator issues a target tracking command to an aircraft according to actual needs, and the aircraft performs an initialization step; step b, in each subsequent frame, according to the target position and size of the previous frame, extracting the features of a candidate area of ​​the target, and performing correlation operation with the target using a filter template f to obtain a response graph; in the response graph, finding the position of the maximum response value as the target position of the current frame; step c, when the aircraft is tracking a target, there is a scale transformation, and scale estimation is performed when each frame is recognized; using an image sensor carried by the unmanned aerial vehicle, through image processing and deep learning algorithms, realizing fast, accurate and robust recognition of similar targets in pictures and videos, as well as precise tracking and positioning of specific targets, thereby improving the efficiency and effect of the unmanned aerial vehicle target tracking technology.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a visual tracking method of an unmanned aerial vehicle using machine vision. Background Art

[0002] A drone is an aircraft that can fly autonomously or remotely without human control. It has the advantages of high flexibility, low cost, and high safety. It is widely used in military, security, rescue, transportation, agriculture and other fields. One of the core functions of a drone is to effectively sense, identify and track targets on the ground or in the air to achieve tasks such as monitoring, positioning, following and attacking targets.

[0003] Drone vision refers to the technology of using image sensors carried by drones, such as cameras, to obtain image information of the surrounding environment, and to achieve autonomous detection, identification, positioning and tracking of targets through image processing and analysis algorithms. Drone vision technology is an important part of drone target tracking technology and one of the key technologies for drone intelligence. However, the current application of drones to achieve the function of intelligent tracking of targets is not stable enough and the accuracy is not high. Summary of the invention

[0004] The purpose of the present invention is to provide a UAV visual tracking method using machine vision to address the deficiencies of the prior art.

[0005] The technical solution of the present invention to solve the above problem is: a method for visual tracking of a drone using machine vision, comprising the following steps:

[0006] Step a: The ground operator issues a target tracking command to the aircraft according to actual needs, and the aircraft performs initialization steps, which specifically include the following steps:

[0007] Step a1, extracting the features of the target according to its position and size, and using a filter algorithm to model the appearance of the target to obtain a filter template f;

[0008] Step a2, using the spatial regularization function w, sets the weights of samples that are far from the target center; w is a Gaussian function, and its form is:

[0009]

[0010] Where (x0, y0) is the center position of the target and σ is the standard deviation of the Gaussian function.

[0011] Step b, in each subsequent frame, according to the target position and size of the previous frame, extract the features of the candidate area of ​​the target, and use the filter template f to perform correlation operation with it to obtain a response map; in the response map, find the position of the maximum response value as the target position of the current frame;

[0012] Step c: When the aircraft is tracking, the target has a scale change. The scale is estimated when each frame is recognized. The specific method of scale estimation includes:

[0013] Step c1, downsampling the candidate regions of the target at different scales, extracting features, and performing correlation operations with the filter template f to obtain response maps at different scales;

[0014] Step c2, interpolating the multi-scale response map to obtain a response map at the original scale;

[0015] Step c3, in the response graphs at different scales, find the position of the maximum response value and the corresponding scale, use them as the target position and size of the current frame, and output the target position in the current frame to the flight control module to control the aircraft to perform tracking actions;

[0016] Step c4: in each subsequent frame, update the filter template f for tracking the next frame.

[0017] Furthermore, in step c3, the flight control module receives the target position and performs tracking action control on the aircraft, which specifically includes the following steps:

[0018] S1, initialization, setting the parameters of the PID controller, including P yaw ,I yaw ,D yaw ,P forward ,I forward ,D forward ; Initialize the position of the drone (x d ,y d ) and yaw angle θ;

[0019] S2, receiving the target position information sent back by the visual module (x t ,y t );

[0020] S3, calculate the expected yaw angle: use the inverse tangent function to calculate the target direction, that is, the expected yaw angle

[0021] θ=arctan2(y t -y d ,x t -x d );

[0022] S4, PID control to adjust the yaw angle: Use the PID controller to calculate the control input of the yaw angle to adjust the attitude of the drone;

[0023] Yaw Input =P yaw ·Yaw e +I yaw ·Yaw i +D yaw ·Yaw d

[0024] Among them, Yaw e =Desired Heading-Current Yaw,

[0025] Yaw i is the integral of the yaw angle error, Yaw d is the derivative of the yaw angle error.

[0026] S5, calculate the expected distance based on the target position:

[0027]

[0028] S6, PID control to adjust forward motion speed: Use the PID controller to calculate the control input of the forward motion speed to adjust the distance between the drone and the target.

[0029] V=P forward Distance+I forward Distance Integral+D forward DistanceDerivative

[0030] Where: Distance Integral is the integral of the distance error, and Distance Derivative is the derivative of the distance error.

[0031] Furthermore, the method of updating the filter template f is to use the target features and response map of the current frame and the filter template f of the previous frame to solve the new filter template f through the least squares error principle. The purpose of updating the filter template f is to enable the filter template f to adapt to the changes in the appearance and movement of the target and improve the tracking accuracy and robustness.

[0032] Furthermore, the specific method of step c1 is: setting a scale factor to 1.05, and then performing multi-scale sampling to form a multi-scale response map.

[0033] The present invention has the beneficial effects:

[0034] The present invention provides a UAV visual tracking method using machine vision, which utilizes the image sensor carried by the UAV, through image processing and deep learning algorithms, to achieve fast, accurate and robust recognition of similar targets in pictures and videos, as well as precise tracking and positioning of specific targets, thereby improving the efficiency and effectiveness of UAV target tracking technology. DETAILED DESCRIPTION

[0035] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0036] A method for visual tracking of a drone using machine vision comprises the following steps:

[0037] Step a: The ground operator issues a target tracking command to the aircraft according to actual needs, and the aircraft performs initialization steps, which specifically include the following steps:

[0038] Step a1, according to the position and size of the target, extract the features of the target, including HOG features and CN features, and use the filter algorithm to model the appearance of the target to obtain the filter template f; HOG features are gradient direction histogram features that describe the target, which can effectively express the edge and shape information of the target; CN features are color name features that describe the target, which can effectively express the color information of the target; the filter algorithm uses frequency domain operations to accelerate the correlation calculation of the target to achieve fast target positioning and tracking;

[0039] Step a2, using the spatial regularization function w, sets the weights of samples that are far from the target center; w is a Gaussian function, and its form is:

[0040]

[0041] Where (x0, y0) is the center position of the target and σ is the standard deviation of the Gaussian function.

[0042] The spatial regularization function w makes the weight of samples closer to the target center larger and the weight of samples farther from the target center smaller, which can effectively suppress the response of the background area and improve the discrimination of the target.

[0043] Step b, in each subsequent frame, according to the target position and size of the previous frame, extract the features of the candidate area of ​​the target, and use the filter template f to perform correlation operation with it to obtain a response map; in the response map, find the position of the maximum response value as the target position of the current frame; the candidate area of ​​the target is a square area centered on the center of the target and with a side length of k times (k>1) of the target size, where k is an adjustable parameter for controlling the size of the search range; the correlation operation uses Fourier transform to convert the time domain operation into a frequency domain operation; the response map is a two-dimensional matrix, which represents the correlation between each position in the candidate area of ​​the target and the filter template f. The larger the response value, the higher the correlation, and the more likely it is the location of the target; the position of the maximum response value can be determined by simple comparison or interpolation methods, which represents the position in the candidate area of ​​the target that best matches the filter template f, that is, the best estimated position of the target.

[0044] Step c: When the aircraft is tracking, the target has a scale change. The scale is estimated when each frame is recognized. The specific method of scale estimation includes:

[0045] Step c1, at different scales, downsample the candidate region of the target, extract features, and perform correlation operations with the filter template f to obtain response maps at different scales; set a scale factor to 1.05, and then sample at multiple scales to form a multi-scale response map. The response map at different scales refers to the size of the candidate region of the target being scaled according to a certain ratio, setting a scale factor to 1.05, and then downsampling at three scales of 0.95, 1, and 1.05 to obtain candidate regions of the target at three different scales. The purpose of this is to handle the scale change of the target, that is, the size of the target may change with the distance.

[0046] Step c2, interpolating the multi-scale response map to obtain a response map at the original scale; interpolation is used to increase image resolution and improve image quality and accuracy; the original scale means that the size of the candidate region of the target is the same as the target size of the previous frame, and the purpose of doing so is to unify the response maps at different scales into one scale for easy comparison and selection;

[0047] Step c3, in the response diagram at different scales, find the position and corresponding scale of the maximum response value as the target position and size of the current frame, and output the target position in the current frame to the flight control module to control the aircraft to perform tracking actions; the position and corresponding scale of the maximum response value can be determined by simple comparison or interpolation methods, and they represent the position and scale in the candidate area of ​​the target that best matches the filter template f.

[0048] Step c4, in each subsequent frame, update the filter template f for tracking the next frame; the method of updating the filter template f is to use the target features and response map of the current frame and the filter template f of the previous frame to solve the new filter template f through the least squares error principle. The purpose of updating the filter template f is to enable the filter template f to adapt to the changes in the appearance and movement of the target and improve the accuracy and robustness of tracking.

[0049] In step c3, the flight control module receives the target position and performs tracking action control on the aircraft, which specifically includes the following steps:

[0050] S1, initialization, setting the parameters of the PID controller, including P yaw ,I yaw ,D yaw ,P forward ,I forward ,D forward ; Among them, P yaw ,I yaw ,D yaw is the yaw control PID parameter, P forward ,I forward ,D forward Control PID parameters for the UAV's speed; initialize the UAV's position (x d ,y d ) and yaw angle θ;

[0051] S2, receiving the target position information sent back by the visual module (x t ,y t );

[0052] S3, calculate the expected yaw angle: use the inverse tangent function to calculate the target direction, that is, the expected yaw angle

[0053] θ=arctan2(y t -y d ,x t -x d );

[0054] S4, PID control to adjust the yaw angle: Use the PID controller to calculate the control input of the yaw angle to adjust the attitude of the drone;

[0055] Yaw Input =P yaw ·Yaw e +I yaw ·Yaw i +D yaw ·Yaw d

[0056] Yaw e=Desired Heading-Current Yaw,

[0057] Yaw i =∫Yaw e dt

[0058]

[0059] Yaw e is the yaw angle error, Yaw i is the time integral of the yaw angle error, Yaw d It is the time derivative of the yaw angle error, Desired Heading is the target heading angle, and Current Yaw is the current UAV heading angle.

[0060] S5, calculate the expected distance based on the target position:

[0061]

[0062] S6, PID control to adjust forward motion speed: Use the PID controller to calculate the control input of the forward motion speed to adjust the distance between the drone and the target.

[0063] V=P forward Distance+I forward Distance Integral+D forward DistanceDerivative

[0064] Where: Distance Integral is the integral of the distance error, Distance Derivative is the derivative of the distance error, and the distance error is the difference between the desired position of the drone and its current position.

[0065] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for visual tracking of a drone using machine vision, characterized in that: The following steps are involved: Step a: The ground operator issues a target tracking command to the aircraft according to actual needs, and the aircraft performs initialization steps, which specifically include the following steps: Step a1, extracting the features of the target according to its position and size, and using a filter algorithm to model the appearance of the target to obtain a filter template f; Step a2, using the spatial regularization function w, sets the weights of samples that are far from the target center; w is a Gaussian function, and its form is: Where (x0, y0) is the center position of the target, and σ is the standard deviation of the Gaussian function; Step b, in each subsequent frame, according to the target position and size of the previous frame, extract the features of the candidate area of ​​the target, and use the filter template f to perform correlation operation with it to obtain a response map; in the response map, find the position of the maximum response value as the target position of the current frame; Step c: When the aircraft is tracking, the target has a scale change. The scale is estimated when each frame is recognized. The specific method of scale estimation includes: Step c1, downsampling the candidate regions of the target at different scales, extracting features, and performing correlation operations with the filter template f to obtain response maps at different scales; Step c2, interpolating the multi-scale response map to obtain a response map at the original scale; Step c3, in the response graphs at different scales, find the position of the maximum response value and the corresponding scale, use them as the target position and size of the current frame, and output the target position in the current frame to the flight control module to control the aircraft to perform tracking actions; Step c4, in each subsequent frame, update the filter template f for tracking the next frame; In step c3, the flight control module receives the target position and performs tracking action control on the aircraft, which specifically includes the following steps: S1, initialization, setting the parameters of the PID controller, including P yaw ,I yaw ,D yaw ,P forward ,I forward ,D forward ; Initialize the position of the drone (x d ,y d ) and yaw angle θ; where P yaw ,I yaw ,D yaw is the yaw control PID parameter, P forward ,I forward ,D forward Control PID parameters for the UAV's speed; S2, receiving the target position information sent back by the visual module (x t ,y t ); S3, calculate the expected yaw angle: use the inverse tangent function to calculate the target direction, that is, the expected yaw angle θ=arctan2(y t -y d ,x t -x d ): S4, PID control to adjust the yaw angle: Use the PID controller to calculate the control input of the yaw angle to adjust the attitude of the drone; Yaw Input =P yaw ·Yaw e +I yaw ·Yaw i +D yaw ·Yaw d Yaw e =Desired Heading-Current Yaw, Yaw i =∫Yaw e dt Yaw e is the yaw angle error, Yaw i is the time integral of the yaw angle error, Yaw d It is the time derivative of the yaw angle error, Desired Heading is the target heading angle, Current Yaw is the current UAV heading angle; S5, calculate the expected distance based on the target position: S6, PID control to adjust the forward motion speed: use the PID controller to calculate the control input of the forward motion speed to adjust the distance between the UAV and the target; V=P forward ·Distance+I forward ·Distance Integral+D forward ·Distance Derivative Where: Distance Integral is the integral of the distance error, Distance Derivative is the derivative of the distance error, and the distance error is the difference between the desired position of the drone and its current position.

2. A method for visual tracking of a drone using machine vision as claimed in claim 1, characterized in that: The method of updating the filter template f is to use the target features and response map of the current frame and the filter template f of the previous frame to solve a new filter template f through the least squares error principle.

3. The method for visual tracking of a drone using machine vision according to claim 1, characterized in that: The specific method of step c1 is: setting a scale factor to 1.05, and then performing multi-scale sampling to form a multi-scale response map.

Images