A method for unmanned aerial vehicle target detection and tracking

Abstract

This application proposes a method for unmanned aerial vehicle (UAV) target detection and tracking, belonging to the field of UAV target detection and localization technology. The method includes the following steps: acquiring a historical image dataset collected by the UAV and preprocessing it to obtain a standard image dataset; constructing a network model and using the optimized network model to perform real-time detection on all acquired actual images; calculating target control parameters for each actual image based on all real-time detection results and the calibration parameters of the binocular depth camera; processing each target control parameter to obtain corresponding stable control parameters and transmitting them to the flight control system; and controlling the UAV to track the target based on all stable control parameters, the UAV's real-time flight status, and the target's actual position information. This application can effectively improve detection accuracy and meet the dual requirements of real-time performance and lightweight design for UAVs in target detection and tracking.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) target detection and positioning technology, and in particular to a method for UAV target detection and tracking. Background Technology

[0002] Currently, drones are widely used in target detection and tracking. Existing drone target detection and localization methods mainly fall into two categories.

[0003] The first category is UAV target detection and localization methods based on ground processing platforms. These methods typically transmit image data collected by the UAV back to the ground processing platform in real time via fiber optic or wireless links. The ground processing platform then performs target identification and localization, and finally sends control commands back to the UAV. This type of method is limited by the bandwidth and stability of the communication link, especially in long-distance scenarios or complex environments. It is prone to problems such as communication link interruptions, command delays, and target tracking failures, severely impacting the response efficiency and execution accuracy of the detection task.

[0004] The second category is UAV target detection and localization methods based on edge computing platforms. These methods typically deploy visual recognition models directly on the edge computing platform carried by the UAV, processing video streams from the UAV in real time and outputting target positions and control commands. Due to their strong anti-interference capabilities and deployment flexibility, they have become a hot research area.

[0005] Despite the many advantages of the second type of method, several key technical bottlenecks still exist:

[0006] First, the deep learning target detection models used generally have a large number of parameters and high computational complexity, which cannot be efficiently deployed on the edge computing platform carried by the UAV. They often suffer from problems such as slow running speed, high power consumption, and limited deployment, making it difficult to meet the UAV's requirements for real-time performance and autonomy.

[0007] Secondly, most existing positioning and control methods use monocular imaging combined with simplified estimation, lacking a precise three-dimensional position calculation process that integrates depth information, resulting in insufficient target positioning accuracy. In addition, the control logic of flight control systems generally relies on rule-based judgment, which cannot form a closed-loop system structure of detection-positioning-control, making it difficult to achieve accurate and continuous automatic tracking.

[0008] Therefore, it is necessary to propose a solution to improve one or more problems existing in the above-mentioned related technical solutions.

[0009] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0010] This application provides a method for target detection and tracking of unmanned aerial vehicles (UAVs), the method comprising the following steps:

[0011] The historical image dataset collected by the drone is acquired and preprocessed to obtain a standard image dataset; the drone is equipped with a binocular depth camera.

[0012] Build Network models, using standard image datasets to... The network model is optimized, and the optimized model is... The network model is deployed on an edge computing platform carried by the drone;

[0013] Utilize the optimized The network model performs real-time detection on all actual images acquired by the binocular depth camera and obtains the pixel coordinates of the center point of the target detection box for each actual image.

[0014] Using the center pixel coordinates of all target detection boxes and the calibration parameters of the stereo depth camera, the target control parameters corresponding to each actual image are calculated respectively.

[0015] Each target control parameter is sequentially filtered, anomaly removed, and smoothed to obtain the corresponding stable control parameters, and all stable control parameters are transmitted to the UAV's flight control system.

[0016] The flight control system controls the UAV to track the target based on all stable control parameters, the UAV's real-time flight status, and the target's actual position information;

[0017] The historical image dataset includes multiple historical images; the standard image dataset includes multiple standard images.

[0018] Furthermore, preprocessing includes sequentially performing data cleaning and filtering, data augmentation, and data adjustment and annotation on all historical images;

[0019] Data cleaning and filtering processes include:

[0020] Based on the area constraint rule, all historical images in the historical image dataset that are abnormally labeled, unidentifiable, have distorted viewpoints, or have cluttered backgrounds are removed to obtain a normal image dataset.

[0021] Data augmentation processes include:

[0022] Each normal image in the normal image dataset is rotated, scaled, and adjusted sequentially. Image compositing techniques are then introduced to enhance the background of all normal images, resulting in an enhanced image dataset.

[0023] Data adjustment and annotation processing includes:

[0024] All augmented images in the augmented image dataset are resized to standard size, and the category and location information of the target involved in each augmented image are labeled using a labeling tool to generate corresponding labeling files, thus obtaining corresponding standard images. All standard images constitute the standard image dataset.

[0025] Furthermore, The network model consists of a backbone network, a neck network, and a head network connected in sequence.

[0026] The backbone network consists of 11 convolutional modules connected unidirectionally in sequence: the 0th convolutional module is... Convolutional module, the first convolutional module is The convolutional module and the second convolutional module are The convolutional module, the third convolutional module is The convolutional module, the fourth convolutional module is The convolutional module, the 5th convolutional module is The convolutional module, the 6th convolutional module is The convolutional module, the 7th convolutional module is The convolutional module, the 8th convolutional module is The convolutional module, the 9th convolutional module is The convolutional module, the 10th convolutional module is Convolutional module;

[0027] The neck network consists of 12 sequentially unidirectionally connected convolutional modules: the 11th convolutional module is... The convolutional module, the 12th convolutional module is The convolutional module, the 13th convolutional module includes The convolutional module, the 14th convolutional module is The convolutional module, the 15th convolutional module is The convolutional module, the 16th convolutional module is The convolutional module, the 17th convolutional module is The convolutional module, the 18th convolutional module is The convolutional module, the 19th convolutional module is The convolutional module, the 20th convolutional module is The convolutional module, the 21st convolutional module is The convolutional module, the 22nd convolutional module is Convolutional module;

[0028] The head network consists of three convolutional modules: the 23rd, 24th, and 25th convolutional modules are all... Convolutional module;

[0029] Specifically, the 4th convolutional module in the backbone network is unidirectionally connected to the 15th convolutional module in the neck network, the 6th convolutional module in the backbone network is unidirectionally connected to the 12th convolutional module in the neck network, and the 10th convolutional module in the backbone network is unidirectionally connected to the 11th and 21st convolutional modules in the neck network, respectively.

[0030] The 13th convolutional module in the neck network is unidirectionally connected to the 18th convolutional module; the 16th convolutional module in the neck network is unidirectionally connected to the 23rd convolutional module in the head network; the 19th convolutional module in the neck network is unidirectionally connected to the 24th convolutional module in the head network; and the 22nd convolutional module in the neck network is unidirectionally connected to the 25th convolutional module in the head network.

[0031] Furthermore, The convolution module performs the following operations:

[0032] The convolution module transforms all the input first feature maps into corresponding main feature maps, and performs linear operations on each main feature map to generate multiple auxiliary feature maps corresponding to each first feature map;

[0033] Each first feature map is concatenated with all its corresponding auxiliary feature maps, and the result of all concatenation operations is used as... The output of the convolution module;

[0034] The overall computational complexity of the convolution module is expressed as follows:

[0035] (1)

[0036] in, express The overall computational cost of the convolution module, express The number of all input channels in the convolution module, This indicates the number of all principal feature maps. This indicates the size of the convolution kernel in the main feature map. Indicates the height of the main feature map. Indicates the width of the main feature map. This indicates the number of all auxiliary feature maps. express Mapping scaling factor, Indicates the height of the auxiliary feature map. Indicates the width of the auxiliary feature map. , All main feature maps have the same height and width. All auxiliary feature maps corresponding to each main feature map have the same height and width. The convolution kernels of all main feature maps have the same size.

[0037] Furthermore, The convolution module performs the following operations:

[0038] The convolution module uses three different sizes of convolution kernels to convolve all the input second feature maps respectively, and then concatenates all the convolutional feature maps obtained after each second feature map convolution to obtain the corresponding fused feature map.

[0039] The expression for the fused feature map is:

[0040] (2)

[0041] in, Indicates the first The fused feature map corresponding to the second feature map Indicates the first The features of all convolutional feature maps corresponding to the second feature map are concatenated. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Perform convolution operation on the second feature map;

[0042] Calculate the mean and variance of all fusion feature values ​​in each fusion channel of each fusion feature map, and use the mean and variance of all fusion feature values ​​to generate the attention weights corresponding to each fusion channel of each fusion feature map.

[0043] The expression for calculating the mean of all fusion feature values ​​in the fusion channel is:

[0044] (3)

[0045] in, Indicates the first The fused feature map corresponding to the second feature map is the first The mean of all fusion feature values ​​in each fusion channel, Indicates the first The height of the fused feature map corresponding to the second feature map. Indicates the first The width of the fused feature map corresponding to each second feature map. Indicates the first The fusion feature map corresponding to the second feature map is the first The first fusion channel Line number The fusion feature value of the column;

[0046] The expression for calculating the variance of all fused feature values ​​in the fused channel is:

[0047] (4)

[0048] in, Indicates the first The fused feature map corresponding to the second feature map is the first The variance of all fused feature values ​​in each fused channel;

[0049] Each attention weight is weighted by its corresponding fusion feature map to obtain a weighted fusion feature map for each fusion channel of each fusion feature map.

[0050] The expression for the weighted fusion feature map is:

[0051] (5)

[0052] in, Indicates the first The fusion feature map corresponding to the second feature map is the first The weighted fusion feature map corresponding to each fusion channel Indicates the first The fusion feature map corresponding to the second feature map is the first Attention weights corresponding to each fusion channel.

[0053] Furthermore, The convolution module performs the following operations:

[0054] Each input third feature map is convolved with its corresponding shared convolution weights to generate an output feature map for each third feature map.

[0055] The expression for the output feature map is:

[0056] (6)

[0057] in, Indicates the first The output feature map corresponding to the third feature map. Indicates the first Shared convolution operation performed on the third feature map Indicates the first The shared convolution weights corresponding to the third feature map This represents the convolution operation. Indicates the first The third feature map, Indicates the bias term;

[0058] Each output feature map is divided into multiple output channel groups, and the mean and variance of the feature values ​​of all output channels in each output channel group are calculated.

[0059] By using the mean and variance of the feature values ​​of all output channels in each output channel group, the feature values ​​of all output channels in the corresponding output channel group are normalized to obtain the normalized data distribution of each output channel group.

[0060] The normalized data distributions of all output channel groups in each output feature map are concatenated to obtain the normalized feature map corresponding to each output feature map.

[0061] The expression for the normalized feature map is:

[0062] (7)

[0063] in, Indicates the output feature map The corresponding normalized feature map, Indicates the first The first learnable parameter of the group output channel group. Indicates the first The mean of the eigenvalues ​​of all output channels in the output channel group. Indicates the first The variance of the eigenvalues ​​of all output channels in the output channel group. This represents a constant used to prevent division by zero errors. Indicates the first The second learnable parameter of the group output channel group;

[0064] Global average pooling is performed on all scale features of each normalized feature map to generate weighted coefficients for each scale feature. Weighted fusion processing is then performed on all scale features to obtain the weighted fused features of each normalized feature map.

[0065] The expression for the weighted fusion feature is:

[0066] (8)

[0067] in, Represents normalized feature maps The weighted fusion characteristics, Represents normalized feature maps The Individual scale features Represents normalized feature maps The Weighting coefficients corresponding to each scale feature;

[0068] The classification network is used to decouple all weighted fusion features and predict the corresponding target category probability.

[0069] By using a regression network, all weighted fused features are decoupled and the bounding box coordinates of the corresponding targets are predicted.

[0070] Furthermore, utilizing the optimized The network model performs real-time detection on all actual images acquired by the binocular depth camera, and the steps to obtain the center point pixel coordinates of the target detection box for each actual image include:

[0071] All actual images captured by the binocular depth camera are sent to the edge computing platform;

[0072] Utilize the optimized The network model processes each actual image received by the edge computing platform in real time to obtain the coordinates of the corresponding target detection box;

[0073] Based on the coordinates of all target detection boxes, calculate the center pixel coordinates of each target detection box. The center pixel coordinates include the horizontal pixel coordinates and the vertical pixel coordinates of the center point.

[0074] The expression for the horizontal pixel coordinates of the center point of the target detection bounding box is:

[0075] (9)

[0076] in, Indicates the first The horizontal pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The left boundary of the target detection box corresponding to the actual image at time t. Indicates the first The right boundary of the target detection box corresponding to the actual image at time t;

[0077] The expression for the vertical pixel coordinates of the center point of the object detection bounding box is:

[0078] (10)

[0079] in, Indicates the first The vertical pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The upper boundary of the target detection box corresponding to the actual image at time t. Indicates the first The lower boundary of the target detection box corresponding to the actual image at time t.

[0080] Furthermore, the steps for calculating the target control parameters for each actual image using the center pixel coordinates of all target detection boxes and the calibration parameters of the stereo depth camera include:

[0081] The calibration parameters of a binocular depth camera include the horizontal field of view and the horizontal resolution of the image.

[0082] Calculate the target depth value of the binocular depth camera based on visual principles;

[0083] The expression for the target depth value is:

[0084] (11)

[0085] in, This represents the target depth value of the stereo depth camera. This indicates the focal length of the binocular depth camera. This indicates the baseline distance of the binocular depth camera. This represents the parallax of a binocular depth camera;

[0086] Using the pixel coordinates of the center point of the target detection box corresponding to each actual image, the optical center coordinates of each actual image, and the calibration parameters of the binocular depth camera, the lateral deviation angle of the target in the corresponding actual image is calculated.

[0087] The expression for the lateral deflection angle of the target is:

[0088] (12)

[0089] in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at a given time. This indicates the horizontal half-resolution of the image from a binocular depth camera. Indicates the first The horizontal pixel offset of the center point of the target detection box corresponding to the actual image at any given time. , Indicates the first The x-coordinate of the optical center of the actual image at that moment. This indicates the horizontal field of view of the binocular depth camera. This indicates the horizontal resolution of the image from the binocular depth camera;

[0090] Using the target depth value, the pixel coordinates of the center point of the target detection box corresponding to each actual image are converted into three-dimensional coordinates in the camera coordinate system;

[0091] Camera coordinate system The expression for the axis coordinates is:

[0092] (13)

[0093] in, Indicates the first The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image at a given time. Axis coordinates Indicates a stereo depth camera Focal length along the axial direction;

[0094] Camera coordinate system The expression for the axis coordinates is:

[0095] (14)

[0096] in, Indicates the first The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image at a given time. Axis coordinates Indicates the first The optical center ordinate of the actual image at that moment. Indicates a stereo depth camera Focal length along the axial direction;

[0097] The target depth value is used as the camera coordinate system corresponding to the center pixel coordinates of the target detection box in each actual image. Axis coordinates;

[0098] The target control parameters for the actual image are derived by combining the 3D coordinates of each camera coordinate system with the corresponding lateral deflection angle of the target. The expression for the target control parameters is as follows: ,in, Indicates the first The target control parameters corresponding to the actual image at a given time.

[0099] Furthermore, the steps of sequentially performing filtering, anomaly removal, and smoothing processing on each target control parameter to obtain the corresponding stable control parameters, and then transmitting all stable control parameters to the UAV's flight control system include:

[0100] The filtering process is adaptively adjusted based on the target depth value, including:

[0101] when When the distance is 1 meter, it indicates that the target and the UAV are at a long distance. The corresponding target control parameters are filtered using a sparse sliding window. At this time, the range of the sparse sliding window is from the 6th frame to the 10th frame.

[0102] when When the distance is 1 meter, it indicates that the target and the UAV are at close range. The corresponding target control parameters are filtered using a continuous sliding window. At this time, the range of the continuous sliding window is from frame 1 to frame 5.

[0103] Frames 1 to 5 and frames 6 to 10 all represent the buffer index numbers relative to the actual image of the current frame. Specifically, frame 1 represents the buffer index number of the actual image of the first frame before the actual image of the current frame, frame 5 represents the buffer index number of the actual image of the fifth frame before the actual image of the current frame, frame 6 represents the buffer index number of the actual image of the sixth frame before the actual image of the current frame, and frame 10 represents the buffer index number of the actual image of the tenth frame before the actual image of the current frame.

[0104] The expression for filtering the lateral deflection angle of the target is:

[0105] (15)

[0106] in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at time [time] after filtering. This represents the total number of all actual images. Indicates the first Time, Number The weights corresponding to the actual image of the frame. Indicates the first Time, Number The lateral tilt angle of the target corresponding to the actual image of the frame;

[0107] For the camera coordinate system The expression for filtering the axis coordinates is:

[0108] (16)

[0109] in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates;

[0110] For the camera coordinate system The expression for filtering the axis coordinates is:

[0111] (17)

[0112] in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates;

[0113] For the camera coordinate system The expression for filtering the axis coordinates is:

[0114] (18)

[0115] in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates;

[0116] The process of removing anomalies includes:

[0117] The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as ;

[0118] The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as ;

[0119] Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , ;

[0120] Set the camera coordinate system separately Reasonable threshold for changes in axis coordinates Reasonable threshold for the change of axis coordinates and The reasonable threshold for the change in axis coordinates. If any change exceeds the corresponding reasonable threshold, the current frame's actual image is determined to be an abnormal jump. In this case, the current frame's actual image is discarded, and the filtering result corresponding to the previous frame's actual image is used to replace the current frame's actual image's filtering result.

[0121] in, Reasonable threshold for the change in axis coordinates express, Reasonable threshold for the change in axis coordinates express, Reasonable threshold for the change in axis coordinates express;

[0122] If none of the changes exceed the corresponding reasonable threshold, the filtering result of the target control parameter corresponding to the actual image of the current frame is smoothed to obtain the corresponding stable control parameter.

[0123] All stability control parameters are transmitted to the flight control system in real time via serial communication.

[0124] Furthermore, the flight control system, based on all stability control parameters, the UAV's real-time flight status, and the target's actual position information, controls the UAV to track the target through the following steps:

[0125] When the target and the drone are at a long distance, the drone adjusts the target's heading angle according to the heading angle at each moment;

[0126] The expression for the target heading angle is:

[0127] (19)

[0128] in, Indicates that the drone is in The target heading angle at any given moment, Indicates that the drone is in The heading angle at any given moment;

[0129] when When, it indicates the first The drone constantly adjusts its target heading angle clockwise.

[0130] when When, it indicates the first The drone constantly adjusts its target heading angle counterclockwise.

[0131] when When, it indicates the first The drone must maintain its current heading at all times;

[0132] When the target and the UAV are at close range, calculate the horizontal yaw angle and vertical pitch angle of the target relative to the UAV in each actual image, and combine them with the heading angle and pitch angle of the UAV at each moment to calculate the target's expected attitude at the corresponding moment. The target's expected attitude includes the target's yaw angle and target pitch angle.

[0133] The expression for the target's horizontal yaw angle relative to the UAV is:

[0134] (20)

[0135] in, Indicates the first The horizontal yaw angle of the target relative to the UAV in the actual image at that moment;

[0136] The expression for the target's vertical pitch angle relative to the UAV is:

[0137] (twenty one)

[0138] in, Indicates the first The vertical pitch angle of the target relative to the UAV in the actual image at that moment;

[0139] The expression for the target yaw angle is:

[0140] (twenty two)

[0141] in, Indicates the first The target yaw angle at that moment;

[0142] The expression for the target pitch angle is:

[0143] (twenty three)

[0144] in, Indicates the first The target pitch angle at any given moment, Indicates that the drone is in The pitch angle at any given moment;

[0145] The drone tracks the target based on its desired attitude;

[0146] when When, it indicates the first The target yaw angle is corrected to the right at any given moment;

[0147] when When, it indicates the first The target yaw angle is corrected to the left at any given moment;

[0148] when When, it indicates the first The target's pitch angle should be corrected upwards at all times;

[0149] when When, it indicates the first The target's pitch angle is constantly corrected downwards;

[0150] when At that time, and When, it indicates the first The drone immediately enters the close-range tracking phase of the target.

[0151] This application provides a method for target detection and tracking of unmanned aerial vehicles (UAVs), which has at least the following beneficial effects:

[0152] (1) This application constructs Network model, in Introduced in network model Convolutional module Convolutional modules and The convolutional module effectively reduces the number of model parameters and computational complexity; by deploying the optimized module on the edge computing platform of the drone... The network model, while maintaining detection accuracy, significantly improves the computing speed and resource utilization of the edge computing platform, meeting the dual requirements of drones for real-time performance and lightweight design.

[0153] (2) This application constructs a complete closed-loop system of "detection-tracking-control", which integrates data enhancement, three-dimensional positioning, dynamic filtering, serial communication and user datagram protocol backhaul mechanisms, and realizes the automated and intelligent operation of target detection, tracking control and remote transmission, and has good practicality and promotion value. Attached Figure Description

[0154] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0155] Figure 1 This illustration shows the steps of a drone target detection and tracking method according to an exemplary embodiment of this application;

[0156] Figure 2 This diagram illustrates a flowchart of an unmanned aerial vehicle (UAV) target detection and tracking method according to an exemplary embodiment of this application.

[0157] Figure 3 The prior art shown in the exemplary embodiments of this application A schematic diagram of the network structure;

[0158] Figure 4 This application demonstrates the construction of the application in an exemplary embodiment. A schematic diagram of the network model structure;

[0159] Figure 5 Existing models in this application are shown In the network The convolution module and the one constructed in this application In network models A comparative diagram of convolutional modules;

[0160] Figure 6 This application demonstrates the construction of the application in an exemplary embodiment. In network models Convolutional modules and regular Convolutional modules and A comparative diagram of convolutional modules;

[0161] Figure 7 In the exemplary embodiments shown in this application A schematic diagram of the convolutional module;

[0162] Figure 8 This diagram illustrates a system schematic for implementing the UAV target detection and tracking method proposed in this application in an exemplary embodiment of the present application. Detailed Implementation

[0163] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0164] Furthermore, the accompanying drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0165] The following is a more detailed description of a UAV target detection and tracking method proposed in this exemplary embodiment. Figure 1 and Figure 2 As shown, the method may include the following steps:

[0166] In this embodiment, step S101 is to acquire a historical image dataset collected by the UAV and preprocess the historical image dataset to obtain a standard image dataset.

[0167] Furthermore, preprocessing includes sequentially performing data cleaning and filtering, data augmentation, and data adjustment and annotation on all historical images.

[0168] Furthermore, the data cleaning and filtering process includes: based on area constraint rules, removing all historical images from the historical image dataset that are abnormally labeled, unidentifiable, have distorted viewpoints, or have cluttered backgrounds, to obtain a normal image dataset.

[0169] Furthermore, the data augmentation processing includes: to address issues such as variable perspective, scale differences, and unstable lighting during drone photography, a multi-strategy data augmentation method is employed. On one hand, rotation, scaling, and brightness adjustment operations are sequentially performed on all normal images in the normal image dataset. On the other hand, image synthesis technology is introduced, preferably in this embodiment. Image synthesis techniques enhance the background diversity of all normal images, resulting in an enhanced image dataset.

[0170] Furthermore, the data adjustment and annotation process includes: uniformly adjusting all enhanced images in the enhanced image dataset to match the constructed... The network model is adapted to a standard size and utilizes... Annotation tools are used to annotate the category and location information of the targets involved in each enhanced image, generating corresponding annotation files and obtaining corresponding standard images. All standard images are combined to form a standard image dataset.

[0171] Here, the drone is equipped with a binocular depth camera. The historical image dataset includes multiple historical images; the standard image dataset includes multiple standard images.

[0172] Step S102 of this embodiment: as follows Figure 3 and Figure 4 As shown, construct Network models, using standard image datasets to... The network model is optimized, and the optimized model is... The network model is deployed on an edge computing platform carried by the drone.

[0173] The optimization here refers to using standard image datasets to... The network model is trained, validated, and tested.

[0174] Furthermore, in the construction of this embodiment The network model consists of a backbone network, a neck network, and a head network connected in sequence.

[0175] The backbone network consists of 11 convolutional modules connected unidirectionally in sequence: the 0th convolutional module is... Convolutional module, the first convolutional module is The convolutional module and the second convolutional module are The convolutional module, the third convolutional module is The convolutional module, the fourth convolutional module is The convolutional module, the 5th convolutional module is The convolutional module, the 6th convolutional module is The convolutional module, the 7th convolutional module is The convolutional module, the 8th convolutional module is The convolutional module, the 9th convolutional module is The convolutional module, the 10th convolutional module is Convolution module.

[0176] The neck network consists of 12 sequentially unidirectionally connected convolutional modules: the 11th convolutional module is... The convolutional module, the 12th convolutional module is The convolutional module, the 13th convolutional module includes The convolutional module, the 14th convolutional module is The convolutional module, the 15th convolutional module is The convolutional module, the 16th convolutional module is The convolutional module, the 17th convolutional module is The convolutional module, the 18th convolutional module is The convolutional module, the 19th convolutional module is The convolutional module, the 20th convolutional module is The convolutional module, the 21st convolutional module is The convolutional module, the 22nd convolutional module is Convolution module.

[0177] The head network consists of three convolutional modules: the 23rd, 24th, and 25th convolutional modules are all... Convolution module.

[0178] Specifically, the 4th convolutional module in the backbone network is unidirectionally connected to the 15th convolutional module in the neck network, the 6th convolutional module in the backbone network is unidirectionally connected to the 12th convolutional module in the neck network, and the 10th convolutional module in the backbone network is unidirectionally connected to the 11th and 21st convolutional modules in the neck network, respectively.

[0179] The 13th convolutional module in the neck network is unidirectionally connected to the 18th convolutional module; the 16th convolutional module in the neck network is unidirectionally connected to the 23rd convolutional module in the head network; the 19th convolutional module in the neck network is unidirectionally connected to the 24th convolutional module in the head network; and the 22nd convolutional module in the neck network is unidirectionally connected to the 25th convolutional module in the head network.

[0180] Depend on Figure 3 and Figure 4 It can be seen that the system constructed in this embodiment... Network models and existing Compared to the previous network, optimizations were made to the 0th, 1st, and 3rd convolutional modules in the backbone network, all of which were modified from the original... The convolution module was replaced with Convolution module. Convolutional modules consume a large amount of computation when extracting features, while Convolutional modules can reduce computational costs while ensuring the effectiveness of feature extraction.

[0181] In this embodiment, as Figure 5As shown, The convolution module performs the following operations:

[0182] first, The convolution module transforms all the input first feature maps into corresponding main feature maps, and performs linear operations on each main feature map to generate multiple auxiliary feature maps corresponding to each first feature map.

[0183] Then, each first feature map and its corresponding auxiliary feature maps are concatenated, and the results of all concatenation operations are used as... The output of the convolution module.

[0184] Figure 5 'a' represents a regular convolution, i.e., the original convolution. Convolution module. The convolutional module effectively reduces redundant computation by generating partially redundant feature maps using linear operations. (The original...) The overall computational complexity of the convolution module is expressed as follows:

[0185]

[0186] in, express The overall computational cost of the convolution module, Indicates the number of input channels. Indicates the number of output channels. This indicates the size of the convolution kernel in the feature map. Indicates the height of the feature map. Indicates the width of the feature map.

[0187] Figure 5 b is Convolution module. The overall computational complexity of the convolution module is expressed as follows:

[0188] (1)

[0189] in, express The overall computational cost of the convolution module, express The number of all input channels in the convolution module, This indicates the number of all principal feature maps. This indicates the size of the convolution kernel in the main feature map. Indicates the height of the main feature map. Indicates the width of the main feature map. This indicates the number of all auxiliary feature maps. express Mapping scaling factor, Indicates the height of the auxiliary feature map. Indicates the width of the auxiliary feature map. , All main feature maps have the same height and width. All auxiliary feature maps corresponding to each main feature map have the same height and width. The convolution kernels of all main feature maps have the same size.

[0190] You can see ,and ,illustrate Convolutional modules can significantly reduce the number of multiply-accumulate operations and improve the operating speed of edge computing platforms.

[0191] Furthermore, in this embodiment, the constructed In the backbone of the network model, the 6th and 8th convolutional modules both convert the original... The convolution module was replaced with Convolution module. The convolution module is in It improves upon the convolution module by inheriting the channel attention mechanism of multi-scale convolution kernels and employing multiple convolution kernels of different sizes in the multi-scale convolution part. convolution kernel, convolution kernel and The convolution kernel is used to perform convolution processing on the input second feature map.

[0192] like Figure 6 As shown, The convolution module performs the following operations:

[0193] first, The convolution module uses three different sizes of convolution kernels to perform convolution processing on all the input second feature maps, and then concatenates all the convolutional feature maps obtained after each second feature map convolution processing to obtain the corresponding fused feature map.

[0194] Furthermore, the expression for the fused feature map is:

[0195] (2)

[0196] in, Indicates the first The fused feature map corresponding to the second feature map Indicates the first The features of all convolutional feature maps corresponding to the second feature map are concatenated. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map.

[0197] Then, the mean and variance of all fusion feature values ​​in each fusion channel of each fusion feature map are calculated, and the attention weights corresponding to each fusion channel of each fusion feature map are generated using the mean and variance of all fusion feature values.

[0198] Furthermore, the expression for calculating the mean of all fusion feature values ​​in the fusion channel is:

[0199] (3)

[0200] in, Indicates the first The fused feature map corresponding to the second feature map is the first The mean of all fusion feature values ​​in each fusion channel, Indicates the first The height of the fused feature map corresponding to the second feature map. Indicates the first The width of the fused feature map corresponding to each second feature map. Indicates the first The fusion feature map corresponding to the second feature map is the first The first fusion channel Line number The fusion feature value of the column.

[0201] Furthermore, the expression for calculating the variance of all fusion feature values ​​in the fusion channel is: (4)

[0202] in, Indicates the first The fused feature map corresponding to the second feature map is the first The variance of all fused feature values ​​in each fused channel.

[0203] Finally, each attention weight is weighted with its corresponding fusion feature map to obtain a weighted fusion feature map for each fusion channel of each fusion feature map.

[0204] Furthermore, the expression for the weighted fusion feature map is:

[0205] (5)

[0206] in, Indicates the first The fusion feature map corresponding to the second feature map is the first The weighted fusion feature map corresponding to each fusion channel Indicates the first The fusion feature map corresponding to the second feature map is the first Attention weights corresponding to each fusion channel.

[0207] Based on statistical analysis of the mean and variance of all fusion feature values ​​in each fusion channel, attention weights are generated for each fusion channel through a series of transformations. These attention weights are then weighted and summed with the fusion feature map obtained after multi-scale convolution to obtain a weighted fusion feature map. These operations enhance the focus on important feature channels, suppress irrelevant information, and improve the quality of the feature map constructed in this embodiment. The network model's ability to extract target features in complex scenarios improves the accuracy of target detection.

[0208] Figure 6 'a' in the context is The diagram illustrates a convolutional module, representing multi-branch feature processing, with each branch passing through multiple... The convolutional modules are then concatenated and output. Figure 6 b in A schematic diagram of a convolutional module, illustrating feature enhancement with residuals, after... The convolution module processes the input and adds it to the original input. Figure 6 c in The diagram illustrates a convolutional module, showing the fusion of multiple branch convolutions, with each branch passing through multiple... The convolutional kernels are rearranged and spliced, and then... The output of the convolution kernel.

[0209] Depend on Figure 3 and Figure 4 It can be seen that the system constructed in this embodiment... In the neck network of the network model, the 13th, 19th, and 22nd convolutional modules are all... The convolutional module replaces the original Convolutional modules enhance the expressive power of low-level features and their integration with lower-level semantics. Introducing cross-scale convolutional kernel combinations strengthens the receptive field, effectively improving... The ability of network models to identify targets in complex backgrounds.

[0210] Depend on Figure 3 and Figure 4 It can be seen that the system constructed in this embodiment... In the head network of the network model, the 23rd, 24th, and 25th convolutional modules all use... The convolutional module replaces the original Convolutional modules. These three convolutional modules are used for target detection at different scales, although... The detection heads of the convolutional modules have the same structure, but their input features come from different feature fusion results.

[0211] The 23rd convolutional module is used to receive high-resolution feature maps (from...). It contains more spatial details, such as rich edge and texture information, making it suitable for detecting small targets.

[0212] The 24th convolutional module is used to receive medium-resolution feature maps (from...). It can take into account both spatial details and semantic abstraction, making it suitable for detecting medium-sized targets.

[0213] The 25th convolutional module is used to receive low-resolution feature maps (from...). It has rich semantic information and a large sensing field, making it suitable for detecting large targets.

[0214] Each detection head outputs the bounding box parameters, confidence level, and class probability of the target. During the inference stage, the final detection result is generated by weighted fusion and non-maximum suppression of the outputs of the three detection heads, thus achieving high-precision detection of multi-scale targets.

[0215] like Figure 7 As shown, The convolution module performs the following operations:

[0216] The first step is to perform a shared convolution operation on each input third feature map and its corresponding shared convolution weights to generate an output feature map for each third feature map.

[0217] Furthermore, the expression for the output feature map is:

[0218] (6)

[0219] in, Indicates the first The output feature map corresponding to the third feature map. Indicates the first Shared convolution operation performed on the third feature map Indicates the first The shared convolution weights corresponding to the third feature map This represents the convolution operation. Indicates the first The third feature map, This indicates the bias term.

[0220] The convolution module employs a shared convolution weight mechanism, where multiple convolution paths reuse the same shared convolution weights, which not only reduces redundant parameters but also improves computational efficiency.

[0221] The design of formula (6) enables the construction of this embodiment. The network model performs feature extraction with fewer parameters, and can significantly reduce resource consumption, especially when running on edge computing platforms.

[0222] The second step is to divide all output channels of each output feature map into multiple output channel groups, and calculate the mean and variance of the feature values ​​of all output channels in each output channel group.

[0223] The third step is to use the mean and variance of the feature values ​​of all output channels in each output channel group to normalize the feature values ​​of all output channels in the corresponding output channel group, so as to obtain the normalized data distribution of each output channel group.

[0224] The normalized data distributions of all output channel groups in each output feature map are concatenated to obtain the normalized feature map corresponding to each output feature map.

[0225] Furthermore, the expression for the normalized feature map is:

[0226] (7)

[0227] in, Indicates the output feature map The corresponding normalized feature map, Indicates the first The first learnable parameter of the group output channel group. Indicates the first The mean of the eigenvalues ​​of all output channels in the output channel group. Indicates the first The variance of the eigenvalues ​​of all output channels in the output channel group. This represents a constant used to prevent division by zero errors. Indicates the first The second learnable parameter of the group output channel group.

[0228] To address the instability issue during training with small batches of data, Convolution module introduction Normalization method. Make The convolutional module can maintain stability under different training conditions, thus improving training efficiency.

[0229] The fourth step is to perform global average pooling on all scale features of each normalized feature map to generate weighted coefficients for each scale feature, and then perform weighted fusion processing on all scale features to obtain the weighted fused features of each normalized feature map.

[0230] Furthermore, the expression for the weighted fusion feature is:

[0231] (8)

[0232] in, Represents normalized feature maps The weighted fusion characteristics, Represents normalized feature maps The Individual scale features Represents normalized feature maps The The weighting coefficients corresponding to each scale feature.

[0233] Here, the weighting coefficients The weighting is calculated through global average pooling of the normalized feature map and adjusted based on the contribution of the convolutional kernel at each scale. This weighting mechanism enhances the detection capability for small targets while improving the recognition accuracy of multi-scale targets.

[0234] The fifth step involves using a classification network to decouple all weighted fusion features and predict the corresponding target category probability.

[0235] The sixth step involves using a regression network to decouple all weighted fusion features and predict the bounding box coordinates of the corresponding targets.

[0236] In steps five and six, through decoupling, Convolutional modules can more effectively reduce mutual interference between different processing steps, thereby improving the accuracy of classification and regression.

[0237] In summary, the implementation of this embodiment was completed on the edge computing platform (Jetson Xavier NX) for the drone. Optimized deployment of the network model. The edge computing platform uses Ubuntu 18.04 as its operating system. The number of parameters in the network model is approximately the same as the original The network speed is reduced by 80%, and the video memory usage is reduced by 15%. This lightweight design... The network model can significantly reduce the inference latency of the edge computing platform while maintaining the same detection accuracy, and improve deployment efficiency and energy efficiency.

[0238] In order to meet the unattended operation requirements of the UAV during the automatic target detection and tracking process, this embodiment configures a self-starting script and an automatic screen recording mechanism on the edge computing platform to ensure that the flight control system can automatically perform target detection tasks and video recording operations after the UAV starts.

[0239] Once the drone is started, the edge computing platform's startup script will automatically run the target detection program after the flight control system has fully started (with a delay of approximately 10 seconds). First, the script sets environment variables to ensure that the required Python libraries (such as pyrealsens2, cv2, etc.) are correctly loaded and navigates to the directory where the target detection program is located. Next, the script will start the system built in this embodiment. Network model. This script extracts each frame from the real-time video stream and uses... The network model performs target detection, and the target detection bounding boxes are superimposed on the image in real time, outputting the detection results, including the target's location, category, and confidence level. In this way, the flight control system can continuously perform target detection tasks during UAV flight and provide target detection bounding boxes in real time.

[0240] Once the drone is started, the edge computing platform's self-starting script will begin video recording simultaneously with the target detection program, after the flight control system has fully started (with a delay of approximately 10 seconds). First, the script will automatically create a directory to save the videos. The script then begins recording a video stream in real time. To ensure the recorded video stream matches the current screen resolution, the script uses... The command dynamically retrieves the current screen resolution and sets the recording parameters. Video streaming is recorded at a frequency of 15 frames per second. The parameters capture the current screen, set the recording time, and generate a new video file every 60 seconds. Recorded content is saved in the directory. The script automatically names video clips with timestamps, ensuring that each clip doesn't overwrite previous recordings. Furthermore, it's configured to capture a shutdown signal to stop recording, ensuring that recorded files are safely closed when the flight control system shuts down or the program terminates, preventing data loss or file corruption.

[0241] Step S103 of this embodiment: Using the optimized The network model performs real-time detection on all actual images acquired by the binocular depth camera, and obtains the center pixel coordinates of the target detection box for each actual image. Step S103 in this embodiment may include the following sub-steps:

[0242] Sub-step S1031: Send all actual images acquired by the binocular depth camera to the edge computing platform.

[0243] Sub-step S1032: Utilize the optimized The network model processes each actual image received by the edge computing platform in real time to obtain the coordinates of the corresponding target detection box.

[0244] Sub-step S1033: Based on the coordinates of all target detection boxes, calculate the center pixel coordinates of each target detection box. The center pixel coordinates include the horizontal and vertical pixel coordinates of the center point.

[0245] Furthermore, the expression for the horizontal pixel coordinates of the center point of the target detection box is:

[0246] (9)

[0247] in, Indicates the first The horizontal pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The left boundary of the target detection box corresponding to the actual image at time t. Indicates the first The right boundary of the target detection box corresponding to the actual image at time t.

[0248] Furthermore, the expression for the vertical pixel coordinates of the center point of the object detection box is:

[0249] (10)

[0250] in, Indicates the first The vertical pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The upper boundary of the target detection box corresponding to the actual image at time t. Indicates the first The lower boundary of the target detection box corresponding to the actual image at time t.

[0251] Step S104 of this embodiment: Using the pixel coordinates of the center points of all target detection boxes and the calibration parameters of the binocular depth camera, calculate the target control parameters corresponding to each actual image. Step S104 of this embodiment may include the following sub-steps:

[0252] Sub-step S1041: Calculate the target depth value of the stereo depth camera using the calibration parameters of the stereo depth camera and based on the principle of vision.

[0253] Furthermore, the calibration parameters of the binocular depth camera include the horizontal field of view and the horizontal resolution of the image. In this embodiment, the binocular depth camera is a RealSense D435i.

[0254] Furthermore, the target depth value of the binocular depth camera is calculated based on visual principles. The expression for the target depth value is:

[0255] (11)

[0256] in, This represents the target depth value of the stereo depth camera. This indicates the focal length of the binocular depth camera. This indicates the baseline distance of the binocular depth camera. This represents the parallax of a binocular depth camera.

[0257] Sub-step S1042: Calculate the lateral deflection angle of the target in the corresponding actual image using the pixel coordinates of the center point of the target detection box corresponding to each actual image, the optical center coordinates of each actual image, and the calibration parameters of the binocular depth camera.

[0258] Furthermore, the expression for the lateral deflection angle of the target is:

[0259] (12)

[0260] in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at a given time. This represents the horizontal half-resolution of the image from a binocular depth camera, which is half the width of the actual image. It is used to normalize pixel offsets and represents the horizontal offset of the target center point relative to the optical center in the actual image plane, linearly mapped to angular space according to the field of view. Indicates the first The horizontal pixel offset of the center point of the target detection box corresponding to the actual image at each time step. , Indicates the first The x-coordinate of the optical center of the actual image at that moment. This indicates the horizontal field of view of the binocular depth camera. This indicates the horizontal resolution of the image from the binocular depth camera.

[0261] Sub-step S1043: Using the target depth value, convert the center point pixel coordinates of the target detection box corresponding to each actual image into three-dimensional coordinates in the camera coordinate system.

[0262] Furthermore, the camera coordinate system The expression for the axis coordinates is:

[0263] (13)

[0264] in, Indicates the first The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image at a given time. Axis coordinates Indicates a stereo depth camera Focal length along the axial direction.

[0265] Furthermore, the camera coordinate system The expression for the axis coordinates is:

[0266] (14)

[0267] in, Indicates the first The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image at a given time. Axis coordinates Indicates the first The optical center ordinate of the actual image at that moment. Indicates a stereo depth camera Focal length along the axial direction.

[0268] Furthermore, the target depth value is used as the camera coordinate system corresponding to the center pixel coordinates of the target detection box in each actual image. Axis coordinates.

[0269] Sub-step S1044: Combine the 3D coordinates of each camera coordinate system with the corresponding lateral deflection angle of the target to form the target control parameters corresponding to the actual image; the expression for the target control parameters is: ,in, Indicates the first The target control parameters corresponding to the actual image at a given time.

[0270] The target control parameters are four-dimensional, and are subject to detection noise and transient fluctuations. To improve the stability of the control data, the target control parameters need to be further processed.

[0271] like Figure 8 As shown, step S104 in this embodiment further includes:

[0272] The network model compresses all acquired real-world images and transmits the compressed images to the ground control station in real time via User Datagram Protocol (UDP). Specifically, after each frame of image detection is completed, the flight control system overlays target detection boxes and recognition information onto the original image, generating an image with the detection results. To reduce network transmission latency and bandwidth consumption, a [specific method / mechanism] is adopted. The format compresses and encodes the processed image to obtain a byte stream data. Then, through... The communication interface sends the compressed image as data packets to the designated IP address and port of the ground control station, enabling the ground control station to remotely monitor the UAV in real time.

[0273] The communication interface is based on standards. For API development, the sending end should be uniformly set to... The ground control station listens to the corresponding port, receives and decodes the data to reconstruct the detection results.

[0274] The ground control station stores all received images and remotely controls the UAV. The ground control station communicates with the UAV's flight control system.

[0275] Step S105 of this embodiment: Each target control parameter is sequentially filtered, anomaly removed, and smoothed to obtain the corresponding stable control parameters, and all stable control parameters are transmitted to the UAV's flight control system. Step S105 of this embodiment may include the following sub-steps:

[0276] Sub-step S1051: Adaptively adjust the filtering process based on the target depth value. This includes:

[0277] when When the distance is 1 meter, it indicates that the target and the UAV are at a long distance. The corresponding target control parameters are filtered using a sparse sliding window. At this time, the range of the sparse sliding window is from the 6th frame to the 10th frame.

[0278] when When the distance is 1 meter, it indicates that the target and the UAV are at close range. The corresponding target control parameters are filtered using a continuous sliding window. At this time, the range of the continuous sliding window is from frame 1 to frame 5.

[0279] Frames 1 to 5 and frames 6 to 10 all represent the buffer index numbers relative to the actual image of the current frame. Specifically, frame 1 represents the buffer index number of the most recent actual image before the current frame, frame 5 represents the buffer index number of the 5th actual image before the current frame, frame 6 represents the buffer index number of the 6th actual image before the current frame, and frame 10 represents the buffer index number of the 10th actual image before the current frame.

[0280] Furthermore, the expression for filtering the lateral deflection angle of the target is as follows:

[0281] (15)

[0282] in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at time [time] after filtering. This represents the total number of all actual images. Indicates the first Time, Number The weights corresponding to the actual image of the frame. Indicates the first Time, Number The lateral tilt angle of the target corresponding to the actual image of the frame.

[0283] Furthermore, regarding the camera coordinate system The expression for filtering the axis coordinates is:

[0284] (16)

[0285] in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates.

[0286] Furthermore, regarding the camera coordinate system The expression for filtering the axis coordinates is:

[0287] (17)

[0288] in, Indicates the first The actual image at a given time, after filtering, in the camera coordinate system. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates.

[0289] Furthermore, regarding the camera coordinate system The expression for filtering the axis coordinates is:

[0290] (18)

[0291] in, Indicates the first The actual image at a given time, after filtering, in the camera coordinate system. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates.

[0292] Sub-step S1052: The anomaly removal process includes:

[0293] The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as The camera coordinate system corresponding to the filtered image of the previous frame. Axis coordinates are marked as ;

[0294] The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as The camera coordinate system corresponding to the current frame's actual image after filtering. Axis coordinates are marked as ;

[0295] Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , Calculate the camera coordinates corresponding to the current frame's actual image and the previous frame's actual image after filtering. Changes in axis coordinates , ;

[0296] Set the camera coordinate system separately Reasonable threshold for changes in axis coordinates Reasonable threshold for the change of axis coordinates and The reasonable threshold for the change in axis coordinates. If any change exceeds the corresponding reasonable threshold, the current frame's actual image is determined to be an abnormal jump. In this case, the current frame's actual image is discarded, and the filtering result corresponding to the previous frame's actual image is used to replace the current frame's actual image's filtering result.

[0297] in, Reasonable threshold for the change in axis coordinates express, Reasonable threshold for the change in axis coordinates express, Reasonable threshold for the change in axis coordinates express.

[0298] Sub-step S1053: If all changes do not exceed the corresponding reasonable threshold, then the filtering result of the target control parameter corresponding to the actual image of the current frame is smoothed to obtain the corresponding stable control parameter.

[0299] Sub-step S1054: Transmit all stability control parameters to the flight control system in real time via serial communication. This process includes:

[0300] The communication hardware interface adopts the edge computing platform. The serial port baud rate is set to 115200, and the communication format is 8-N-1.

[0301] The data encapsulation method involves encapsulating four 32-bit floating-point numbers, i.e. Combined with a 1-bit target status flag (i.e., 1 for target presence, 0 for target loss), using... The method uses structured packetization to form fixed-length binary data frames. The data frame format is: frame header (2 bytes, fixed value 0xAA 0xAA) + (4 bytes)+ (4 bytes)+ (4 bytes)+ (1 byte) + frame tail (2 bytes, fixed value 0x55 0x55), total length is fixed.

[0302] In terms of transmission strategy, after completing the filtering and anomaly removal processing of each frame of valid image data, the system transmits a complete frame of image data in real time. If a change in target status is detected (e.g., from "missing" to "present"), an immediate data transmission is forcibly triggered to update the image information of the flight control system and ensure real-time status synchronization. At the receiving end, the flight control system performs data frame boundary identification based on the frame header and frame tail, parses out the four-dimensional pose parameters and target status flags, and dynamically adjusts the UAV's attitude to achieve continuous tracking of ground targets.

[0303] In addition, to enhance the robustness of the flight control system, when no target is detected or detection fails in a certain frame of the actual image, the flight control system will automatically call the effective control parameters of the previous frame of the actual image for inherited control, maintain the stable operation of the UAV, and avoid sudden motion changes caused by detection anomalies in a single frame of the actual image.

[0304] Step S106 of this embodiment: The flight control system controls the UAV to track the target based on all stability control parameters, the real-time flight status of the UAV, and the actual position information of the target. Step S106 of this embodiment may include the following sub-steps:

[0305] Sub-step S1061: When the target and the UAV are at a long distance, the UAV adjusts the target's heading angle according to the heading angle at each moment.

[0306] Furthermore, the expression for the target heading angle is:

[0307] (19)

[0308] in, Indicates that the drone is in The target heading angle at any given moment, Indicates that the drone is in The heading angle at any given moment.

[0309] when When, it indicates the first The drone constantly adjusts its target heading angle clockwise.

[0310] when When, it indicates the first The drone constantly adjusts its target heading angle counterclockwise.

[0311] when When, it indicates the first The drone maintains its current heading at all times.

[0312] Sub-step S1062: When the target and the UAV are at close range, calculate the horizontal yaw angle and vertical pitch angle of the target relative to the UAV in each actual image, and combine the heading angle and pitch angle of the UAV at each moment to calculate the target's expected attitude at the corresponding moment. The target's expected attitude includes the target yaw angle and the target pitch angle.

[0313] Furthermore, the expression for the target's horizontal yaw angle relative to the UAV is:

[0314] (20)

[0315] in, Indicates the first The horizontal yaw angle of the target relative to the drone in the actual image at that moment.

[0316] Furthermore, the expression for the target's vertical pitch angle relative to the UAV is:

[0317] (twenty one)

[0318] in, Indicates the first The vertical pitch angle of the target relative to the UAV in the actual image at that moment.

[0319] Furthermore, the expression for the target yaw angle is:

[0320] (twenty two)

[0321] in, Indicates the first The target yaw angle at any given moment.

[0322] Furthermore, the expression for the target pitch angle is:

[0323] (twenty three)

[0324] in, Indicates the first The target pitch angle at any given moment, Indicates that the drone is in The pitch angle at any given moment.

[0325] Sub-step 1063: The UAV tracks the target according to the target's desired attitude.

[0326] when When, it indicates the first The target yaw angle is corrected to the right at any given moment;

[0327] when When, it indicates the first The target yaw angle is corrected to the left at any given moment;

[0328] when When, it indicates the first The target's pitch angle should be corrected upwards at all times;

[0329] when When, it indicates the first The target's pitch angle is constantly corrected downwards;

[0330] when At that time, and When, it indicates the first The drone immediately enters the close-range tracking phase of the target.

[0331] To verify the effectiveness and superiority of the UAV target detection and tracking method proposed in this application, the following simulation experiment was conducted.

[0332] This simulation experiment was designed for the core improved module, namely... Convolutional module Convolutional modules and Ablation experiments on convolutional modules. By enabling and disabling different combinations of convolutional modules, the performance changes of four indicators of the flight control system—detection accuracy, precision, number of parameters, and computational cost—were evaluated. Details are shown in Table 1 below:

[0333] Table 1: Performance Changes of Four Indicators for Evaluating Flight Control Systems

[0334]

[0335] The following conclusions can be drawn from the experimental results:

[0336] First, in Table 1, 0 indicates disabled, and 1 indicates enabled. Compared to the existing... The network constructed in this embodiment The number of parameters and computational cost of the network model have both decreased significantly. For example, enabling it alone... When using the convolution module, the number of parameters is... Descending to The amount of computation is from Descending to For example, enabling it separately. The convolution module has a parameter count of [number missing]. Descending to The amount of computation is from Descending to This demonstrates that by activating these core improvement modules, the flight control system exhibits excellent lightweighting effects.

[0337] Secondly, enable in the head network The convolution module shows a more significant reduction in both the number of parameters and computational cost; the number of parameters is reduced from... Descending to The amount of computation is from Descending to Meanwhile, the flight control system can still maintain good detection accuracy, with a detection accuracy of [missing information]. This indicates Convolutional modules improve While maintaining a lightweight network model, it effectively preserves detection performance, validating the effectiveness of the method employed in the head network in this application. The rationality of the convolution module.

[0338] Finally, when multiple core improvement modules are jointly optimized, the following trend in overall performance can be observed:

[0339] First, activate simultaneously Convolutional modules and When using convolutional modules, The detection accuracy of the network model reaches The accuracy rate reached And the number of parameters is from Descending to The amount of computation is from Descending to It demonstrates a good balance between performance and resources.

[0340] Second, activate simultaneously Convolutional modules and When using the convolution module, the number of parameters is... Descending to The amount of computation is from Descending to The detection accuracy is... .

[0341] Third, activate simultaneously Convolutional modules and When using the convolution module, the number of parameters is... Descending to The detection accuracy is The accuracy rate is This demonstrates the excellent balance between lightweight design and performance.

[0342] Fourth, activate simultaneously Convolutional module Convolutional modules and When using convolutional modules, The network model achieves maximum lightweightness, with the number of parameters reduced from [previous value]. Descending to The amount of computation is from Descending to At the same time, it maintained a relatively ideal level in terms of both detection accuracy and precision, with a detection accuracy of [missing information]. The accuracy rate is This fully demonstrates the deployment of edge computing platforms on drones. The efficiency and practicality of network models.

[0343] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0344] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0345] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and such modifications or substitutions should all be covered within the scope of protection of this application.

[0346] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for target detection and tracking of unmanned aerial vehicles (UAVs), characterized in that, The method includes the following steps: A historical image dataset collected by a drone is acquired, and the historical image dataset is preprocessed to obtain a standard image dataset; the drone is equipped with a binocular depth camera. Build The network model utilizes the standard image dataset to analyze the... The network model is optimized, and the optimized model is then used to further optimize the network model. The network model is deployed on the edge computing platform carried by the drone; The network model comprises: a trunk network, a neck network and a head network connected in sequence. The network model comprises: a trunk network, a neck network and a head network connected in sequence. The backbone network comprises 11 convolutional modules connected unidirectionally in sequence: the 0th convolutional module is... Convolutional module, the first convolutional module is The convolutional module and the second convolutional module are The convolutional module, the third convolutional module is The convolutional module, the fourth convolutional module is The convolutional module, the 5th convolutional module is The convolutional module, the 6th convolutional module is The convolutional module, the 7th convolutional module is The convolutional module, the 8th convolutional module is The convolutional module, the 9th convolutional module is The convolutional module, the 10th convolutional module is Convolutional module; The neck network comprises 12 convolutional modules connected unidirectionally in sequence: the 11th convolutional module is... The convolutional module, the 12th convolutional module is The convolutional module, the 13th convolutional module includes The convolutional module, the 14th convolutional module is The convolutional module, the 15th convolutional module is The convolutional module, the 16th convolutional module is The convolutional module, the 17th convolutional module is The convolutional module, the 18th convolutional module is The convolutional module, the 19th convolutional module is The convolutional module, the 20th convolutional module is The convolutional module, the 21st convolutional module is The convolutional module, the 22nd convolutional module is Convolutional module; The head network comprises 3 convolution modules: the 23rd convolution module, the 24th convolution module and the 25th convolution module are all convolution modules; Specifically, the fourth convolutional module in the backbone network is unidirectionally connected to the fifteenth convolutional module in the neck network, the sixth convolutional module in the backbone network is unidirectionally connected to the twelfth convolutional module in the neck network, and the tenth convolutional module in the backbone network is unidirectionally connected to the eleventh and twenty-first convolutional modules in the neck network, respectively. The 13th convolutional module in the neck network is unidirectionally connected to the 18th convolutional module; the 16th convolutional module in the neck network is unidirectionally connected to the 23rd convolutional module in the head network; the 19th convolutional module in the neck network is unidirectionally connected to the 24th convolutional module in the head network; and the 22nd convolutional module in the neck network is unidirectionally connected to the 25th convolutional module in the head network. Using the optimized The network model performs real-time detection on all actual images acquired by the binocular depth camera and obtains the center pixel coordinates of the target detection box for each actual image. Using the center point pixel coordinates of all the target detection boxes and the calibration parameters of the binocular depth camera, the target control parameters corresponding to each actual image are calculated respectively. Each of the target control parameters is sequentially filtered, anomaly removed, and smoothed to obtain the corresponding stable control parameters, and all the stable control parameters are transmitted to the flight control system of the UAV. The flight control system controls the UAV to track the target based on all the stability control parameters, the real-time flight status of the UAV, and the actual position information of the target; The historical image dataset includes multiple historical images; the standard image dataset includes multiple standard images. The The convolution module performs the following operations: The The convolution module uses three different sizes of convolution kernels to perform convolution processing on all the input second feature maps respectively, and concatenates all the convolution feature maps obtained after each second feature map is processed by the convolution to obtain the corresponding fused feature map; The expression for the fused feature map is: (2) in, Indicates the first The fused feature map corresponding to the second feature map Indicates the first The features of all convolutional feature maps corresponding to the second feature map are concatenated. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Convolution operation is performed on the second feature map. Indicates the use of size The convolution kernel for the first Perform convolution operation on the second feature map; Calculate the mean and variance of all fusion feature values ​​in each fusion channel of each fusion feature map, and use the mean and variance of all fusion feature values ​​to generate attention weights corresponding to each fusion channel of each fusion feature map. The expression for calculating the mean of all the fusion feature values ​​in the fusion channel is: (3) in, Indicates the first The fused feature map corresponding to the second feature map is the first The mean of all fusion feature values ​​in each fusion channel, Indicates the first The height of the fused feature map corresponding to the second feature map. Indicates the first The width of the fused feature map corresponding to each second feature map. Indicates the first The fusion feature map corresponding to the second feature map is the first The first fusion channel Line number The fusion feature value of the column; The expression for calculating the variance of all the fusion feature values ​​in the fusion channel is: (4) wherein, represents the variance of all fusion feature values in the i-th fusion channel of the fusion feature map corresponding to the j-th second feature map; the variance of all fusion feature values in the i-th fusion channel of the fusion feature map corresponding to the j-th second feature map;​ Each attention weight is weighted by the corresponding fusion feature map to obtain a weighted fusion feature map corresponding to each fusion channel of each fusion feature map; The expression for the weighted fusion feature map is: (5) in, Indicates the first The fusion feature map corresponding to the second feature map is the first The weighted fusion feature map corresponding to each fusion channel Indicates the first The fusion feature map corresponding to the second feature map is the first Attention weights corresponding to each fusion channel.

2. The method of claim 1, wherein, The preprocessing includes sequentially performing data cleaning and filtering, data augmentation, and data adjustment and annotation on all the historical images. The data cleaning and filtering process includes: Based on the area constraint rule, all historical images in the historical image dataset that are abnormally labeled, unidentifiable, have distorted viewpoints, or have cluttered backgrounds are removed to obtain a normal image dataset. The data augmentation process includes: Each normal image in the normal image dataset is rotated, scaled, and adjusted sequentially, and image compositing technology is introduced to enhance the background of all the normal images to obtain an enhanced image dataset. The data adjustment and annotation process includes: All enhanced images in the enhanced image dataset are adjusted to standard size, and the category and location information of the target in each enhanced image are labeled using a labeling tool to generate corresponding labeling files, thus obtaining the corresponding standard images. All the standard images constitute the standard image dataset.

3. The method of claim 1, wherein, The The convolution module performs the following operations: The The convolution module respectively converts all the input first feature maps into corresponding main feature maps, and respectively performs linear operation on each main feature map to generate a plurality of auxiliary feature maps corresponding to each first feature map. respectively, and all splicing operation results are taken as the output of the convolution module The The expression of the overall computation of the convolution module is: （1） in, express The overall computational cost of the convolution module, express The number of all input channels in the convolution module, This indicates the number of all principal feature maps. This indicates the size of the convolution kernel in the main feature map. Indicates the height of the main feature map. Indicates the width of the main feature map. This indicates the number of all auxiliary feature maps. express Mapping scaling factor, Indicates the height of the auxiliary feature map. Indicates the width of the auxiliary feature map. , All the main feature maps have the same height and the same width. All the auxiliary feature maps corresponding to each main feature map have the same height and the same width. The convolution kernels of all the main feature maps have the same size.

4. The method of claim 1, wherein, The The convolution module performs the following operations: Each input third feature map is subjected to a shared convolution operation with its corresponding shared convolution weights to generate an output feature map corresponding to each third feature map. The expression for the output feature map is: (6) in, Indicates the first The output feature map corresponding to the third feature map. Indicates the first Shared convolution operation performed on the third feature map Indicates the first The shared convolution weights corresponding to the third feature map This represents the convolution operation. Indicates the first The third feature map, Indicates the bias term; Each output feature map is divided into multiple output channel groups, and the mean and variance of the feature values ​​of all output channels in each output channel group are calculated. The mean and variance of the feature values ​​of all output channels in each output channel group are used to normalize the feature values ​​of all output channels in the corresponding output channel group, so as to obtain the normalized data distribution of each output channel group. The normalized data distributions of all output channel groups in each output feature map are concatenated to obtain the normalized feature map corresponding to each output feature map. The expression for the normalized feature map is: (7) in, Indicates the output feature map The corresponding normalized feature map, Indicates the first The first learnable parameter of the group output channel group. Indicates the first The mean of the eigenvalues ​​of all output channels in the output channel group. Indicates the first The variance of the eigenvalues ​​of all output channels in the output channel group. This represents a constant used to prevent division by zero errors. Indicates the first The second learnable parameter of the group output channel group; Global average pooling is performed on all scale features of each normalized feature map to generate weighted coefficients for each scale feature. Weighted fusion processing is then performed on all scale features to obtain the weighted fusion features of each normalized feature map. The expression for the weighted fusion feature is: (8) in, Represents normalized feature maps The weighted fusion characteristics, Represents normalized feature maps The Individual scale features Represents normalized feature maps The Weighting coefficients corresponding to each scale feature; The weighted fusion features are decoupled using a classification network, and the corresponding class probability of the target is predicted. The weighted fusion features are decoupled using a regression network, and the bounding box coordinates of the corresponding targets are predicted.

5. The method of claim 1, wherein, The utilization of the optimized The steps of the network model performing real-time detection on all actual images acquired by the binocular depth camera and obtaining the center pixel coordinates of the target detection box for each actual image include: All actual images captured by the binocular depth camera are sent to the edge computing platform; Using the optimized The network model processes each actual image received by the edge computing platform in real time to obtain the coordinates of the corresponding target detection box; Based on the coordinates of all the target detection boxes, calculate the center point pixel coordinates of each target detection box, wherein the center point pixel coordinates include the horizontal pixel coordinates and the vertical pixel coordinates of the center point; The expression for the horizontal pixel coordinates of the center point of the target detection box is: (9) in, Indicates the first The horizontal pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The left boundary of the target detection box corresponding to the actual image at time t. Indicates the first The right boundary of the target detection box corresponding to the actual image at time t; The expression for the vertical pixel coordinates of the center point of the target detection box is: (10) in, Indicates the first The vertical pixel coordinates of the center point of the target detection box corresponding to the actual image at time t. Indicates the first The upper boundary of the target detection box corresponding to the actual image at time t. Indicates the first The lower boundary of the target detection box corresponding to the actual image at time t.

6. The method of claim 5, wherein, The step of calculating the target control parameters corresponding to each actual image using the center point pixel coordinates of all the target detection boxes and the calibration parameters of the binocular depth camera includes: The calibration parameters of the binocular depth camera include the horizontal field of view of the binocular depth camera and the horizontal resolution of the image of the binocular depth camera; The target depth value of the binocular depth camera is calculated based on visual principles. The expression for the target depth value is: (11) in, This represents the target depth value of the stereo depth camera. This indicates the focal length of the binocular depth camera. This indicates the baseline distance of the binocular depth camera. This represents the parallax of a binocular depth camera; Using the center pixel coordinates of the target detection box corresponding to each actual image, the optical center coordinates of each actual image, and the calibration parameters of the binocular depth camera, the lateral deviation angle of the target in the corresponding actual image is calculated. The expression for the lateral deflection angle of the target is: (12) in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at a given time. This indicates the horizontal half-resolution of the image from a binocular depth camera. Indicates the first The horizontal pixel offset of the center point of the target detection box corresponding to the actual image at any given time. , Indicates the first The x-coordinate of the optical center of the actual image at that moment. This indicates the horizontal field of view of the binocular depth camera. This indicates the horizontal resolution of the image from the binocular depth camera; Using the target depth value, the center point pixel coordinates of the target detection box corresponding to each actual image are converted into three-dimensional coordinates in the camera coordinate system; The camera coordinate system The expression for the axis coordinates is: (13) wherein, represents the center point of the target detection frame of the actual image at the time point pixel coordinates of the camera coordinate system corresponding to the axis coordinates, represents the focal length in the direction of the axis of the binocular depth camera; the camera coordinate system of the camera The expression of the axis coordinates is: (14) in, Indicates the first The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image at a given time. Axis coordinates Indicates the first The optical center ordinate of the actual image at that moment. Indicates a stereo depth camera Focal length along the axial direction; The target depth value is used as the camera coordinate system corresponding to the center point pixel coordinates of the target detection box in each actual image. Axis coordinates; The three-dimensional coordinates of each camera coordinate system and the corresponding lateral deflection angle of the target are used to form the target control parameters of the actual image; the expression of the target control parameters is: ,in, Indicates the first The target control parameters corresponding to the actual image at a given time.

7. The method of claim 6, wherein, The steps of sequentially performing filtering, anomaly removal, and smoothing processing on each of the target control parameters to obtain the corresponding stable control parameters, and transmitting all the stable control parameters to the flight control system of the UAV, include: Adaptively adjust the filtering process based on the target depth value, including: when When the distance is 1 meter, it indicates that the target and the UAV are at a long distance. The corresponding target control parameters are filtered using a sparse sliding window. At this time, the range of the sparse sliding window is controlled from the 6th frame to the 10th frame. when When the distance is 1 meter, it indicates that the target and the UAV are in a close range stage. The corresponding target control parameters are filtered using a continuous sliding window. At this time, the range of the continuous sliding window is controlled from frame 1 to frame 5. Frames 1 through 5, and frames 6 through 10, each represent a buffer index number relative to the actual image in the current frame. Specifically, frame 1 represents the buffer index number of the actual image in the first frame preceding the actual image in the current frame, frame 5 represents the buffer index number of the actual image in the fifth frame preceding the actual image in the current frame, frame 6 represents the buffer index number of the actual image in the sixth frame preceding the actual image in the current frame, and frame 10 represents the buffer index number of the actual image in the tenth frame preceding the actual image in the current frame. The expression for filtering the lateral deflection angle of the target is: (15) in, Indicates the first The lateral deflection angle of the target corresponding to the actual image at time [time] after filtering. This represents the total number of all actual images. Indicates the first Time, Number The weights corresponding to the actual image of the frame. Indicates the first Time, Number The lateral tilt angle of the target corresponding to the actual image of the frame; an expression for the filtering of the axis coordinates of the camera coordinate system is: an expression for the filtering of the axis coordinates of the camera coordinate system is: (16) in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates; For the camera coordinate system The expression for the filtering process on the axis coordinates is: (17) in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates; For the camera coordinate system The expression for the filtering process on the axis coordinates is: (18) in, Indicates the first The camera coordinate system corresponding to the actual image at a given time after filtering. Axis coordinates Indicates the first Time, Number The camera coordinate system corresponding to the pixel coordinates of the center point of the target detection bounding box in the actual image of the frame. Axis coordinates; The process of removing anomalies includes: The camera coordinate system corresponding to the previous frame's actual image after the filtering process. Axis coordinates are marked as The camera coordinate system corresponding to the previous frame of the actual image after the filtering process. Axis coordinates are marked as The camera coordinate system corresponding to the previous frame of the actual image after the filtering process. Axis coordinates are marked as ; The camera coordinate system corresponding to the actual image of the current frame after the filtering process. Axis coordinates are marked as The camera coordinate system corresponding to the actual image of the current frame after the filtering process. Axis coordinates are marked as The camera coordinate system corresponding to the actual image of the current frame after the filtering process. Axis coordinates are marked as ; Calculate the camera coordinate system corresponding to the actual image in the current frame and the actual image in the previous frame after the filtering process. Changes in axis coordinates , Calculate the camera coordinate system corresponding to the actual image in the current frame and the actual image in the previous frame after the filtering process. Changes in axis coordinates , Calculate the camera coordinate system corresponding to the actual image in the current frame and the actual image in the previous frame after the filtering process. Changes in axis coordinates , ; Set the camera coordinate system separately Reasonable threshold for changes in axis coordinates Reasonable threshold for the change of axis coordinates and If any change in the axis coordinate exceeds the corresponding reasonable threshold, the actual image of the current frame is determined to be an abnormal jump. In this case, the actual image of the current frame is removed, and the filtering result corresponding to the actual image of the previous frame is used to replace the filtering result corresponding to the actual image of the current frame. Among them, the Reasonable threshold for the change in axis coordinates It means that the Reasonable threshold for the change in axis coordinates It means that the Reasonable threshold for the change in axis coordinates express; If none of the changes exceed the corresponding reasonable threshold for the change, then the filtering result of the target control parameter corresponding to the actual image of the current frame is smoothed to obtain the corresponding stable control parameter. All the aforementioned stability control parameters are transmitted to the flight control system in real time via serial communication.

8. The method of claim 7, wherein, The flight control system controls the UAV to track the target based on all the stability control parameters, the real-time flight status of the UAV, and the actual position information of the target. The steps include: When the target and the UAV are at a long distance, the UAV adjusts the target's heading angle according to the heading angle at each moment; The expression for the target heading angle is: (19) in, Indicates that the drone is in The target heading angle at any given moment, Indicates that the drone is in The heading angle at any given moment; when When, it indicates the first At any given moment, the UAV adjusts the target heading angle clockwise; when When, it indicates the first At any given moment, the UAV adjusts the target heading angle counterclockwise; when When, it indicates the first The drone maintains its current heading at the specified time. When the target and the UAV are at close range, the horizontal yaw angle and vertical pitch angle of the target relative to the UAV in each actual image are calculated respectively, and the target's expected attitude at each moment is calculated by combining the heading angle and pitch angle of the UAV at each moment. The target's expected attitude includes the target yaw angle and the target pitch angle. The expression for the horizontal yaw angle of the target relative to the UAV is: (20) in, Indicates the first The horizontal yaw angle of the target relative to the UAV in the actual image at that moment; The expression for the vertical pitch angle of the target relative to the UAV is: (21) in, Indicates the first The vertical pitch angle of the target relative to the UAV in the actual image at that moment; The expression for the target yaw angle is: (22) wherein, represents the target yaw angle at the time instant t. The expression for the target pitch angle is: (23) in, Indicates the first The target pitch angle at any given moment, Indicates that the drone is in The pitch angle at any given moment; The UAV tracks the target according to the target's desired attitude; When the target yaw angle is corrected to the right at the first time; When the target yaw angle is corrected to the left at the first time; When the target pitch angle is corrected upward at the time . When the target pitch angle is corrected downward at the time the target pitch angle is corrected downward at the time When , and , indicates that the UAV enters a close tracking phase to the target at the moment.

Images