A visual closed loop based crossing machine detection tracking control method and system

Abstract

The application provides a kind of crossing machine detection tracking control method and system based on visual closed loop.The method designs tracking and stabilizing strategy for pan-tilt-free crossing machine platform, which adopts visual closed loop scheme to use the pixel relative deviation of target anchor frame center point and image center point as feedback for closed loop control, so that the image center point of crossing machine camera converges to the target anchor frame center point.The total delay of pixel deviation data from detection to control execution is reduced compared to traditional pan-tilt, and the tracking error is significantly reduced in high-speed target maneuvering scenarios.The application uses YOLO11 target detection network based on LECA convolution layer, changes the serial processing of local feature enhancement branch and channel attention branch to parallel processing, greatly improves the processing efficiency while ensuring sufficient adaptability to small targets, motion blur and light changes, and is suitable for pan-tilt-free crossing machine tracking tasks with high real-time requirements.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing and flight control technology for unmanned aerial vehicles (UAVs), specifically to a visual closed-loop detection, tracking, and control method and system for racing drones. Background Technology

[0002] In recent years, with the booming development of the low-altitude economy, small rotary-wing drones, especially racing drones, have shown great potential in aerial photography, racing, and inspection due to their unique advantages of small size, light weight, and high maneuverability. However, the surge in the number of drones has also brought serious challenges to public safety management. "Black flight" drones pose a serious threat to air traffic order and security in key areas. Air-to-air racing drone tracking technology has become a key requirement for maintaining low-altitude security and has urgent application value in border patrol, key area protection, and countering illegal drones.

[0003] Currently, air-to-air drone tracking mainly relies on onboard visual perception systems, but this faces multiple technical bottlenecks in practical applications. At the visual perception level, target drones in air-to-air scenarios typically exhibit low pixel density, high feature similarity, and significant motion blur in images. General-purpose visual detection models are mostly designed for ground or static scenes, lacking sufficient accuracy and robustness for detecting small air-to-air targets, making them unsuitable for tracking demands under high-speed maneuvering conditions. At the tracking control level, in pursuit of ultimate dynamic performance, racing drones generally adopt a "gimbal-less" design with the camera rigidly connected to the fuselage. This results in strong coupling between the camera's line of sight and the fuselage's motion, making traditional gimbal-based independent tracking and stabilization strategies unsuitable for direct application.

[0004] While some existing technologies attempt to improve tracking performance, limitations remain: Firstly, visual detection models lack adaptability and are not robust to factors such as target scale changes and motion blur in air-to-air scenarios, making it difficult to balance lightweight design with accuracy. Secondly, control strategies are incompatible with platform characteristics; traditional gimbal stabilization and tracking control strategies cannot be directly applied to highly maneuverable, gimbal-less racing drone platforms. Furthermore, existing methods have low integration, often focusing on a single aspect and lacking end-to-end optimization from perception to control, making it difficult to meet real-time requirements.

[0005] Therefore, the problems with the existing technology can be summarized as follows: Low system integration and poor real-time performance are problems. Existing methods mostly focus on single-stage sensing or control, lacking end-to-end optimization from detection to execution. Insufficient coordination between visual detection, decision-making, and control modules makes it difficult to meet real-time requirements, such as high frame rate processing, on embedded platforms. Furthermore, it is difficult to balance lightweight models with detection accuracy, and limited edge computing resources further exacerbate performance bottlenecks.

[0006] Therefore, there is an urgent need in this field for a systematic solution that integrates lightweight, high-precision visual inspection, embedded real-time computing, and control algorithms designed specifically for gimbal-less racing drones, in order to improve tracking stability and real-time performance and fully realize the potential of racing drones in air-to-air missions. Summary of the Invention

[0007] In view of this, the present invention provides a visual closed-loop-based detection and tracking control method and system for racing drones, which is suitable for "gimbal-free" designs where the camera is rigidly connected to the drone body. In air-to-air scenarios, it can improve the detection efficiency, detection accuracy and robustness of small targets to meet the tracking requirements under high-speed maneuvering conditions.

[0008] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0009] A vision-based closed-loop detection and tracking control method for racing drones, applicable to gimbal-less racing drone platforms where the camera is rigidly connected to the drone body, includes: Step 1: The camera captures images of the air-to-air scene; Step 2: Image input is used for target detection based on the YOLO11 architecture target detection network; the convolutional layer Conv in the backbone network of the YOLO11 architecture is replaced by the LECA convolutional layer. The LECA convolutional layer enhances feature details and calibrates channel importance simultaneously by combining the parallel processing and fusion of the local feature enhancement branch and the channel attention branch, combined with the jump connection between the input and output of the parallel branch. Step 3: Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, calculate the control command for the racing drone, so that the center point of the racing drone camera image converges to the center point of the target anchor frame, thus realizing closed-loop tracking control.

[0010] Preferably, in the LECA convolutional layer, both the local feature enhancement branch and the channel attention branch transform the feature dimension to a uniform intermediate channel dimension. Then, change the output dimension to ensure the consistency of the dimensions of the two branches.

[0011] Preferably, in the LECA convolutional layer, after the input features are processed by the convolutional layer and the first BN batch normalization layer, they enter the local enhancement branch and the channel attention branch for parallel computation; The local enhancement branch sequentially passes through a first 1×1 convolutional layer, a second batch normalization layer, a first SiLU activation function layer, a 3×3 grouped convolutional layer, a third batch normalization layer, and a second SiLU activation function layer to refine local features; wherein, the first 1×1 convolutional layer transforms the feature dimension to a uniform intermediate channel dimension. Then, the output dimension is changed by 3×3 grouped convolutional layers; The channel attention branch sequentially passes through an adaptive pooling layer, a second 1×1 convolutional layer, a third SiLU activation function, another third 1×1 convolutional layer, and a Sigmoid activation function layer to generate channel-specific attention weights; wherein, the second 1×1 convolutional layer transforms the feature dimension to a unified intermediate channel dimension. Then, the output dimension is transformed by the third 1×1 convolutional layer; The output of the local enhancement branch is multiplied element-wise with the attention weights to complete the fusion of features and attention, and obtain fused features; The fusion features are connected to the branch inputs to form the output features of the LECA convolutional layer.

[0012] Preferably, the LECA convolutional layer further includes skip connections; if the input feature has the same dimension as the fused feature, then skip connections are performed directly; otherwise, the dimensions are adjusted by a 3×3 convolutional layer and a fourth BN batch normalization layer before connection.

[0013] Preferably, the C3k2 feature extraction module in the backbone network of the YOLO11 architecture is replaced by the Ghostv2 module; the input features entering the Ghostv2 module are first processed by the first Ghost convolutional layer and the parallel DFC convolutional layer, and the parallel outputs are multiplied and then sent to the second Ghost convolutional layer; the output of the second Ghost convolutional layer is added to the input features of the Ghostv2 module to obtain the output of the Ghostv2 module.

[0014] Preferably, in the Ghostv2 module, the outputs of the first Ghost convolutional layer and the DFC convolutional layer are multiplied together and then pass through a depth-separable convolutional layer before entering the second Ghost convolutional layer.

[0015] Preferably, the backbone network of the YOLO11 architecture extracts multi-scale features, the feature fusion network performs multi-scale feature enhancement processing, and then the detection results are output through multiple detection heads; The backbone network includes four stages of feature extraction, each consisting of at least one LECA convolutional layer and a Ghostv2 module stacked together. The input features are processed through the four stages and then processed by the SPPF module to output multi-scale features. The feature fusion network includes the original three-layer fusion stage of the YOLO11 architecture and a new fusion stage connected thereafter. Each layer in the three-layer fusion stage enhances multi-scale feature representation through upsampling, concatenation, and C3k2 and Conv modules. The new fusion stage upsamples the features output by the three-layer fusion stage, concatenates the upsampled features with the features output by the Ghostv2 module in the first stage of the backbone network, and then processes them through a C3k2 module before inputting them into the new detection head for target detection.

[0016] Preferably, step 3 specifically includes: Step 301: Select the target that is closest to the target in the previous frame, and calculate the pixel relative deviation between the center point of the target anchor frame and the center point of the image; Step 302: Calculate the control parameters for the racing drone based on the control mode indicated by the camera installation method. (1) When the camera is coaxially mounted with the fuselage, it will work in GPS mode and the racing drone will fly at a constant speed. Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, the PI control algorithm is used to generate the heading offset control amount, the lateral offset control amount and the throttle offset control amount. (2) If there is an angle between the camera and the fuselage, the drone will operate in attitude mode and accelerate uniformly to compensate for the angle. Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, the PID control algorithm is used to generate the heading control quantity, roll control quantity and throttle control quantity. Step 303: Construct control commands based on control quantities, send them to the flight control board and execute tracking; the flight control board collects the current aircraft attitude and motion state to update the model inference and control algorithm parameters of the target detection network, and returns to step 1.

[0017] Preferably, the method further includes: when no target is detected in step 2, entering the target loss time judgment: if the loss time is less than a set threshold, execute hold control, maintain the control command of the previous frame, take the target of the previous frame as the target position of the current frame, and drive the roll angle of the aircraft to change within a set range to search for the target by scanning left and right; if the loss time is greater than or equal to the set threshold, trigger the safety mode, set the racing aircraft to fly at a set constant speed, expand the search range to recapture the target.

[0018] The present invention also provides a visual closed-loop detection and tracking control system for racing drones, which includes a camera, a target detection network, a control module, and an execution module; There is no gimbal connection between the camera and the drone body; the camera captures air-to-air scene images. The target detection network performs target detection based on images captured by a camera, and the target detection results are sent to the control module. The target detection network is a network structure based on the YOLO11 architecture. In the YOLO11 architecture, the convolutional layer Conv in the backbone network is replaced by the LECA convolutional layer. The LECA convolutional layer enhances feature details and calibrates channel importance simultaneously through the parallel processing and fusion of local feature enhancement branches and channel attention branches, combined with the skip connections between the input and output of the parallel branches. Based on the target detection results, the control module obtains the pixel relative deviation between the center point of the target anchor frame and the center point of the image; using the pixel relative deviation as feedback, it calculates the control commands for the racing drone and controls the execution module to perform actions, so that the center point of the racing drone camera image converges to the center point of the target anchor frame, thereby achieving closed-loop tracking control.

[0019] Beneficial effects: (1) This invention designs a tracking and stabilization strategy for gimbal-less racing drone platforms: This strategy adopts a visual closed-loop scheme, using the relative pixel deviation between the center point of the target anchor frame and the center point of the image as feedback to calculate the control commands for the racing drone, so that the center point of the camera image converges to the center point of the target anchor frame, realizing closed-loop tracking control. The scheme of this invention has no gimbal mechanical delay, and the total delay from pixel deviation data detection to control execution is reduced compared with the traditional gimbal scheme, solving the problem of control command lag or instability during tracking, and ensuring tracking smoothness and accuracy. In high-speed target maneuvering scenarios, the tracking offset error is significantly reduced, and the target miss rate is greatly reduced. Moreover, the gimbal-less racing drone can reduce hardware costs, lighten the overall weight of the body, and increase the endurance, making it more suitable for long-term air-to-air operations.

[0020] (2) This invention provides a more efficient, accurate, and robust real-time visual detection method: In air-to-air scenarios, target UAVs typically exhibit low pixel ratios, high feature similarity, and significant motion blur in images. Meanwhile, racing drones prioritize high maneuverability and require high control efficiency. To address these issues, this invention uses LECA convolutional layers in the YOLO11 architecture. These layers process the local feature enhancement branch and the channel attention branch in parallel, fusing their outputs through element-wise multiplication, and then retaining the original information through skip connections. This design avoids directly weighting degraded features (such as motion blur or areas of abnormal lighting) and instead reconstructs feature representations by enhancing local details and channel importance, thereby more effectively suppressing signal distortion. Moreover, compared to serial structures, parallel processing also solves the problems of information transmission delay and computational accumulation, significantly improving feature reconstruction efficiency. While ensuring sufficient adaptability to small targets, motion blur, and lighting changes, it greatly improves processing efficiency. Therefore, the target detection network of the present invention can improve the detection efficiency, detection accuracy and robustness of small targets in air-to-air scenarios, and is suitable for tracking tasks of gimbal-less racing drones with high real-time requirements.

[0021] Furthermore, the use of the Ghostv2 module reduces computational complexity while maintaining feature expressiveness, achieving a balance between accuracy and lightweight design, further ensuring the detection efficiency and accuracy of small targets; the addition of a detection head allows detection on a larger feature map, preserving more detailed information about the target, thereby achieving accurate detection of even smaller targets.

[0022] A2A-YOLO achieves joint optimization with modules such as Ghostv2 and micro-detection heads by embedding LECA convolutional layers into the detection backbone network, thereby directly improving the model's feature discrimination ability under degradation conditions.

[0023] (3) This invention provides a more efficient and stable gimbal-free control closed loop: The vision-based closed-loop control strategy (including GPS mode and attitude mode) of this invention directly matches the hardware characteristics of the gimbal-free racing drone platform. Heading, roll, and throttle control quantities are generated through a proportional-derivative or proportional-derivative-integral controller, and combined with a graded protection mechanism after target loss (such as scan search and safety mode), stable tracking under high-speed maneuvers is achieved. Furthermore, this invention deeply integrates the perception, decision-making, and control modules, reducing coupling errors between modules through a "detection-decision-control-feedback" closed-loop design. It is suitable for scenarios such as border patrol and key area protection, improving the efficiency of low-altitude safety management.

[0024] (4) This invention adopts a more real-time and efficient embedded deployment: the model is converted through ONNX and RKNN formats, and parallel inference is performed on the RK3588 AI computing board using a thread pool, achieving millisecond-level target detection and deviation calculation. This deployment method fully utilizes edge computing power and ensures the real-time response of the system at high frame rates. Attached Figure Description

[0025] Figure 1 A schematic diagram of the A2A-YOLO model used in the object detection network.

[0026] Figure 2 This is a schematic diagram of the LECA convolutional layer.

[0027] Figure 3 This is a structural diagram of the Ghostv2 module.

[0028] Figure 4 This is a flowchart of a visual closed-loop detection and tracking control method for racing drones according to an embodiment of the present invention. Detailed Implementation

[0029] This invention provides a vision-based closed-loop detection and tracking control scheme for racing drones. The core idea is to rigidly connect the camera to the drone's body instead of using a gimbal, thus avoiding the mechanical delay caused by a gimbal. Based on this structure, the relative pixel deviation between the target anchor frame center point and the image center point is used as feedback to calculate the drone's control commands, causing the camera's image center point to converge towards the target anchor frame center point, achieving closed-loop tracking control. Without the mechanical delay of a gimbal, the total delay from pixel deviation data detection to control execution is reduced compared to traditional gimbal solutions. In scenarios involving high-speed target maneuvering, the tracking offset error is significantly reduced, and the target miss rate is greatly lowered.

[0030] Furthermore, this invention has high requirements for control efficiency. Therefore, this invention uses LECA convolutional layers in the YOLO11 architecture, which process the local feature enhancement branch and the channel attention branch in parallel and then fuse them, instead of the traditional serial processing. While ensuring sufficient adaptability to small targets, motion blur and lighting changes, it greatly improves processing efficiency and is suitable for gimbal-less racing drones with high real-time requirements.

[0031] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] This invention relates to a visual closed-loop detection and tracking control system for racing drones, comprising a camera, a target detection network, a control module, an execution module, and a sensing module. The camera is fixedly connected to the drone's fuselage, without a gimbal structure. The target detection network and control module are deployed within an AI computing board (RK3588), serving as the edge computing core responsible for running the target detection model and control logic. The execution module is implemented using a flight control board. The sensing module includes a gyroscope and / or a GPS module. The execution module is connected to the drone's actuators via the flight control board, and the sensing module is connected to the AI ​​computing board via the flight control board.

[0033] The camera captures air-to-air scene images and sends them to the AI ​​computing board. The target detection network on the AI ​​computing board performs target detection, and the control module calculates pixel deviations based on the target detection results to generate control commands. These generated control commands are sent to the flight control board, where they are converted into PWM signals to drive the actuators. Simultaneously, the flight control board collects the current aircraft attitude and motion status from the gyroscope and / or GPS module, feeding this data back to the AI ​​computing board to verify whether the aircraft's motion status meets expectations and to update the parameters of the model inference and control algorithms.

[0034] Figure 1 The network structure of the target detection network of this invention is shown. The target detection network of this invention is an improvement on the YOLO11 architecture, forming a new network structure called A2A-YOLO. Figure 1 As shown, the A2A-YOLO consists of a backbone network, a feature fusion network, and multiple detector heads. The backbone network is used to extract multi-scale features from the input image, the feature fusion network performs multi-scale feature enhancement processing, and then the detection results are output through multiple detector heads.

[0035] The A2A-YOLO includes three important improvements: The first improvement is to replace the Conv convolutional layer in the original backbone network with LECA (Local Enhanced Channel Attention), and design a parallel processing and fusion structure of local feature enhancement branch and channel attention branch within the LECA convolutional layer. Combined with the jump connection from parallel branch input to output, feature details are enhanced and channel importance is calibrated simultaneously.

[0036] The second improvement is the replacement of the C3k2 feature extraction module in the original backbone network with the Ghostv2 module. The Ghostv2 module enhances feature representation capabilities while reducing computational cost through lightweight feature generation of Ghost convolutional layers, receptive field expansion of DFC convolutional layers, and optional depthwise separable convolutions and residual connections. It is suitable for lightweight deep learning deployment scenarios such as edge devices, including tasks such as drone target detection and mobile device image recognition.

[0037] The third improvement is the addition of a fourth upsampling stage to the feature fusion network, along with a tiny detection head, to perform detection on a larger feature map, retaining more detailed information about the target and thus enabling accurate detection of even smaller targets.

[0038] A2A-YOLO achieves joint optimization with modules such as Ghostv2 and micro-detection heads by embedding LECA convolutional layers into the detection backbone network, thereby directly improving the model's feature discrimination ability under degradation conditions.

[0039] In this embodiment, the backbone network includes four stages of feature extraction (divided into stages by a Ghostv2 module). Each feature extraction stage consists of at least one LECA convolutional layer and one Ghostv2 module stacked together. The input features are then processed by the SPPF module after the four stages of feature extraction to output multi-scale features. Figure 1 As shown, the first stage of the backbone network consists of two LECA convolutional layers and a Ghostv2 module stacked sequentially; the second to fourth stages each consist of a LECA convolutional layer and a Ghostv2 module stacked sequentially, and finally the multi-scale features extracted by the backbone network are output through an SPPF module.

[0040] Regarding the first improvement, Figure 2 This paper presents a deep learning feature extraction LECA convolutional layer structure that integrates local feature enhancement, channel attention mechanism, and adaptive skip connections. As shown in the figure, the input features are processed by a 3×3 convolutional layer and a batch normalization layer, and then enter the local enhancement branch and the channel attention branch for parallel computation.

[0041] The local enhancement branch sequentially enhances local features through a 1×1 convolutional layer, a batch normalization (BN) layer, a SiLU activation function layer, a 3×3 grouped convolutional layer, another BN normalization layer, and a SiLU activation function layer. The channel attention branch sequentially generates channel-wise attention weights through an adaptive pooling layer, a 1×1 convolutional layer, a third SiLU activation function layer, a 1×1 convolutional layer, and a Sigmoid activation function layer.

[0042] Here, both the local feature enhancement branch and the channel attention branch transform the feature dimension to a unified intermediate channel dimension. Then, the changes are applied to the output dimension to ensure dimensional consistency between the two branches. Figure 2 In the structure shown, the 1×1 convolutional layer in the local feature enhancement branch transforms the feature dimension to a uniform intermediate channel dimension. Then, a 3×3 grouped convolutional layer transforms the output dimension; the first 1×1 convolutional layer in the channel attention branch transforms the feature dimension to a unified intermediate channel dimension. Then, the output dimension is changed by the second 1×1 convolutional layer.

[0043] Next, the output of the local enhancement branch is multiplied element-wise with the attention weights to fuse the features and attention, obtaining fused features. This fused feature is then connected to the branch input from the skip connections to form the output features of the LECA convolutional layer.

[0044] Furthermore, the LECA convolutional layer also includes adaptive skip connections if the input features have the same dimension as the fused features, i.e., ... Figure 2 In the example, if the number of channels C, height H, and width W of the branch input and output are completely the same, then a skip connection is directly performed. Otherwise, the dimensions are adjusted by a 3×3 convolutional layer and a BN batch normalization layer before connection, and the final output is a feature map with dimensions B×C×H×W.

[0045] Overall, the LECA convolutional layer enhances feature details through local enhancement branches, calibrates inter-channel information through channel attention branches, and achieves cross-layer information fusion through adaptive skip connections, thereby improving the robustness and effectiveness of feature representation. It is suitable for deep learning scenarios requiring fine feature extraction, such as object detection and image classification. The mathematical expression of the LECA convolutional layer module is as follows: Given input feature map First, intermediate feature maps are obtained through convolution and batch normalization operations. ,in For convolution weights, This represents the convolution operation. Let be the activation function. The formula for calculating the local enhancement branch is:

[0046] in, This is the result of processing locally enhanced branches. and These are the weights of the convolutional layer. This indicates a batch normalization operation.

[0047] The formula for calculating the channel attention branch is:

[0048] in, Indicates global average pooling. and for Convolution weights.

[0049] Finally, the output of the LECA convolutional layer is:

[0050] in, This indicates element-wise multiplication.

[0051] Traditional attention mechanisms (such as CBAM and SEBlock) follow the paradigm of "attention weights directly modulating the original feature map," that is, multiplying the generated attention map element-wise with the input feature map.

[0052] This approach tends to amplify noise and is computationally redundant in complex degradation scenarios. In contrast, the LECA convolutional layer breaks through this inherent paradigm, proposing that "attention parameters do not need to be directly applied to the original feature map, but rather applied to the locally enhanced feature branches." Specifically, the LECA convolutional layer processes the local feature enhancement module and the channel attention gate in parallel, fuses their outputs through element-wise multiplication, and then preserves the original information through skip connections. This design avoids directly assigning weights to degraded features (such as motion blur or regions with abnormal lighting), instead reconstructing feature representations by enhancing local details and channel importance, thereby more effectively suppressing signal distortion.

[0053] The LECA convolutional layer structurally achieves deep synergy between local enhancement and channel attention, a stark contrast to the sequential or independent processing methods of traditional approaches. Its local enhancement branch employs grouped convolutions and small receptive field operations, focusing on extracting edge and texture details from motion-blurred targets; the channel attention gate dynamically calibrates channel contributions through global average pooling and nonlinear convolutional layers, mitigating feature shifts caused by illumination variations. Furthermore, the LECA convolutional layer unifies the intermediate channel dimensions. Ensuring dimensionality consistency between the two types of attention transformations enhances training stability. In contrast, the sequential structure of traditional methods (such as CBAM) leads to information transmission delays and computational accumulation, while the parallel fusion mechanism of the LECA convolutional layer significantly improves feature reconstruction efficiency. Experiments show that the introduction of this module improves the average accuracy (AP) of detection by 6.5% on the Det-Fly dataset with only a 0.5M increase in the number of parameters, while maintaining real-time performance in multi-scale object detection tasks, making it better suited for air-to-air object detection tasks.

[0054] The LECA convolutional layer is specifically designed for motion blur and illumination variation problems in air-to-air UAV detection, and its performance far surpasses general attention methods. A2A-YOLO integrates it into the detection backbone network, achieving joint optimization with modules such as the Ghostv2 module and the miniature detection head, directly improving the model's feature discrimination ability under degradation conditions. For example, on the strong light / weak light test set of the Det-Fly dataset, the A2A-YOLO model with the LECA convolutional layer improves the AP to 76.4%, a 13.0% improvement over YOLO11; in motion blur scenarios, the AP reaches 84.9%, a 5.5% improvement over YOLO11. Furthermore, this module achieves an AP of 86.4% in infrared detection tasks (such as the IRSTD-1K dataset), demonstrating its cross-modal generalization ability.

[0055] Regarding the second improvement, the C3k2 feature extraction module in the backbone network is replaced with the Ghostv2 module. Figure 3 This demonstrates the structure of the Ghostv2 module and the working principle of its internal sub-modules (Ghost convolutional layer and DFC convolutional layer). The Ghostv2 module on the left takes a feature map as input, first feeding it into the first Ghost convolutional layer. It can also optionally pass the map through the DFC convolutional layer in parallel. After multiplying the outputs of the two convolutional layers, it can optionally pass through a depthwise separable convolutional layer before being fed into another Ghost convolutional layer. The output of the second Ghost convolutional layer is added to the initial input features of the Ghostv2 module to obtain the Ghostv2 module output.

[0056] Figure 3 The Ghost convolutional layer in the upper right part receives the input feature map, and after convolution to obtain intermediate features, it fuses the intermediate features with some channels of the original input through skip connections containing multiple Φ transforms to obtain the output feature map. In this way, redundant features are generated through inexpensive transformation to achieve lightweighting. Figure 3 The DFC convolutional layer in the lower right half first downsamples the input feature map, then performs horizontal and vertical fully connected operations respectively, and finally upsamples to obtain the output, expanding the receptive field through separate fully connected operations.

[0057] Overall, the Ghostv2 module enhances feature representation capabilities while reducing computational cost through lightweight feature generation using Ghost convolutional layers and receptive field expansion using DFC convolutional layers, combined with optional depthwise separable convolutions and residual connections. This makes it suitable for lightweight deep learning deployment scenarios such as edge devices, including tasks like drone target detection and mobile device image recognition. The Ghostv2 module is used for lightweight feature extraction, reducing computational cost through depthwise separable convolutions and linear transformations. This architecture significantly reduces model complexity and is suitable for resource-constrained embedded platforms.

[0058] Regarding the third improvement, please refer to [link / reference]. Figure 1 In this embodiment, the feature fusion network enhances multi-scale feature representation through upsampling, concatenation, and C3k2 and Conv modules, ultimately achieving target detection using a micro-detection head and multiple conventional detection heads. Specifically, the feature fusion network includes the original three-layer fusion stage of the YOLO11 architecture, followed by a new fusion stage. Each layer in the three-layer fusion stage enhances multi-scale feature representation through upsampling, concatenation, and C3k2 and Conv modules, with each of the three fusion stages corresponding to a conventional detection head. The new fusion stage upsamples the features output from the three-layer fusion stage, concatenates the upsampled features with the features output from the Ghostv2 module in the first stage of the backbone network, and then processes them through a C3k2 module before inputting them into the new detection head (micro-detection head) to achieve target detection. The detection head optimizes the output layer for small target detection, improving detection accuracy.

[0059] at the same time Figure 1 The upper right corner of the diagram breaks down the internal structure of key sub-modules such as the Conv module, C3k2 module, SPPF module, C2PSA module, and PSA module, clarifying their component composition and data flow. The overall design, through lightweight modules, multi-scale feature fusion, and targeted attention mechanisms, ensures the accuracy of small target detection and environmental robustness in air-to-air scenarios while meeting the real-time deployment requirements of UAV edge computing platforms. It can be used to achieve high-precision real-time detection of UAVs under motion blur and lighting variations.

[0060] Figure 4 This paper illustrates the execution logic of a vision-based closed-loop detection and tracking control system for a racing drone, based on an embodiment of the present invention. Starting with system initialization, the system sequentially completes image acquisition and preprocessing, target detection, control mode decision-making, control command generation and execution, and state updates, forming a closed-loop control process. As shown in the figure, it includes the following steps: Step 1: Perform system initialization.

[0061] In this step, the parameters of the RK3588 AI computing board, flight control board, camera and other modules are configured and communication links are established. The control mode is input according to the camera position. GPS mode is used when the camera is coaxial with the fuselage, and attitude mode is used when there is an angle between the camera and the fuselage.

[0062] The initial parameters for the A2A-YOLO object detection model weights and dual-mode control algorithm are loaded: proportional-integral (PI) parameter control is used in GPS mode, and proportional-integral-derivative (PID) parameter control is used in attitude mode. During A2A-YOLO model deployment, the trained model is exported to ONNX format and converted to RKNN format using rknn-toolkit2. Then, a thread pool deployment algorithm is implemented on RK3588 to build a thread pool for model inference, fully utilizing computing power to deploy multiple model inference threads.

[0063] Step 2: The camera captures air-to-air scene images and performs preprocessing.

[0064] In this embodiment, the camera captures air-to-air scene images at a resolution of 1920×1080 and a frame rate of 60FPS. The RK3588 platform performs preprocessing operations on the images, such as scaling (to 640×640), normalization (mapping pixel values ​​to [0,1]), and channel conversion (BGR to RGB), to provide standardized input for target detection.

[0065] Step 3: Input the image into the A2A-YOLO target detection module for target detection.

[0066] Step 4: Determine if the target has been detected. If it has been detected, proceed to steps 5 through 8; otherwise, proceed to step 8.

[0067] Step 5: If a target is detected, select the target closest to the target in the previous frame and calculate the relative pixel deviation between the target's anchor box center and the image center, including Δx and Δy. Here, Δx is the proportion of the horizontal deviation pixel value to the image width, and Δy is the proportion of the vertical deviation pixel value to the image height.

[0068] Step 6: Calculate the control quantity based on the control mode used, and then execute the steps.

[0069] (1) When the camera is coaxially mounted with the fuselage, it operates in GPS mode and the racing drone flies at a constant speed. Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, a PI (proportional-integral) control algorithm is used to generate heading offset control, lateral offset control, and throttle offset control. Specifically, the control uses two PI controllers. The heading offset control and lateral offset control share a set of PI control parameters, while the throttle offset control uses a different set of control parameters.

[0070] (2) If there is an angle between the camera and the fuselage, the aircraft operates in attitude mode, accelerating uniformly to offset the angle and keep the camera horizontal. Based on the relative pixel deviation between the center point of the target anchor frame and the center point of the image, a PID (proportional-integral-integral) control algorithm is used to generate heading control, roll control, and throttle control. Specifically, two PID controllers are used. The heading control and roll control share a set of proportional-integral-derivative control parameters, while the throttle control uses a different set of PID control parameters.

[0071] Step 7: Construct control commands based on the control quantities, send them to the flight controller board, and execute tracking. Proceed to Step 9.

[0072] In this step, after the control mode decision is completed, commands containing constant speed, heading offset control, lateral offset control, and throttle offset control are generated in GPS mode, and commands containing constant acceleration, heading control, roll control, and throttle control are generated in attitude mode. The flight control board converts the control commands into motor PWM signals, executes the sending of control commands and the execution of tracking operations, drives the quadcopter motors to adjust attitude and speed, and achieves visual closed-loop tracking of air-to-air targets.

[0073] Step 8: If no target is detected, proceed to target loss time determination.

[0074] If the time lost is less than the threshold (e.g., 2s), hold control is executed to maintain the control commands of the previous frame (roll angle, pitch angle, speed / acceleration), and the target of the previous frame is used as the target position of this frame. At the same time, the aircraft is driven to scan left and right (roll angle ±10° cycle) to search for the target.

[0075] If the loss time is greater than or equal to the set threshold, trigger the safety mode, set the racing drone to fly at a constant speed of 5m / s, expand the search area to reacquire the target. Proceed to step nine.

[0076] Step Nine: Enter the update phase.

[0077] The flight control board collects the current aircraft attitude (roll, pitch, and yaw angles output from the gyroscope) and motion state (speed and position output from the GPS module), and feeds this data back to the RK3588 platform to verify whether the aircraft's motion state meets expectations. In the preferred scheme, the model inference and control algorithm parameters of the target detection network can be updated based on the aircraft attitude and motion state. For example, the parameters of the trained target detection network model can be changed according to the forward flight speed to adapt to different flight conditions. Alternatively, the motion fuzzy compensation coefficient can be dynamically adjusted based on the aircraft speed to update the PI / PID parameters of the control algorithm, achieving more efficient and accurate tracking. If tracking continues, the process returns to step two, continuously iterating the above process to ensure real-time and continuous tracking. Otherwise, tracking ends.

[0078] The "continue tracking" option can be determined based on the target distance, the aircraft's motion status, or external commands.

[0079] This concludes the process.

[0080] It is evident that this visual closed-loop control system, through a closed-loop design of "detection-decision-control-feedback," combined with the usage mechanism of GPS mode and attitude mode, and a hierarchical protection mechanism after target loss, achieves stable and robust tracking of FAW targets in air-to-air scenarios. It fully adapts to the technical characteristics of gimbal-less hardware architecture and meets the tracking needs in high-speed maneuvering and complex environments.

[0081] In summary, the solution of the present invention has the following effects: (1) A cheaper and more efficient UAV air-to-air tracking solution. This invention is developed based on a racing drone without a gimbal, which reduces hardware costs, lightens the overall weight of the drone, and increases the endurance, making it more suitable for long-term air-to-air operations. Without the mechanical delay of a gimbal, the total delay from pixel deviation data detection to control execution is reduced compared to traditional gimbal solutions. In scenarios where the target is maneuvering at high speed, the tracking offset error is significantly reduced, and the target miss rate is greatly reduced.

[0082] (2) A more efficient, accurate and robust real-time visual detection method. The A2A-YOLO model, which is optimized for air-to-air scenes, is adopted. The LECA convolutional layer module enhances local feature details and channel attention mechanism, improving the adaptability to small targets, motion blur and illumination changes. At the same time, the Ghostv2 module reduces computational complexity while ensuring feature expressiveness, achieving a balance between accuracy and lightweight design.

[0083] (3) More efficient and stable gimbal-free control closed loop. The control strategy based on the vision closed loop of this invention (including GPS mode and attitude mode) directly matches the hardware characteristics of the gimbal-free racing drone platform. Heading, roll, and throttle control quantities are generated through a proportional-derivative or proportional-derivative-integral controller, and combined with a graded protection mechanism after target loss (such as scan search and safety mode), stable tracking under high-speed maneuvers is achieved. Furthermore, this invention deeply integrates the perception, decision-making, and control modules, reducing coupling errors between modules through a "detection-decision-control-feedback" closed-loop design. It is suitable for scenarios such as border patrol and key area protection, improving the efficiency of low-altitude safety management.

[0084] (4) More real-time and efficient embedded deployment. The model is converted to ONNX and RKNN formats and parallel inference is performed on the RK3588 AI computing board using a thread pool, achieving millisecond-level target detection and bias calculation. This deployment method fully utilizes edge computing power and ensures the system's real-time response at high frame rates.

[0085] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A visual closed-loop-based detection, tracking, and control method for racing drones, applicable to gimbal-less racing drone platforms where the camera and body are rigidly connected, characterized in that... The method includes: Step 1: The camera captures images of the air-to-air scene; Step 2: Image input is used for target detection based on the YOLO11 architecture target detection network; the convolutional layer Conv in the backbone network of the YOLO11 architecture is replaced by the LECA convolutional layer. The LECA convolutional layer enhances feature details and calibrates channel importance simultaneously by combining the parallel processing and fusion of the local feature enhancement branch and the channel attention branch, combined with the jump connection between the input and output of the parallel branch. Step 3: Based on the relative pixel deviation between the center point of the target anchor frame and the center point of the image, calculate the control command for the racing drone, so that the center point of the racing drone camera image converges to the center point of the target anchor frame, thus realizing closed-loop tracking control; Step 3 specifically includes: Step 301: Select the target that is closest to the target in the previous frame, and calculate the pixel relative deviation between the center point of the target anchor frame and the center point of the image; Step 302: Calculate the control parameters for the racing drone based on the control mode indicated by the camera installation method. (1) When the camera is coaxially mounted with the fuselage, it will work in GPS mode and the racing drone will fly at a constant speed. Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, the PI control algorithm is used to generate the heading offset control amount, the lateral offset control amount and the throttle offset control amount. (2) If there is an angle between the camera and the fuselage, the drone will operate in attitude mode and accelerate uniformly to compensate for the angle. Based on the pixel relative deviation between the center point of the target anchor frame and the center point of the image, the PID control algorithm is used to generate the heading control quantity, roll control quantity and throttle control quantity. Step 303: Construct control commands based on control quantities, send them to the flight control board and execute tracking; the flight control board collects the current aircraft attitude and motion state to update the model inference and control algorithm parameters of the target detection network, and returns to step 1.

2. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 1, characterized in that, In the LECA convolutional layer, both the local feature enhancement branch and the channel attention branch transform the feature dimension to a unified intermediate channel dimension. Then, change the output dimension to ensure the consistency of the dimensions of the two branches.

3. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 2, characterized in that, In the LECA convolutional layer, the input features are processed by the convolutional layer and the first batch normalization layer, and then enter the local enhancement branch and the channel attention branch for parallel computation. The local enhancement branch sequentially passes through a first 1×1 convolutional layer, a second batch normalization layer, a first SiLU activation function layer, a 3×3 grouped convolutional layer, a third batch normalization layer, and a second SiLU activation function layer to refine local features; wherein, the first 1×1 convolutional layer transforms the feature dimension to a uniform intermediate channel dimension. Then, the output dimension is changed by 3×3 grouped convolutional layers; The channel attention branch sequentially passes through an adaptive pooling layer, a second 1×1 convolutional layer, a third SiLU activation function layer, another third 1×1 convolutional layer, and a Sigmoid activation function layer to generate channel-wise attention weights; wherein, the second 1×1 convolutional layer transforms the feature dimension to a unified intermediate channel dimension. Then, the output dimension is transformed by the third 1×1 convolutional layer; The output of the local enhancement branch is multiplied element-wise with the attention weights to complete the fusion of features and attention, and obtain fused features; The fusion features are connected to the branch inputs to form the output features of the LECA convolutional layer.

4. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 3, characterized in that, The LECA convolutional layer further includes skip connections; if the input features have the same dimension as the fused features, skip connections are performed directly; otherwise, the dimensions are adjusted by a 3×3 convolutional layer and a fourth BN batch normalization layer before connection.

5. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 1, characterized in that, The C3k2 feature extraction module in the backbone network of the YOLO11 architecture is replaced by the Ghostv2 module. The input features entering the Ghostv2 module are first processed by the first Ghost convolutional layer and the parallel DFC convolutional layer. The parallel outputs are multiplied and then sent to the second Ghost convolutional layer. The output of the second Ghost convolutional layer is added to the input features of the Ghostv2 module to obtain the output of the Ghostv2 module.

6. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 5, characterized in that, In the Ghostv2 module, the outputs of the first Ghost convolutional layer and the DFC convolutional layer are multiplied together and then pass through a depth-separable convolutional layer before entering the second Ghost convolutional layer.

7. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 5, characterized in that, The YOLO11 architecture's backbone network extracts multi-scale features, the feature fusion network performs multi-scale feature enhancement processing, and then outputs detection results through multiple detection heads; The backbone network includes four stages of feature extraction, each consisting of at least one LECA convolutional layer and a Ghostv2 module stacked together. The input features are processed through the four stages and then processed by the SPPF module to output multi-scale features. The feature fusion network includes the original three-layer fusion stage of the YOLO11 architecture and a new fusion stage connected thereafter. Each layer in the three-layer fusion stage enhances multi-scale feature representation through upsampling, concatenation, and C3k2 and Conv modules. The new fusion stage upsamples the features output by the three-layer fusion stage, concatenates the upsampled features with the features output by the Ghostv2 module in the first stage of the backbone network, and then processes them through a C3k2 module before inputting them into the new detection head for target detection.

8. The visual closed-loop-based detection, tracking, and control method for racing drones as described in claim 1, characterized in that, The method further includes: when no target is detected in step 2, entering the target loss time judgment: if the loss time is less than a set threshold, execute hold control, maintain the control command of the previous frame, take the target of the previous frame as the target position of the current frame, and drive the roll angle of the aircraft to change within a set range to search for the target by scanning left and right; if the loss time is greater than or equal to the set threshold, trigger the safety mode, set the racing aircraft to fly at a set constant speed, expand the search range to re-acquire the target.

9. A visual closed-loop-based detection and tracking control system for racing drones, used to execute the visual closed-loop-based detection and tracking control method according to any one of claims 1-8, characterized in that, The system includes a camera, a target detection network, a control module, and an execution module; There is no gimbal connection between the camera and the drone body; The camera captures images of the air-to-air scene. The target detection network performs target detection based on images captured by a camera, and the target detection results are sent to the control module. The target detection network is a network structure based on the YOLO11 architecture. In the YOLO11 architecture, the convolutional layer Conv in the backbone network is replaced by the LECA convolutional layer. The LECA convolutional layer enhances feature details and calibrates channel importance simultaneously through the parallel processing and fusion of local feature enhancement branches and channel attention branches, combined with the skip connections between the input and output of the parallel branches. Based on the target detection results, the control module obtains the pixel relative deviation between the center point of the target anchor box and the center point of the image; Using the relative deviation of pixels as feedback, the control commands for the racing drone are calculated, and the actions of the control execution module are controlled so that the center point of the racing drone's camera image converges to the center point of the target anchor frame, thus achieving closed-loop tracking control.

Images