Urban scene-oriented cloud side-end cooperative unmanned aerial vehicle vehicle relay tracking system and method

Abstract

The invention discloses an unmanned aerial vehicle relay tracking system and method for cloud side-end cooperation in an urban scene, and belongs to the technical field of intelligent traffic and unmanned aerial vehicle cooperative control. In order to solve the problems of insufficient endurance and limited sight distance of existing single-machine tracking in an urban large-range scene, a physically distributed and logically unified cloud-side-end three-level architecture is constructed, wherein a cloud end is responsible for natural language instruction analysis and global road network monitoring; the edge end serves as a regional scheduling hub and is responsible for operating a high-precision detection model, extracting a target Re-ID feature fingerprint and calculating a cross-machine relay time window; and the terminal unmanned aerial vehicle only operates a lightweight tracking model and a visual servo control law. According to the method, the urban area is divided into a plurality of honeycomb grids, seamless locking and relay of a target among unmanned aerial vehicles in different grid responsibility areas are realized by using a cross-view feature relay protocol and a collaborative finite-state machine, and the engineering problem of long-time-sequence and large-range vehicle tracking is effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of unmanned aerial vehicle (UAV) application technology, computer vision, edge computing, and intelligent transportation, and in particular to a vehicle tracking system and method that is applicable to complex urban road scenarios and realizes multi-UAV grid relay based on a cloud-edge-device collaborative architecture. Background Technology

[0002] With the rapid development of smart cities and intelligent transportation systems, the demand for real-time monitoring and continuous tracking of dynamic traffic targets is increasing. Drones, with their high mobility and wide field of view, have become an important supplement to ground-based monitoring. However, in complex, large-scale urban tracking tasks, existing single-drone systems face severe technical bottlenecks: first, limited endurance makes it difficult to support long-term tracking across regions; second, insufficient onboard computing power makes it difficult for embedded platforms to handle the real-time inference of high-precision deep learning models; and third, relay coordination is difficult, as urban buildings can easily cause line-of-sight interruptions, and traditional multi-drone relay relies solely on GPS coordinate transmission, which is prone to target identification errors due to positioning drift or changes in perspective. Existing technologies mostly focus on improving single tracking algorithms, lacking a mature system architecture that can effectively integrate cloud-edge-device resources and achieve automated relay scheduling through computing power offloading. Therefore, a gridded drone-vehicle relay tracking system based on cloud-edge-device collaboration is urgently needed. Summary of the Invention

[0003] A cloud-edge-device collaborative architecture is constructed to offload computing power. A three-layer distributed architecture with physical distribution and logical unity is established. In this architecture, the cloud is responsible for global road network monitoring and command parsing; computationally intensive deep learning detection and road network topology analysis tasks are offloaded to resource-rich edge servers; and latency-sensitive visual servo control tasks are retained to be executed on the drone terminal, solving the problem of insufficient computing power on a single machine.

[0004] A grid-based deployment and relay scheduling strategy is adopted. The urban area is divided into several honeycomb-shaped responsibility grids, abandoning the single-machine global search mode. By utilizing the relay scheduling engine at the edge and combining the kinematic prediction results of the target vehicle, when the target is about to leave the current grid, a time window is calculated and standby drones in adjacent grids are woken up in advance to fill the gap, thus solving the problem of limited range.

[0005] A cross-field-of-view feature fingerprint relay mechanism is proposed. Addressing the challenges of "target loss" or "identity recognition errors" caused by changes in viewing angle and GPS drift during inter-drone handover, this invention does not rely on hard association using geographic coordinates. Instead, it extracts the target's high-dimensional visual feature vector and transmits it between drones via an edge network. The relay drone uses this feature fingerprint to compare its similarity with targets within its field of view, ensuring that the locked target is the same physical target. Detailed Implementation

[0006] System hardware architecture and environment deployment See Figure 1 This embodiment provides a cloud-edge-device collaborative drone vehicle relay tracking system for urban scenarios. The system has a three-level distributed structure in terms of physical architecture: cloud control layer (100), edge computing layer (200), and drone terminal layer (300). Data interaction between the layers is achieved through a heterogeneous network.

[0007] The cloud-based management layer (100) is deployed on a public cloud server, running the Ubuntu Server operating system. As the top-level control hub of the system, this layer is configured with a lightweight large language model interface service for parsing users' natural language commands (such as "track the red car"); it integrates third-party map service interfaces to provide geographic information support; and it also deploys a web-based visual monitoring platform and a business database to monitor task status in real time and store historical tracking data.

[0008] The edge computing layer (200) is deployed on fixed ground facilities at various grid nodes in the city. The hardware uses high-performance edge servers equipped with server-level parallel computing units. The software environment includes a collaborative state arbitrator responsible for multi-machine task distribution; a road network topology scheduling module for calculating relay time windows; an intelligent visual perception service that deploys the TensorRT high-performance inference engine to accelerate the YOLO11 object detection model and Re-ID vehicle re-identification model running on the server; and a feature fingerprint relay pool for caching and transmitting the feature vectors of targets.

[0009] The UAV terminal layer (300) employs multiple quadcopter UAVs as mobile data acquisition terminals. Each UAV is equipped with a low-power embedded AI computing module (such as the NVIDIA Jetson series) and a Pixhawk 4 flight controller. Onboard sensors include a high-definition gimbal camera, an RTK-GPS positioning module, and a laser altitude-keeping radar. The onboard unit focuses on lightweight tasks, running LiteTrack tracking nodes and MAVLink communication nodes, responsible for image acquisition and basic visual servo control.

[0010] Detailed communication protocols and data definitions The system employs heterogeneous network communication between its various layers. The cloud-edge link utilizes a 5G network, transmitting control commands based on the MQTT protocol. The command format is defined as a JSON object. The edge-end link utilizes a high-bandwidth wireless communication link (such as a 5G private network, Wi-Fi 6, or a dedicated image transmission link). Video streams are transmitted using the RTSP protocol, and critical control signaling uses the MQTT protocol to ensure reliable command transmission and status synchronization. The onboard communication system connects the mission computer within the UAV to the flight controller via a UART serial port, running the MAVLink protocol and configured with a high-speed communication rate to meet the requirements of high-frequency attitude control.

[0011] Detailed Explanation of Core Workflow First, semantic initiation and grid wake-up. The user inputs natural language commands on the cloud-based management interface. In this embodiment, for ease of explanation, tracking a specific vehicle is used as an example. Assume the user inputs the command: "Tracking the red sedan traveling east on Renmin Road." The cloud integrates a third-party map service interface (in this embodiment, the Gaode Map Web Service API is used) and calls a geocoding service to convert the text "Renmin Road" into a corresponding set of vector geographic coordinates. Subsequently, a spatial indexing algorithm is used to match the coordinate set with a pre-set urban grid database to retrieve the responsibility grid ID (e.g., Grid-01) covered by the road segment. The cloud sends a wake-up command packet containing structured target features to the edge computing node corresponding to Grid-01 based on the ID, completing the conversion from natural language to machine commands.

[0012] Second, local visual servo tracking. The first UAV-A drone took off and searched for targets.

[0013] For detection and targeting, the edge-end uses a deep learning target detection model (preferably a YOLO series algorithm, such as YOLO11, in this embodiment) to periodically detect the video stream transmitted back from the UAV-A (the frequency can be set to 10-15 Hz). After identifying targets that meet semantic features (such as a red car), its position in the image coordinate system is calculated, and the bounding box coordinates are transmitted back to the UAV-A through the communication link.

[0014] Visual servo control: The UAV-A activates the onboard lightweight visual tracking module (in this embodiment, the LiteTrack algorithm is used, but other high real-time tracking algorithms such as SiamRPN and KCF can also be used). The onboard computer calculates the target center. With image center Pixel deviation. Define horizontal deviation. Vertical deviation The yaw rate is calculated using a cascaded PID control law. and pitch angular velocity : in , , These are the proportional, integral, and differential gain coefficients, respectively. This is the integral variable. The instruction is sent to the flight controller via the SET_POSITION_TARGET_LOCAL_NED message, ensuring the drone's nose remains aligned with the target vehicle for close following.

[0015] Third, cross-view relay and feature relay When the UAV-A approaches the grid boundary following the target, a cross-view relay process is triggered. Referring to Figure 2, this process illustrates the temporal interactions from position prediction (S201-S202), waking up the standby machine (S203-S205), to feature fingerprint injection (S206-S208). The specific algorithm and logic steps are as follows: 1. Topology Prediction and Extrapolation: The edge-end determines that a vehicle is about to enter an adjacent Grid-06 grid based on a high-precision map. If the target is briefly obscured by trees in the junction area, the edge-end uses a Kalman filter algorithm to perform a short-term extrapolation of the target position. A state vector is defined. ,in The target's position coordinates, For the goal Velocity component along the axial direction. Using the prediction equation. ,in The state transition matrix ( (System sampling time interval) The edge unit uses this equation to calculate the theoretical coordinates of the target within the blind zone, guiding the gimbal of the next UAV-B drone to pre-aim at the area.

[0016] 3. Feature fingerprint extraction: The highest quality frame of the target image in the UAV-A field of view is cropped from the edge and input into the Re-ID network to extract a high-dimensional feature vector. .

[0017] 4. Feature Injection and Matching: UAV-B takes off and arrives at the handover point. The edge will then inject vectors. The matching module sends the data to the UAV-B. The UAV-B detects all vehicles within its field of view and extracts their feature vectors. And calculate the cosine similarity: in, The dimension of the feature vector. Representing vectors The first in Each component represents the similarity of a vehicle within the field of view. When the system determines that they are the same target, the UAV-B automatically locks onto the vehicle.

[0018] Fourth, collaborative handshake and exception handling As shown in phases three and four of Figure 2 (S209-S215), in order to prevent both drones from going out of control or the target from being lost at the same time, the system executes a strict "request-verify-release" handshake protocol.

[0019] 1. After successfully locking onto the target, the UAV-B sends an Ack-Locked signal to the edge device.

[0020] 2. After receiving the confirmation signal, the edge device sends Release (mission release) and RTL (return to home) commands to UAV-A.

[0021] 3. If UAV-B fails to lock onto the target within 10 seconds of reaching the designated position (i.e., all...) The system enters "cooperative search mode." At this time, UAV-A does not return to base but instead ascends to 100 meters for a wide-area overview, keeping the target within its field of view; simultaneously, UAV-B performs a spiral flight search. If the handover is not completed within 30 seconds, the edge terminal sends an alarm to the cloud, requesting manual intervention or the use of roadside surveillance cameras for location assistance.

[0022] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0023] Fifth, the collaborative state machine logic of unmanned aerial vehicles (UAVs) Referring to Figure 3, the system utilizes a cooperative finite state machine to manage the behavior transitions of the UAV in each of the aforementioned stages. The specific state definitions are as follows: Standby status: stationed in the hangar awaiting activation.

[0024] Semantic response takeoff: Takeoff in response to cloud commands.

[0025] Area tracking: Perform visual servo control (corresponding to "Part Two" above).

[0026] Handover and coordination: This state is entered when a boundary warning is triggered (corresponding to the "third" part above), and a cross-machine link is established.

[0027] Task Release: Release permissions after confirming successful relay (corresponding to "Part Four" above).

[0028] Return to home for charging: Perform RTL automatic return to home. Attached Figure Description

[0029] Figure 1 This is an overall architecture diagram of the gridded drone vehicle relay tracking system based on cloud-edge-device collaboration provided in this embodiment of the invention, which shows the connection relationship and data flow between the cloud, the edge, and the drone.

[0030] Figure 2 This is a timing diagram of cross-grid relay interaction in an embodiment of the present invention, which shows the complete communication process from prediction, wake-up, feature injection to lock handshake.

[0031] Figure 3 This is a state transition logic diagram of a collaborative finite state machine in an embodiment of the present invention, illustrating the state switching conditions of the UAV at different mission stages.

Claims

1. A cloud-edge-device collaborative unmanned aerial vehicle (UAV) relay tracking system and method for urban scenarios, characterized in that, The system adopts a three-layer distributed architecture of cloud-edge-device. It includes: a cloud-based management layer, configured with a semantic parsing module and a global geographic information service interface, used to parse user-input natural language commands into target feature descriptors and predict the responsibility grid area that a target vehicle is about to enter based on real-time road network data; an edge computing layer, deployed at distributed ground helipad nodes or edge servers, communicating with the cloud and multiple drone terminals; this layer is configured with a deep learning target detection module and a re-identification feature extraction module for performing high-computing-power perception and recognition tasks, and a relay scheduling engine for calculating the handover time window between adjacent drones; and a drone terminal layer containing multiple drones distributed in different cellular grids, each drone equipped with an embedded computing unit and flight control system for running a lightweight visual tracking model and visual servo control algorithm to perform real-time following tasks within the grid.

2. The system according to claim 1, characterized in that, The system employs a cross-field-of-view feature fingerprint relay mechanism to achieve seamless multi-drone relay. The specific steps are as follows: When the currently on-duty UAV-A approaches the grid boundary, the edge computing layer controls the gimbal of UAV-A to acquire high-definition images of the target vehicle; the edge computing layer uses a Re-ID network to extract the high-dimensional appearance feature vector, i.e., the feature fingerprint, of the target from the image and injects it into the detection network of the next standby UAV-B through a communication link; after UAV-B takes off and arrives at the handover area, it uses the feature fingerprint to perform cosine similarity comparison on all vehicles in the field of view, automatically locking the target with the highest feature fingerprint matching degree, and realizing identity confirmation with zero prior information.

3. The system according to claim 1, characterized in that, The edge computing layer is configured with a collaborative finite state machine to manage multi-machine collaborative logic. The state machine includes standby, semantic response takeoff, area tracking, handover collaboration, task release, and return-to-home states. These states are mutually exclusive and switched between each other through preset conditions. The system sets a strict "request-verify-release" handshake protocol: only when the edge terminal receives the "visual lock confirmation" signal sent by the relay UAV-B will it send the "task release" command to the current UAV-A to prevent the tracking chain from breaking.

4. The system according to claim 1, characterized in that, The relay scheduling engine has a trajectory prediction function based on road network topology: the edge computing layer combines the road network topology structure of the high-precision map with the historical motion vector of the target vehicle to eliminate unreachable path branches; and uses Kalman filtering to perform short-term linear extrapolation of the target's position in the blind spot of the field of view, guiding the relay UAV to establish a ground observation field of view in advance.

5. A method for tracking unmanned aerial vehicles based on the system described in any one of claims 1-4, characterized in that, The following detailed steps are included: (1) Semantic activation and grid wake-up. The cloud control layer parses the user's natural language commands, combines the GIS road network data to locate the initial coordinates of the target, and sends the wake-up command containing the target feature descriptor to the edge computing layer of the corresponding responsible grid; (2) Local visual servo tracking. The current duty UAV uses a visual sensor to acquire images, calculates the deviation error between the target center and the image center, and uses a PID control law to calculate the yaw rate and pitch rate to control the UAV attitude to keep the target in the center of the field of view; (3) Cross-view relay decision. The edge computing layer monitors the target position in real time. When the target position coordinates predicted by the Kalman filter algorithm cross the current grid boundary, it determines that the relay process is triggered, wakes up the standby UAV in the adjacent grid, and plans the intercept waypoint according to the predicted coordinates, and directs the standby UAV to fly to the handover area; (4) Feature fingerprint relay and locking. The edge computing layer extracts the high-dimensional appearance feature vector of the target in the field of view of the currently on-duty UAV and transmits it to the standby UAV that has arrived at the handover area; the standby UAV calculates the cosine similarity between the candidate target in the field of view and the feature vector, and locks the target when the maximum similarity exceeds the preset threshold; (5) Collaborative handshake and task release. The standby UAV sends a lock confirmation signal to the edge computing layer, and the edge computing layer then sends a task release command to the currently active UAV, completing the smooth transfer of tracking rights.

Images