Traffic law enforcement and tracking method and system based on linkage of unmanned aerial vehicle and high point video

Abstract

The present application relates to a traffic law enforcement and tracking method and system based on unmanned aerial vehicle and high point video linkage, belonging to the technical field of intelligent traffic management, comprising: acquiring high point video stream, identifying vehicles and abnormal events through a target detection model, outputting obstacle information and target initial features; calling static road network data, converting the obstacle information into geographic coordinates and mapping to the static road network data, fusing and deduplicating the overlapping areas, labeling the obstacle occupation areas and survival time length, and generating a global obstacle distribution map; reading the global obstacle distribution map, generating a blind area-free obstacle avoidance tracking route; controlling the unmanned aerial vehicle to approach the target area based on the blind area-free obstacle avoidance tracking route, acquiring a detailed video stream and extracting deep features, time and space matching the target initial features to identify the target vehicle, acquiring the target vehicle driving features, and generating a law enforcement record. The present application realizes high-low altitude cooperative tracking, improves matching stability, and automatically forms a complete law enforcement evidence chain.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic management technology, specifically relating to a traffic enforcement and tracking method and system based on the linkage of unmanned aerial vehicles and high-point video. Background Technology

[0002] At present, traffic supervision of the mixed road network of urban roads, highways and forest fire prevention channels mainly adopts two operation methods: fixed-point high-point monitoring and independent capture, and drone patrol and enforcement. The data of the two sets of equipment are independent of each other, and there are many technical defects that are difficult to solve in collaborative management.

[0003] Fixed high-point surveillance offers wide coverage and can identify vehicles, roadblocks, and smoke points in forests over a broad area. However, the fixed camera positions result in numerous blind spots, and vehicles are easily lost after leaving the monitoring field of view. Long-distance images have limited clarity, making it difficult to extract details such as license plates and vehicle textures. Relying solely on global features can lead to confusion between multiple vehicles, and cross-point continuous matching accuracy is insufficient. Conventional target detection models are trained on general image datasets and are not optimized for various scenarios such as forests, urban areas, and highways, including occlusion, backlighting, and fog. This results in a high probability of missed or false detections of illegally parked vehicles, road obstructions, and fire smoke points. Furthermore, existing recognition outputs only provide simple bounding box information, failing to convert dynamic vehicles and pedestrians, or static roadblocks, into road network geographic coordinates. They also cannot generate a global obstacle distribution map with time attributes, thus failing to provide effective data support for intelligent flight path planning for drones.

[0004] Especially in the complex road network scenario of "high-speed interchange connecting mountainous and forest fire prevention channels", there are not only high-speed vehicles and low-speed agricultural vehicles, but also road obstructions caused by debris accumulation, floating smoke from forest fires, and illegally parked vehicles in emergency areas. At the same time, it is affected by complex environmental interferences such as morning fog, strong midday sunlight, evening backlight, and obstruction by branches and leaves in the forest. Traditional single-scenario detection models and fixed route schemes are completely unable to meet the collaborative law enforcement needs of such mixed road networks.

[0005] Drones offer advantages such as maneuverability and close-range access, as well as high-definition evidence collection; however, their inefficient, unguided, all-area cruising is extremely low. Traditional drone routes are manually preset and fixed, unable to dynamically adjust their paths based on real-time road obstacles and no-fly zones. During flight, they are prone to collisions with moving vehicles and static obstacles, posing flight safety hazards. High-point monitoring and drones operate on two separate systems, with video streams and feature data not shared. Initial target features output from high-point monitoring cannot be compared spatiotemporally with the deep detail features collected by the drone, making it impossible to assign a unified identifier to the same vehicle. Even after a suspect vehicle is detected by high-point monitoring, the drone struggles to accurately track it, and if the target is lost, the complete trajectory cannot be reconstructed.

[0006] Traditional vehicle tracking methods rely solely on single-device coordinate calculations, failing to construct a complete target motion model and lacking collaborative processing logic for Kalman filter coordinate prediction and Hungarian algorithm data association. This makes the tracking link highly susceptible to interruption in scenarios involving traffic obstruction or lane changes. Furthermore, the various stages of the overall enforcement process are fragmented: target identification, route scheduling, feature matching, trajectory fitting, violation determination, and evidence archiving operate independently. This prevents the automatic integration of temporal trajectory coordinates, high and low altitude video footage, and violation determination conclusions to generate standardized enforcement records. The resulting evidence is scattered and fails to meet the requirements for a complete evidence chain archiving in non-on-site traffic management enforcement.

[0007] Existing air-ground collaborative solutions lack integrated supporting processes such as lightweight model training, obstacle road network fusion mapping, multi-constraint dynamic route solving, high and low altitude feature matching, continuous trajectory generation, and automatic evidence collection and archiving. The degree of automation is low, and many steps rely on manual operation, which cannot meet the needs of unmanned collaborative tracking and law enforcement in multiple scenarios and all weather conditions. Summary of the Invention

[0008] To address the aforementioned problems in the existing technology, this invention provides a traffic enforcement and tracking method and system based on the linkage of unmanned aerial vehicles (UAVs) and high-point video.

[0009] The objective of this invention can be achieved through the following technical solution: a traffic enforcement and tracking method based on the linkage of unmanned aerial vehicles (UAVs) and high-altitude video, comprising: Acquire high-point video streams of traffic scenes, input them into a pre-built target detection model to identify vehicles and abnormal events, and output obstacle information and initial target features; Retrieve pre-stored static road network data, convert the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and map it onto the static road network data, merge and remove duplicates in overlapping areas, mark the obstacle-occupied area and duration, and generate a global obstacle distribution map; Read the global obstacle distribution map, extract the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generate a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguish between dynamic and static obstacles based on the obstacle duration and use different detour methods to generate a blind-spot-free obstacle avoidance and tracking route. Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, acquire detailed video streams of the target vehicle and extract depth features; the depth features are spatiotemporally matched with the target initial features to identify the target vehicle, acquire the target vehicle's driving characteristics, and generate an enforcement record based on the target vehicle's driving characteristics.

[0010] Specifically, the pre-built target detection model is obtained based on the following method: A dedicated image dataset was acquired on-site. The dedicated image dataset includes image materials covering various scenarios such as highways, urban roads, and forest areas, including morning fog, midday strong light, evening backlight, and nighttime supplemental lighting conditions. The labeled objects include vehicles, pedestrians, roadblocks, smoke points, and illegally parked targets. A lightweight single-stage object detection network is used to build a basic recognition framework. The dedicated image dataset is split into a training set and a validation set, and the network is iteratively trained and its performance is verified. After the training reaches the target, the lightweight single-stage object detection network is pruned, quantized and compressed. The network weight parameters are continuously adjusted based on the verification results. After the parameters of the lightweight single-stage object detection network converge and stabilize, the network is solidified to generate the object detection model.

[0011] Specifically, the output process of the obstacle information and the initial features of the target is as follows: The system receives the high-point video stream of the traffic scene, performs image denoising and size normalization preprocessing, and then inputs it into the target detection model. The system extracts shallow texture and deep semantic features of the image frame by frame through the lightweight single-stage target detection network, performs pixel-level localization of static road obstacles, dynamic people and vehicles, and forest smoke points, and outputs the obstacle information. For the identified vehicle target, global contour features, main body color features, and texture details of the front and body are extracted and integrated to generate the initial features of the target.

[0012] Specifically, the generation process of the global obstacle distribution map is as follows: Retrieve the pre-stored static road network data, which includes road coordinates, lane distribution, regulatory area boundaries, and geographic coordinates of no-fly zones for drones to avoid. Receive the obstacle information with a timestamp, the obstacle information carrying the pixel coordinates and bounding box size of static roadblocks, dynamic people and vehicles, and forest smoke points respectively; The pre-configured mapping and transformation relationship between the pixel coordinate system and the on-site geographic coordinate system is invoked. Based on the installation height, pitch angle and lens distortion correction coefficient of the high-point camera, scale compensation and projection conversion are completed. The pixel coordinates and bounding box size in the obstacle information are converted into geographic coordinates and geographic coverage that are consistent with the static road network data. The converted geographic boundary is superimposed and mapped to the corresponding position in the static road network data. The spatial neighborhood comparison algorithm is invoked to traverse the overlapping obstacle coverage areas within the same road network unit and calculate the area overlap ratio. If the area overlap ratio exceeds the preset fusion threshold, it is determined to be a spatially overlapping obstacle area. For obstacles of the same type, the duplicate marked areas are directly deleted, and obstacles of different types are merged and fused to form overlapping boundaries. Based on the timestamp and combined with the preset obstacle lifecycle determination rules, each fused obstacle area is assigned a temporal value to distinguish it, the obstacle-occupied area and the duration of the obstacle are marked, and the global obstacle distribution map with spatiotemporal attributes is generated.

[0013] Specifically, the planning process for the blind-spot-free obstacle avoidance and tracking route is as follows: Read the global obstacle distribution map with spatiotemporal attributes, and extract all obstacle-occupied areas, obstacle duration, and no-fly zone boundaries within the distribution map; Using the geographic coordinates of the target vehicle as the end point of the flight path and the coordinates of the UAV airport as the starting point of the flight path, the basic flight path is generated by incorporating multiple constraints such as flight safety distance, maximum detour threshold and evidence collection field of view coverage. The basic flight path is segmented and verified, and the path segments that intersect with the obstacle-occupied area and the no-fly zone are eliminated; The dynamic or static nature of an obstacle is determined based on its duration: if the duration is less than a preset duration threshold, it is determined to be a dynamic obstacle, and the drone is controlled to detour around it at a distance less than a first detour threshold; if the duration is greater than or equal to the preset duration threshold, it is determined to be a static obstacle or a permanent no-fly zone, and the drone is controlled to detour around it at a distance greater than a second detour threshold, where the second detour threshold is greater than the first detour threshold. The drone's flight speed and image acquisition pitch angle are synchronized to generate the blind-spot-free obstacle avoidance and tracking route.

[0014] Specifically, the extraction process of the deep features is as follows: Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, and the detailed video stream of the target vehicle is collected in real time and image noise reduction and perspective distortion correction are performed. Based on the corrected detailed video stream, local vehicle detail features, global body color features, and character texture features are extracted. The local detail features of the vehicle, the global color features of the vehicle body, and the character texture features are normalized and fused to obtain the depth features with unified dimensions.

[0015] Specifically, the spatiotemporal matching process between the depth features and the initial target features is as follows: Extract the acquisition timestamps and geographic coordinates corresponding to the high-point video stream and the detail video stream respectively, and construct spatiotemporal constraints; Calculate the cosine similarity between the depth features and the initial features of the target, and perform an initial feature comparison with a preset similarity threshold; When the similarity is lower than the preset similarity threshold, the corresponding feature pair is directly discarded; when the similarity reaches the preset similarity threshold, feature pairs whose video acquisition time difference and spatial coordinate deviation fall within the preset allowable range are selected; feature combinations that simultaneously meet the similarity threshold and the spatiotemporal constraints are retained, bound to a unified vehicle identifier, and the target vehicle is identified.

[0016] Specifically, the process of acquiring the driving characteristics of the target vehicle includes: Based on the initial features of the target, the depth features, and the unified vehicle identifier bound by the spatiotemporal matching, a target motion model is constructed by combining the spatiotemporal obstacle data corresponding to the global obstacle distribution map with the blind-spot-free obstacle avoidance and tracking route. The system retrieves the target position and driving speed data corresponding to the historical detection frames of the high-point video stream, predicts the target's current frame coordinates, associates the predicted coordinates with the target's real-time position collected by the UAV along the blind-spot-free obstacle avoidance tracking route, and corrects the target coordinate prediction values; iteratively updates the target's real-time position parameters, continuously fits and associates the temporal positioning parameters, and generates the target vehicle's driving characteristics.

[0017] Specifically, the process for generating the law enforcement record is as follows: Based on the target vehicle's driving characteristics, the corresponding high-point video stream and detailed video stream footage are matched, the geographic location coordinates and collection timestamps corresponding to the trajectory are associated, the behavior determination results, video footage and trajectory data are integrated, and the law enforcement record is generated and saved.

[0018] Specifically, the traffic enforcement and tracking system based on the linkage between drones and high-altitude video includes: High-point perception module: acquires high-point video streams of traffic scenes, inputs them into a pre-built target detection model to identify vehicles and abnormal events, and outputs obstacle information and initial target features; Map building module: retrieves pre-stored static road network data, converts the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and maps it to the static road network data, merges and removes duplicates in overlapping areas, marks the obstacle-occupied area and duration, and generates a global obstacle distribution map; Flight path generation module: Reads the global obstacle distribution map, extracts the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generates a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguishes between dynamic and static obstacles based on the obstacle duration and adopts different detour methods to generate a blind-spot-free obstacle avoidance and tracking flight path. Record generation module: performs spatiotemporal matching between the depth features and the target initial features to identify the target vehicle, obtains the target vehicle's driving characteristics, and generates a law enforcement record based on the target vehicle's driving characteristics.

[0019] The beneficial effects of this invention are as follows: By employing high-point and drone-based air-to-ground collaborative data collection, and binding vehicles with unified identifiers through spatiotemporal matching, blind spots in the monitoring field of view are eliminated, and target loss and vehicle feature confusion are avoided during the tracking process, resulting in higher target matching accuracy.

[0020] By integrating obstacle information with the road network to generate a global obstacle distribution map, and dynamically planning blind-spot-free obstacle avoidance routes, flight risks can be mitigated and the efficiency of drone patrol and evidence collection can be improved.

[0021] By relying on motion models combined with filtering prediction and data association algorithms, continuous and complete vehicle trajectories can be generated even in occlusion and lane change scenarios, resulting in strong tracking stability.

[0022] The entire process automatically integrates trajectory data, video footage, and violation determination results to generate standardized law enforcement records with a complete chain of evidence, reducing manual processing workload and achieving fully automated collaborative law enforcement. Attached Figure Description

[0023] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0024] Figure 1 This is a flowchart of the method in this invention; Figure 2 This is a block diagram of the spatiotemporal feature matching logic in this invention. Detailed Implementation

[0025] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0026] Please see Figure 1-2 Traffic enforcement and tracking methods based on the linkage of drones and high-point video include: Acquire high-point video streams of traffic scenes, input them into a pre-built target detection model to identify vehicles and abnormal events, and output obstacle information and initial target features; Retrieve pre-stored static road network data, convert the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and map it onto the static road network data, merge and remove duplicates in overlapping areas, mark the obstacle-occupied area and duration, and generate a global obstacle distribution map; Read the global obstacle distribution map, extract the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generate a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguish between dynamic and static obstacles based on the obstacle duration and use different detour methods to generate a blind-spot-free obstacle avoidance and tracking route. Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, acquire detailed video streams of the target vehicle and extract depth features; the depth features are spatiotemporally matched with the target initial features to identify the target vehicle, acquire the target vehicle's driving characteristics, and generate an enforcement record based on the target vehicle's driving characteristics.

[0027] Specifically, the pre-built target detection model is obtained based on the following method: A dedicated image dataset was acquired on-site. The dedicated image dataset includes image materials covering various scenarios such as highways, urban roads, and forest areas, including morning fog, midday strong light, evening backlight, and nighttime supplemental lighting conditions. The labeled objects include vehicles, pedestrians, roadblocks, smoke points, and illegally parked targets. A lightweight single-stage object detection network is used to build a basic recognition framework. The dedicated image dataset is split into a training set and a validation set, and the network is iteratively trained and its performance is verified. After the training reaches the target, the lightweight single-stage object detection network is pruned, quantized and compressed. The network weight parameters are continuously adjusted based on the verification results. After the parameters of the lightweight single-stage object detection network converge and stabilize, the network is solidified to generate the object detection model.

[0028] Specifically, the output process of the obstacle information and the initial features of the target is as follows: The system receives the high-point video stream of the traffic scene, performs image denoising and size normalization preprocessing, and then inputs it into the target detection model. The system extracts shallow texture and deep semantic features of the image frame by frame through the lightweight single-stage target detection network, performs pixel-level localization of static road obstacles, dynamic people and vehicles, and forest smoke points, and outputs the obstacle information. For the identified vehicle target, global contour features, main body color features, and texture details of the front and body are extracted and integrated to generate the initial features of the target.

[0029] Example 1 This embodiment is applied to a complex road network scenario where suburban expressway interchanges connect to mountain firebreaks. This road segment simultaneously includes expressway sections, rural auxiliary roads, and forest fire patrol sections, resulting in complex operating conditions. These conditions include high-speed vehicles, low-speed agricultural vehicles, road obstructions caused by debris accumulation, smoke plumes from forest fires, and illegally parked vehicles in emergency areas. Furthermore, this scenario is affected by complex environmental factors such as morning fog, direct midday sunlight, evening backlighting, and obstruction from forest foliage. The present invention's target detection, obstacle recognition, and initial target feature extraction process based on high-point video is adapted to this scenario, as detailed below: High-point monitoring images were collected for this complex road network during four time periods: foggy weather, strong light, backlight, and nighttime supplemental lighting. The images fully covered various conditions, including vehicle obstruction, road rockfalls, forest smoke, and illegally parked vehicles. After collection, the images were annotated at the pixel level. The annotated objects included cars, trucks, agricultural vehicles, pedestrians, rock obstacles, tree trunks blocking the road, forest smoke, and illegally parked vehicles. The bounding box range and occlusion degree labels of each target were recorded simultaneously. All annotated images were integrated to generate a scene-specific image dataset A.

[0030] A lightweight single-stage object detection network is built as the basic recognition framework. The dataset A is split into a training sample set A1 and a validation sample set A2 according to a fixed ratio. A1 is used to carry out multiple rounds of iterative training. After each fixed round of iteration, A2 is called to perform full recognition accuracy verification. An adaptive learning rate control strategy is adopted during the training process. If vehicle detection is missed or smoke point is misidentified during verification, all weight parameters of the network are corrected layer by layer in reverse. After multiple rounds of verification and recognition accuracy stabilized at the preset threshold B, the model pruning, quantization, and compression operation was initiated to eliminate redundant channels with weights approaching zero and to quantize high-order floating-point parameters into low-order integer parameters to reduce computing power consumption. After compression, a long-term stability test was conducted for a duration T, monitoring various target recognition errors throughout the process. Once the parameters converged without fluctuations, the network weights and inference logic were solidified, a dedicated target detection model M was generated, and deployed to a high-point edge computing device.

[0031] High-definition cameras at the highest points of the road section output high-point video streams around the clock. After receiving a single frame image, the edge device performs two-level standardized preprocessing: Gaussian filtering is used to eliminate image noise caused by fog and lens dust, and then the images with different resolutions are uniformly scaled to the standard size S to complete the normalization mapping of pixel values. The preprocessed image is then sent to the model M for inference calculation.

[0032] Model M extracts image features in layers. Shallow convolution extracts road texture and shallow texture features of obstacle edges, while deep convolution extracts deep semantic features of vehicle outlines and smoke color. It completes pixel-level localization of static road obstacles, dynamic people and vehicles, and forest smoke points. It outputs pixel coordinates P, bounding box size B, and category label for each target. It packages all target data in a single frame and binds them to the acquisition time sequence marker T0, integrates and generates real-time obstacle information C, and sends it to the map building module.

[0033] For targets classified as motor vehicles, a multi-branch feature extraction logic is employed: a global convolutional branch extracts the complete vehicle contour vector, a mean pooling branch statistically analyzes large-area pixels of the vehicle body to generate a dominant color vector, and a local detail branch separately extracts headlight and paint texture detail vectors from the front and middle areas of the vehicle body. The contour, dominant color, and detail vectors are then dimensionally aligned, stitched together, and fused to generate a unique multi-dimensional visual vector representing a single vehicle. This vector serves as the initial feature D output for the target, along with the target's geographic coordinates and temporal marker T0, providing raw data for subsequent road network mapping and feature matching.

[0034] Specifically, the generation process of the global obstacle distribution map is as follows: Retrieve the pre-stored static road network data, which includes road coordinates, lane distribution, regulatory area boundaries, and geographic coordinates of no-fly zones for drones to avoid. Receive the obstacle information with a timestamp, the obstacle information carrying the pixel coordinates and bounding box size of static roadblocks, dynamic people and vehicles, and forest smoke points respectively; The pre-configured mapping and transformation relationship between the pixel coordinate system and the on-site geographic coordinate system is invoked. Based on the installation height, pitch angle and lens distortion correction coefficient of the high-point camera, scale compensation and projection conversion are completed. The pixel coordinates and bounding box size in the obstacle information are converted into geographic coordinates and geographic coverage that are consistent with the static road network data. The converted geographic boundary is superimposed and mapped to the corresponding position in the static road network data. The spatial neighborhood comparison algorithm is invoked to traverse the overlapping obstacle coverage areas within the same road network unit and calculate the area overlap ratio. If the area overlap ratio exceeds the preset fusion threshold, it is determined to be a spatially overlapping obstacle area. For obstacles of the same type, the duplicate marked areas are directly deleted, and obstacles of different types are merged and merged to form overlapping boundaries. Based on the timestamp and combined with the preset obstacle lifecycle determination rules, each fused obstacle area is assigned a temporal value to distinguish it, the obstacle-occupied area and the duration of the obstacle are marked, and the global obstacle distribution map with spatiotemporal attributes is generated.

[0035] Example 2 This embodiment continues the aforementioned complex road network scenario of suburban expressway interchanges connecting mountain firebreaks, completing the entire process of generating a global obstacle distribution map. Based on the timestamped obstacle information C output in Embodiment 1, and combined with pre-stored static road network data, coordinate mapping, spatial fusion, and temporal annotation are completed to generate a global obstacle distribution map with spatiotemporal attributes. The specific process is as follows: Retrieve pre-stored static road network data S. S fully carries four types of geographic reference information: road segment topology coordinates, lane level distribution range, traffic monitoring area boundary coordinates, and drone avoidance no-fly zone geographic coordinates, serving as a unified coordinate system for all obstacle spatial mapping.

[0036] The system receives obstacle information C from high points in real time. C encapsulates the pixel coordinates P and bounding box size B of three types of targets: static road obstacles, dynamic people and vehicles, and forest smoke points. Each data entry is bound to a unified time sequence marker T0.

[0037] The pre-configured mapping relationship H between the pixel coordinate system and the on-site geographic coordinate system is invoked to complete the coordinate conversion process: based on the scene camera installation height, pitch angle, and lens distortion correction coefficient, scale compensation and projection conversion are completed, and the pixel coordinates P and bounding box size B of each obstacle target are converted frame by frame and target by target into standard geographic coordinates G and geographic coverage area that match the static road network S; after the conversion is completed, the geographic boundaries corresponding to all obstacles are accurately superimposed and mapped to the corresponding positions of the road network topology, and the alignment and matching of image recognition obstacle information with the real physical road network space is completed.

[0038] For spatially overlapping obstacle coverage areas in the mapped road network, the system initiates a complete spatial fusion deduplication mechanism: it calls a spatial neighborhood comparison algorithm to traverse the static road barriers, dynamic pedestrian and vehicle areas, and forest smoke point areas superimposed within the same road network unit, calculates the percentage of overlapping area, and if the percentage of overlapping area exceeds a preset fusion threshold, it is determined to be a spatially overlapping obstacle area; for obstacles of the same type, duplicate marked areas are directly deleted, and overlapping boundaries of different types of obstacles are merged and fused, retaining the true and effective obstacle coverage range throughout the process and eliminating all redundant and duplicate data.

[0039] For spatially overlapping obstacle coverage areas: After pixel coordinate to geographic coordinate mapping, multiple obstacle bounding boxes of different types and time sequences are projected onto the same road network geographic area, forming overlapping and repeatedly labeled blocks. This covers scenarios such as dynamic pedestrians and vehicles overlapping with fixed road obstacles, multiple vehicles overlapping areas, and forest smoke points coinciding with road obstacle projections. Merging refers to merging overlapping blocks into a continuous area; redundancy removal refers to deleting duplicate labeled boundaries generated by repeated superposition of the same obstacle, ensuring that each obstacle within the road network retains only a unique effective coverage area.

[0040] Based on the temporal marker T0 carried by the obstacle information C, and combined with the system's preset obstacle lifecycle determination rules, temporal values ​​are assigned to distinguish each fused obstacle area: long-term fixed roadblocks are marked with constant occupancy attributes, short-term passing vehicles and instantaneous forest smoke are marked with dynamic temporary attributes, and the obstacle occupancy space range R and obstacle duration T1 are marked for each obstacle area simultaneously.

[0041] By integrating three types of data—original road network topology information, deduplicated obstacle spatial coverage, and temporal attributes of each obstacle—a global obstacle distribution map with complete spatiotemporal dual-dimensional attributes is generated. The distribution map can provide real-time feedback on the obstacle distribution status at any time and on any road segment, providing high-precision standardized spatial data support for subsequent UAV multi-constraint obstacle avoidance route planning.

[0042] Specifically, the planning process for the blind-spot-free obstacle avoidance and tracking route is as follows: Read the global obstacle distribution map with spatiotemporal attributes, and extract all obstacle-occupied areas, obstacle duration, and no-fly zone boundaries within the distribution map; Using the geographic coordinates of the target vehicle as the end point of the flight path and the coordinates of the UAV airport as the starting point of the flight path, and incorporating multiple constraints such as flight safety distance, maximum detour threshold and evidence collection field of view coverage, a multi-constraint path solving algorithm is used to generate the basic flight path. The basic flight path is segmented and verified, and the path segments that intersect with the obstacle-occupied area and the no-fly zone are eliminated; The dynamic or static nature of an obstacle is determined based on its duration: if the duration is less than a preset duration threshold, it is determined to be a dynamic obstacle, and the drone is controlled to detour around it at a distance less than a first detour threshold; if the duration is greater than or equal to the preset duration threshold, it is determined to be a static obstacle or a permanent no-fly zone, and the drone is controlled to detour around it at a distance greater than a second detour threshold, where the second detour threshold is greater than the first detour threshold. The drone's flight speed and image acquisition pitch angle are synchronized to generate the blind-spot-free obstacle avoidance and tracking route.

[0043] Example 3 This embodiment continues the composite road network scenario of suburban expressway interchanges connecting mountain firebreaks described in Embodiments 1 and 2. Based on the initial target feature D and multi-dimensional vehicle visual feature data output in Embodiment 1, and the global obstacle distribution map with complete spatiotemporal attributes generated in Embodiment 2, it fully completes the entire technical process of blind-spot-free obstacle avoidance and tracking route planning, UAV close-range depth feature extraction, high and low altitude dual-source feature spatiotemporal matching, scene-adaptive target motion model construction, continuous target trajectory iterative fitting, intelligent violation judgment, and standardized law enforcement record generation. The specific process is as follows: Complete planning of blind-spot-free obstacle avoidance and tracking routes: The system reads the obstacle distribution map of the entire region, and completely extracts the space range R occupied by all obstacles in the map, the duration of the obstacles T1, and the boundary of the no-fly zone; it sets the real-time geographical coordinates G1 of the suspected vehicle identified by high point as the end point of the flight path, and the fixed coordinates G2 of the normalized UAV site as the starting point of the flight path, and incorporates multiple constraints such as flight safety distance, maximum detour threshold, and evidence collection field of view coverage, and calls the multi-constraint path solving algorithm to iteratively calculate and generate the initial basic flight path.

[0044] The cost function of the multi-constraint path solving algorithm is expressed as:

[0045] Where P is a candidate path point, D(P) is the shortest distance cost between the path point and the obstacle-occupied area and the no-fly zone, S(P) is the path smoothness cost, C(P) is the path length cost, and α, β, and γ are the weight coefficients of each cost. The algorithm iteratively solves for the path point sequence that minimizes f(P) as the basic flight path.

[0046] Geospatial verification is performed segment by segment on the basic flight path to accurately remove path segments that intersect with or overlap with obstacle-occupied areas and no-fly zones. The detour distance is dynamically adjusted according to the obstacle type: for short-term dynamic obstacles, a small-amplitude flexible detour strategy is adopted based on T1 (detour distance is less than the first detour threshold), while for long-term fixed roadblocks and permanent no-fly zones, a large-scale replanning detour is implemented (detour distance is greater than the second detour threshold, and the second detour threshold is greater than the first detour threshold). The rated safe flight speed range of the UAV and the pitch angle of the onboard camera image acquisition are simultaneously incorporated to uniformly optimize the flight path curvature, flight step length, and hovering evidence collection points, and finally generate a blind-spot-free obstacle avoidance and tracking flight path L that adapts to real-time road conditions and meets the needs of close-range high-definition evidence collection.

[0047] Specifically, the extraction process of the deep features is as follows: Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, and the detailed video stream of the target vehicle is collected in real time and image noise reduction and perspective distortion correction are performed. Based on the corrected detailed video stream, local vehicle detail features, global body color features, and character texture features are extracted. The local detail features of the vehicle, the global color features of the vehicle body, and the character texture features are normalized and fused to obtain the depth features with unified dimensions.

[0048] Specifically, the spatiotemporal matching process between the depth features and the initial target features is as follows: Extract the acquisition timestamps and geographic coordinates corresponding to the high-point video stream and the detail video stream respectively, and construct spatiotemporal constraints; Calculate the cosine similarity between the depth features and the initial features of the target, and perform an initial feature comparison with a preset similarity threshold; When the similarity is lower than the preset similarity threshold, the corresponding feature pair is directly discarded; when the similarity reaches the preset similarity threshold, feature pairs whose video acquisition time difference and spatial coordinate deviation fall within the preset allowable range are selected; feature combinations that simultaneously meet the similarity threshold and the spatiotemporal constraints are retained, bound to a unified vehicle identifier, and the target vehicle is identified.

[0049] Specifically, the process of acquiring the driving characteristics of the target vehicle includes: Based on the initial features of the target, the depth features, and the unified vehicle identifier bound by the spatiotemporal matching, a target motion model is constructed by combining the spatiotemporal obstacle data corresponding to the global obstacle distribution map with the blind-spot-free obstacle avoidance and tracking route. The system retrieves the target position and driving speed data corresponding to the historical detection frames of the high-point video stream, predicts the target's current frame coordinates, associates the predicted coordinates with the target's real-time position collected by the UAV along the blind-spot-free obstacle avoidance tracking route, and corrects the target coordinate prediction values; iteratively updates the target's real-time position parameters, continuously fits and associates the temporal positioning parameters, and generates the target vehicle's driving characteristics.

[0050] Specifically, the process for generating the law enforcement record is as follows: Based on the target vehicle's driving characteristics, the corresponding high-point video stream and detailed video stream footage are matched, the geographic location coordinates and collection timestamps corresponding to the trajectory are associated, the behavior determination results, video footage and trajectory data are integrated, and the law enforcement record is generated and saved.

[0051] Example 4 This embodiment continues the obstacle avoidance and tracking route L with obstacle avoidance constraints generated in Embodiment 3. Based on the close-range high-definition perception data after the UAV approaches the target, it fully executes the entire process of depth feature extraction, high- and low-altitude dual-source feature spatiotemporal matching, scene adaptation target motion model construction, continuous trajectory iterative fitting, and standardized law enforcement record generation. The specific process is as follows: Drone depth feature extraction: After receiving the dispatch command, the UAV autonomously flies along the flight path L to approach the target vehicle. The onboard high-definition vision equipment continuously collects close-range detailed video streams. To address the image defects caused by flying in the forest, atmospheric perspective distortion, and uneven lighting, Gaussian noise reduction and perspective inverse distortion correction are performed on each frame of the image in sequence to repair image distortion and remove noise, and output standardized high-definition detailed images.

[0052] The corrected image is fed into a drone-specific multi-branch feature extraction network. A multi-layered progressive convolutional network progressively extracts local details of the vehicle's front structure, body texture, and rear shape. An independent mean pooling branch filters out light and shadow interference, stably extracting the overall dominant color features of the vehicle body. A dedicated license plate sub-network locks onto local areas of the license plate, enhancing the extraction of character stroke texture features. The vehicle outline vector, body color vector, and license plate character vector are unified in dimension and normalized and weighted, eliminating differences in multi-branch dimensions to generate a unified depth feature E.

[0053] Spatiotemporal matching of high and low altitude features and target identification: The system establishes a spatiotemporal matching constraint system for high and low altitude dual devices, extracting the time sequence marker T0 of the high point video stream, the geographic coordinates G of the high point target, and the real-time acquisition timestamp and close-range positioning coordinates of the UAV detailed video stream, respectively, and constructing dual matching conditions of temporal continuity constraint and spatial position constraint.

[0054] The cosine similarity between the initial feature D and the depth feature E of the target is calculated for each group. The calculated value is compared with the preset similarity threshold Th to complete the initial screening. The time difference and spatial coordinate offset of the two groups of features are checked for the second time. Only feature pairs whose time difference and spatial deviation both fall within the preset allowable error range are retained. All qualified feature combinations are summarized and interfering pairs are eliminated. A unique and unified vehicle identifier is assigned to the same vehicle to complete the cross-device target identity binding for coarse high-point recognition and fine UAV recognition, avoiding the problem of multiple vehicles being confused.

[0055] Target motion model construction: The model is defined by three types of input data sources: The first type is identity binding data, which includes the initial features D, deep features E, and unified vehicle identifiers of the matched target, used to lock the target body without jumping around throughout the process; the second type is road network constraint data, taken from the global obstacle distribution map, which includes all spatiotemporal boundaries of the road network's feasible driving areas and obstacle-prohibited areas, and reasonable boundaries to restrict vehicle movement; the third type is observation sampling data, taken from the blind-spot-free obstacle avoidance tracking route L, which includes all observation points of the UAV, corresponding sampling time sequences, and matching the image acquisition cycle of the device.

[0056] Define the core state variables of the model: Select the vehicle's two-dimensional geographic coordinates, instantaneous driving speed, and driving heading angle as the basic state variables, and build a continuous state update equation system as the basic computational framework of the model.

[0057] To adapt to the motion characteristics of this scenario, adaptive correction factors are added: For situations such as vehicles traveling at a constant speed, accelerating to overtake, and changing lanes in emergency situations on highways, as well as situations such as slow driving, obstacle avoidance, and short-term delays due to tree obstruction on forest roads, three types of adaptive correction factors, namely speed change, direction change, and obstruction, are added to the equation set to solve the problem of excessive prediction deviation of the general motion model under complex road conditions.

[0058] Embedded road network spatiotemporal constraints complete model boundary limitation: obstacles and no-fly zones within the distribution map are used as invalid solution elimination conditions for the model. During the calculation process, motion states that exceed the actual road range are automatically filtered to ensure that the model's motion logic conforms to the physical traffic rules of the on-site road, thus completing the construction of the target motion model M1 adapted to this hybrid road network scenario.

[0059] The correlation between trajectory iterative fitting and the Hungarian algorithm: After model M1 is built, the target position and historical driving speed data of continuous historical detection frames of high-point video are retrieved and all are input into the Kalman filter. The theoretical coordinates of the target in the current frame are estimated based on the filter state prediction equation, and the prediction error is corrected based on the observation equation and historical sampling data. The optimal predicted position of the target is then output.

[0060] Construct an association cost matrix: Each element C(i,j) in the matrix represents the cost of associating the i-th detection box with the j-th trajectory; for the matching between the target position predicted by the Kalman filter and the detection box in the current frame, the cost is (1 - the intersection-union ratio of the predicted position and the detection box); for the association involving the matching between the depth feature E and the initial feature D of the target, the cost is (1 - cosine similarity); for detection box and trajectory pairs that cannot be associated in physical space after verification, the corresponding cost is set to infinity.

[0061] The correlation cost matrix is ​​solved using the Hungarian algorithm to achieve optimal target matching across frames. The predicted coordinates output by the filter are globally optimally matched with the real-time target positions acquired by the UAV, and the coordinate prediction deviation is corrected frame by frame. All state parameters of the target position, velocity, and heading are continuously updated iteratively, and the discrete positioning points in the full time series are smoothly fitted and temporally continuously correlated to finally generate a complete target vehicle driving feature Q with no identification breakpoints, low positional deviation, and close to the actual driving state of the vehicle.

[0062] Law enforcement record generation: The system retrieves preset rules for highway and forest area monitoring zones, and verifies the vehicle's travel path segment by segment based on the complete target vehicle's driving characteristics Q. It automatically determines whether there are various violations such as entering ground-controlled restricted areas, illegally parking in emergency lanes, illegally passing through forest roads, or prolonged lingering on roads, thus completing automated violation determination.

[0063] The system accurately associates high-point panoramic video footage and close-up detail video footage obtained by drones with the target throughout the entire process, binds continuous geographic coordinates to the trajectory, and marks the time sequence T0 for frame-by-frame acquisition. It integrates four types of data—vehicle feature information, complete spatiotemporal trajectory, dual-source video evidence, and violation judgment conclusions—into a standardized and encapsulated data set to generate a standardized law enforcement record that has a complete and traceable evidence chain and complies with traffic management non-site enforcement standards. Simultaneously, it completes local solidification and storage, realizing an integrated closed-loop operation of air-ground collaborative identification, flight route scheduling, feature matching, trajectory tracking, and evidence archiving.

[0064] Specifically, the traffic enforcement and tracking system based on the linkage between drones and high-altitude video includes: High-point perception module: acquires high-point video streams of traffic scenes, inputs them into a pre-built target detection model to identify vehicles and abnormal events, and outputs obstacle information and initial target features; Map building module: retrieves pre-stored static road network data, converts the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and maps it to the static road network data, merges and removes duplicates in overlapping areas, marks the obstacle-occupied area and duration, and generates a global obstacle distribution map; Flight path generation module: Reads the global obstacle distribution map, extracts the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generates a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguishes between dynamic and static obstacles based on the obstacle duration and adopts different detour methods to generate a blind-spot-free obstacle avoidance and tracking flight path. Record generation module: performs spatiotemporal matching between the depth features and the target initial features to identify the target vehicle, obtains the target vehicle's driving characteristics, and generates a law enforcement record based on the target vehicle's driving characteristics.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A traffic enforcement and tracking method based on the linkage of drones and high-point video, characterized in that, Includes the following steps: Acquire high-point video streams of traffic scenes, input them into a pre-built target detection model to identify vehicles and abnormal events, and output obstacle information and initial target features; Retrieve pre-stored static road network data, convert the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and map it onto the static road network data, merge and remove duplicates in overlapping areas, mark the obstacle-occupied area and duration, and generate a global obstacle distribution map; Read the global obstacle distribution map, extract the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generate a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguish between dynamic and static obstacles based on the obstacle duration and use different detour methods to generate a blind-spot-free obstacle avoidance and tracking route. The specific process for generating the global obstacle distribution map is as follows: Retrieve the pre-stored static road network data, which includes road coordinates, lane distribution, regulatory area boundaries, and geographic coordinates of no-fly zones for drones to avoid. Receive the obstacle information with a timestamp, the obstacle information carrying the pixel coordinates and bounding box size of static roadblocks, dynamic people and vehicles, and forest smoke points respectively; The pre-configured mapping and transformation relationship between the pixel coordinate system and the on-site geographic coordinate system is invoked. Based on the installation height, pitch angle and lens distortion correction coefficient of the high-point camera, scale compensation and projection conversion are completed. The pixel coordinates and bounding box size in the obstacle information are converted into geographic coordinates and geographic coverage that are consistent with the static road network data. The converted geographic boundary is superimposed and mapped to the corresponding position in the static road network data. The spatial neighborhood comparison algorithm is invoked to traverse the overlapping obstacle coverage areas within the same road network unit and calculate the area overlap ratio. If the area overlap ratio exceeds the preset fusion threshold, it is determined to be a spatially overlapping obstacle area. For obstacles of the same type, the duplicate marked areas are directly deleted, and obstacles of different types are merged and fused to form overlapping boundaries. Based on the timestamp and combined with the preset obstacle lifecycle determination rules, each fused obstacle area is assigned a temporal value to distinguish it, the obstacle-occupied area and the duration of the obstacle are marked, and the global obstacle distribution map with spatiotemporal attributes is generated. The specific planning process for the blind-spot-free obstacle avoidance and tracking route is as follows: Read the global obstacle distribution map with spatiotemporal attributes, and extract all obstacle-occupied areas, obstacle duration, and no-fly zone boundaries within the distribution map; Using the geographic coordinates of the target vehicle as the end point of the flight path and the coordinates of the UAV airport as the starting point of the flight path, the basic flight path is generated by incorporating multiple constraints such as flight safety distance, maximum detour threshold and evidence collection field of view coverage. The basic flight path is segmented and verified, and the path segments that intersect with the obstacle-occupied area and the no-fly zone are eliminated; The dynamic or static nature of an obstacle is determined based on its duration: if the duration is less than a preset duration threshold, it is determined to be a dynamic obstacle, and the drone is controlled to detour around it at a distance less than a first detour threshold; if the duration is greater than or equal to the preset duration threshold, it is determined to be a static obstacle or a permanent no-fly zone, and the drone is controlled to detour around it at a distance greater than a second detour threshold, where the second detour threshold is greater than the first detour threshold. Synchronously match the UAV's flight speed and image acquisition pitch angle to generate the blind-spot-free obstacle avoidance and tracking route; Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, acquire detailed video streams of the target vehicle and extract depth features; the depth features are spatiotemporally matched with the target initial features to identify the target vehicle, acquire the target vehicle's driving characteristics, and generate an enforcement record based on the target vehicle's driving characteristics.

2. The method according to claim 1, characterized in that, The pre-built target detection model is obtained based on the following method: A dedicated image dataset was acquired on-site. The dedicated image dataset includes image materials covering various scenarios such as highways, urban roads, and forest areas, including morning fog, midday strong light, evening backlight, and nighttime supplemental lighting conditions. The labeled objects include vehicles, pedestrians, roadblocks, smoke points, and illegally parked targets. A lightweight single-stage object detection network is used to build a basic recognition framework. The dedicated image dataset is split into a training set and a validation set. The network is iteratively trained and its performance is verified separately. After the training reaches the target, the lightweight single-stage object detection network is pruned, quantized, and compressed. The network weight parameters are continuously adjusted based on the verification results. Once the parameters of the lightweight single-stage target detection network converge and stabilize, the network is solidified to generate the target detection model.

3. The method according to claim 2, characterized in that, The specific output process of the obstacle information and the initial features of the target is as follows: The system receives the high-point video stream of the traffic scene, performs image denoising and size normalization preprocessing, and then inputs it into the target detection model. The system extracts shallow texture and deep semantic features of the image frame by frame through the lightweight single-stage target detection network, performs pixel-level localization of static road obstacles, dynamic people and vehicles, and forest smoke points, and outputs the obstacle information. For the identified vehicle target, global contour features, main body color features, and texture details of the front and body are extracted and integrated to generate the initial features of the target.

4. The method according to claim 1, characterized in that, The specific extraction process of the depth features is as follows: Based on the blind-spot-free obstacle avoidance and tracking route, the UAV is controlled to autonomously navigate to the target area, and the detailed video stream of the target vehicle is collected in real time and image noise reduction and perspective distortion correction are performed. Based on the corrected detailed video stream, local vehicle detail features, global body color features, and character texture features are extracted. The local detail features of the vehicle, the global color features of the vehicle body, and the character texture features are normalized and fused to obtain the depth features with unified dimensions.

5. The method according to claim 1, characterized in that, The specific process of spatiotemporal matching between the depth features and the initial target features is as follows: Extract the acquisition timestamps and geographic coordinates corresponding to the high-point video stream and the detail video stream respectively, and construct spatiotemporal constraints; Calculate the cosine similarity between the depth features and the initial features of the target, and perform an initial feature comparison with a preset similarity threshold; When the similarity is lower than the preset similarity threshold, the corresponding feature pair is directly discarded; when the similarity reaches the preset similarity threshold, feature pairs whose video acquisition time difference and spatial coordinate deviation fall within the preset allowable range are selected; feature combinations that simultaneously meet the similarity threshold and the spatiotemporal constraints are retained, bound to a unified vehicle identifier, and the target vehicle is identified.

6. The method according to claim 5, characterized in that, The specific process for obtaining the target vehicle's driving characteristics includes: Based on the initial features of the target, the depth features, and the unified vehicle identifier bound by the spatiotemporal matching, a target motion model is constructed by combining the spatiotemporal obstacle data corresponding to the global obstacle distribution map with the blind-spot-free obstacle avoidance and tracking route. The system retrieves the target position and driving speed data corresponding to the historical detection frames of the high-point video stream, predicts the target's current frame coordinates, associates the predicted coordinates with the target's real-time position collected by the UAV along the blind-spot-free obstacle avoidance tracking route, and corrects the target coordinate prediction values; iteratively updates the target's real-time position parameters, continuously fits and associates the temporal positioning parameters, and generates the target vehicle's driving characteristics.

7. The method according to claim 6, characterized in that, The specific process for generating the law enforcement record is as follows: Based on the target vehicle's driving characteristics, the corresponding high-point video stream and detailed video stream footage are matched, the geographic location coordinates and collection timestamps corresponding to the trajectory are associated, the behavior determination results, video footage and trajectory data are integrated, and the law enforcement record is generated and saved.

8. A traffic enforcement and tracking system based on the linkage of unmanned aerial vehicles and high-point video, used to implement the method described in any one of claims 1-7, characterized in that, include: High-point perception module: acquires high-point video streams of traffic scenes, inputs them into a pre-built target detection model to identify vehicles and abnormal events, and outputs obstacle information and initial target features; Map building module: retrieves pre-stored static road network data, converts the obstacle information into geographic coordinates through a pre-configured pixel-geographic coordinate mapping relationship and maps it to the static road network data, merges and removes duplicates in overlapping areas, marks the obstacle-occupied area and duration, and generates a global obstacle distribution map; Flight path generation module: Reads the global obstacle distribution map, extracts the obstacle-occupied area, the obstacle duration, and the no-fly zone boundary; generates a basic flight path with the target vehicle's geographical coordinates as the endpoint and the UAV airport coordinates as the starting point; distinguishes between dynamic and static obstacles based on the obstacle duration and adopts different detour methods to generate a blind-spot-free obstacle avoidance and tracking flight path. Record generation module: performs spatiotemporal matching between the depth features and the target initial features to identify the target vehicle, obtains the target vehicle's driving characteristics, and generates a law enforcement record based on the target vehicle's driving characteristics.

Images