Vehicle identification and traffic incident detection method and system based on visual angle of unmanned aerial vehicle

Abstract

The invention relates to the technical field of intelligent traffic monitoring, in particular to a vehicle identification and traffic incident detection method and system based on the view angle of an unmanned aerial vehicle, and the method comprises the steps: obtaining the aerial traffic video data of the unmanned aerial vehicle and the flight control data of the unmanned aerial vehicle; performing target identification on the image in the video data by adopting a target detection algorithm to obtain vehicle information; tracking the vehicle in the video data by using a target tracking algorithm, and mapping the identified vehicle to a unified coordinate system in combination with the flight control data to generate vehicle trajectory data; constructing a comprehensive road description model based on the vehicle trajectory data and the vehicle information; and forming a traffic incident recognition module based on the comprehensive road description model, and recognizing the input aerial video according to a preset traffic incident recognition rule. According to the invention, by optimizing the vehicle identification algorithm, constructing the road description model and designing the efficient traffic event identification module, the application efficiency of the unmanned aerial vehicle aerial video in the traffic monitoring field is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic monitoring technology, and in particular to a vehicle identification and traffic incident detection method and system based on the perspective of an unmanned aerial vehicle (UAV). Background Art

[0002] With increasing traffic volume, traditional roadside traffic monitoring equipment faces limitations in coverage and deployment flexibility. Drones, with their advantages such as high maneuverability and wide field of view, are becoming an important auxiliary means of traffic monitoring. However, existing drone traffic monitoring technology still has shortcomings in small-target vehicle recognition, multi-dimensional traffic data integration, and intelligent identification of traffic events.

[0003] CN202010406068.5 discloses a method and system for automatic monitoring of traffic incidents involving drones on highways. The method includes obtaining real-time video data collected when a drone automatically cruises the current road section, performing video image processing on the real-time video data to obtain a video image sequence, extracting a target vehicle from the video image sequence using a dynamic target extraction algorithm to obtain category information and contour information of the target vehicle, calculating the position of the target vehicle in the next frame using a target tracking algorithm based on the category information and contour information of the target vehicle in the current frame, calculating the actual distance between adjacent frames of the target vehicle based on the contour information of the same target vehicle in two adjacent frames of video images, calculating the speed of the target vehicle based on the actual distance between adjacent frames of the target vehicle, and discerning the driving behavior of the target vehicle based on the speed and contour information of the target vehicle. However, this technical solution has obvious flaws: first, relying on contour information for vehicle tracking and distance calculation is prone to errors in complex scenarios such as vehicle occlusion and lighting changes; second, the method of calculating vehicle speed based solely on the distance between adjacent frames is not accurate enough when the drone is moving or the perspective changes; third, this method mainly relies on single speed and contour features for behavior judgment, and lacks comprehensive analysis of multi-dimensional information such as the relationship between vehicles and road driving direction, resulting in limited accuracy and applicability of traffic event recognition. Summary of the Invention

[0004] In view of this, the present invention proposes a vehicle identification and traffic incident detection method and system based on the perspective of a drone to solve the problems that the existing drone traffic monitoring technology still has shortcomings in small target vehicle identification, multi-dimensional traffic data fusion and intelligent identification of traffic incidents.

[0005] The technical solution of the present invention is implemented as follows: The present invention provides a vehicle identification and traffic incident detection method based on the perspective of an unmanned aerial vehicle, comprising the following steps:

[0006] S1. Obtain UAV aerial traffic video data and UAV flight control data;

[0007] S2. Use target detection algorithm to identify the image in the video data and obtain vehicle information;

[0008] S3. Based on the identified vehicle, a target tracking algorithm is used to track the vehicle in the video data, and the identified vehicle is mapped to a unified coordinate system in combination with the flight control data to generate vehicle trajectory data;

[0009] S4. Build a comprehensive road description model based on the vehicle trajectory data and vehicle information, where the comprehensive road description model includes vehicle motion parameters, road driving directions, and relationships between vehicles;

[0010] S5. A traffic event recognition module is formed based on the comprehensive road description model, and the input aerial video is recognized according to preset traffic event recognition rules to obtain a recognition result.

[0011] Based on the above technical solution, preferably, step S2 specifically includes:

[0012] S21, extracting the first frame image of the video data, and preprocessing the first frame image;

[0013] S22. Input the preprocessed image into a pre-trained target detection model, analyze the extracted features through the target detection model, and output the identified vehicle information, wherein the vehicle information includes the vehicle category and vehicle bounding box data, and the vehicle bounding box data includes the vehicle center point coordinates, vehicle width, vehicle height, and vehicle rotation angle.

[0014] On the basis of the above technical solution, preferably, the target detection model is an improved YOLOv11-OBB model, and the network structure of the target detection model includes:

[0015] The backbone network includes multiple convolutional layers and an SPD-Conv module. The SPD-Conv module consists of an SPD layer and a non-strided convolutional layer. The SPD layer converts the input feature map from space to depth according to the scaling factor, divides the feature map into multiple sub-maps and concatenates them along the channel dimension. The non-strided convolutional layer uses a convolution with a stride of 1 to adjust the channel dimension.

[0016] The neck network adopts a path aggregation network structure to perform top-down and bottom-up bidirectional feature fusion on the multi-scale feature maps output by the backbone network;

[0017] The detection head adopts a task-aligned dynamic detection head structure to perform vehicle identification and positioning on the multi-scale fusion feature map output by the neck network. It includes a task decomposition module, a classification branch, and a positioning branch. The positioning branch uses deformable convolution for feature enhancement. The classification branch and the positioning branch collaborate through task interaction features, and respectively set a scale adjustment layer to adjust the scale of the output.

[0018] Based on the above technical solution, preferably, step S3 specifically includes:

[0019] S31. Based on the identified vehicles, a multi-target tracking algorithm is used to perform data association and cross-frame tracking on vehicle targets in consecutive frames of the traffic video data to generate a motion trajectory of each vehicle in the image coordinate system;

[0020] S32, establishing a coordinate mapping relationship between the image coordinate system and the camera coordinate system according to the UAV flight control data;

[0021] S33. Convert the vehicle motion trajectory in the pixel coordinate system to the geographic reference coordinate system through a coordinate mapping relationship to generate vehicle trajectory data in a unified coordinate system.

[0022] Based on the above technical solution, preferably, step S32 includes:

[0023] According to the coordinate mapping relationship, the image coordinate system of the first frame is mapped to the camera coordinate system, and the camera coordinate system of the first frame is used as the unified coordinate system;

[0024] Map the image coordinate system of the nth frame to the camera coordinate system based on the coordinate mapping relationship, and project the camera coordinate system of the nth frame to the unified coordinate system;

[0025] The coordinate mapping process uses the following formula:

[0026]

[0027] Where, is the vehicle coordinate when shooting the first frame, is the vehicle coordinate when shooting the nth frame, K -1 is the inverse matrix of the camera intrinsic parameter matrix, T n_1 The transformation matrix of the camera coordinate system when shooting the nth frame is converted to the camera coordinate system when shooting the first frame, R n_1 is the rotation matrix, t n_1 is the translation vector; K n is the camera intrinsic parameter matrix; f x is the focal length in the x-axis direction of the image; f y is the focal length in the y-axis direction of the image; c xis the x-coordinate of the image principal point (projection of the optical center on the image plane); c y is the y coordinate of the image principal point; s is the tilt factor of the pixel coordinate axis.

[0028] Based on the above technical solution, preferably, step S4 specifically includes:

[0029] S41. Calculating motion parameters of each vehicle based on the vehicle trajectory data and vehicle information, where the motion parameters include at least real-time speed and acceleration;

[0030] S42, extracting the road driving direction and spatial range of the lane space based on the vehicle trajectory data, vehicle information and motion parameters;

[0031] S43. Analyze and quantify inter-vehicle relationships based on vehicle position information in a road travel direction coordinate system, where the inter-vehicle relationships include at least relative distances and relative speeds between the vehicles.

[0032] S44. Construct a comprehensive road description model of the vehicle based on the motion parameters, the road driving direction, and the relationship between vehicles.

[0033] Based on the above technical solution, preferably, the process of extracting the road driving direction in the lane space in step S42 includes: determining the spatial distribution pattern of vehicle movement based on vehicle trajectory data, identifying the spatial boundary range of the lane in combination with vehicle information, calculating the velocity vector of each vehicle and normalizing it, performing cluster analysis on the road driving direction vector using a clustering algorithm, identifying the dominant driving flow direction in the lane space, eliminating abnormal driving direction vectors through statistical methods, and determining the road driving direction in the lane space.

[0034] Based on the above technical solution, preferably, step S43 specifically includes:

[0035] S431, obtaining the trajectory data and vehicle information of all vehicles in the current frame, and initializing the current vehicle index to the first vehicle;

[0036] S432, taking the current vehicle as a benchmark, sequentially selecting other vehicles as target vehicles for relationship analysis;

[0037] S433, calculate the Euclidean distance between the current vehicle and the target vehicle, and determine whether the distance is less than a preset distance threshold. If not, select the next target vehicle and return to step S432; if so, proceed to the next step;

[0038] S434: Calculate the relative angle between the vehicles based on the current vehicle orientation vector and the vector connecting the two vehicles, and determine the type of spatial relationship between the vehicles based on the relative angle range;

[0039] S435: Calculate and record the relative distance and relative speed between vehicles that determine the spatial relationship type, and store the relationship type between the vehicles in a vehicle relationship matrix;

[0040] S436: Determine whether all vehicles have been traversed. If not, update the counter and continue traversing. If completed, output the relationship between vehicles.

[0041] Based on the above technical solution, preferably, step S5 specifically includes:

[0042] S51. Constructing a traffic event recognition module based on the comprehensive road description model, wherein the traffic event recognition module includes preset traffic event recognition rules;

[0043] S52: Input the aerial video to a traffic event recognition module, which analyzes and processes the input data based on traffic event recognition rules;

[0044] S53. When the analysis result meets the conditions set by the discrimination rule, determine that a corresponding traffic event has occurred and output a recognition result; the type of the traffic event includes at least one of vehicle congestion, abnormal parking, illegal lane change, wrong-way driving, and potential traffic accident.

[0045] The present invention also provides a vehicle identification and traffic incident detection system based on the perspective of a drone, and a data acquisition module for collecting traffic video data and drone flight control data;

[0046] Traffic identification module, including:

[0047] a vehicle identification unit for extracting vehicle information of each vehicle in traffic video data;

[0048] The vehicle trajectory extraction unit is used to associate and track the identified vehicles, and map the vehicle positions to a unified coordinate system in combination with the UAV flight control data, thereby generating the target vehicle's trajectory data;

[0049] A road description model unit, used to integrate vehicle trajectory data and vehicle information and build a comprehensive road description model;

[0050] The event recognition unit is used to build a traffic event recognition module based on the comprehensive road description model, recognize the input aerial video according to the preset traffic event recognition rules, and obtain the recognition result.

[0051] The vehicle identification and traffic incident detection method and system based on the drone perspective of the present invention have the following beneficial effects compared with the existing technology:

[0052] (1) The present invention realizes a complete technical chain from drone aerial video data acquisition, vehicle accurate identification, trajectory tracking to intelligent identification of traffic events by constructing a vehicle identification and traffic event detection method based on the perspective of drones. This solution integrates drone flight control data and video data, combines improved target detection algorithms and multi-target tracking technology, and constructs a comprehensive road description model that includes vehicle motion parameters, driving directions, and relationships between vehicles. Ultimately, it realizes accurate identification of various traffic events such as vehicle congestion, abnormal parking, illegal lane changes, and traffic accidents. Compared with traditional roadside monitoring equipment, this solution has the characteristics of wide coverage, flexible deployment, and obvious perspective advantages. It effectively makes up for the shortcomings of existing traffic monitoring technology and provides a new technical path for intelligent traffic management.

[0053] (2) The present invention introduces the SPD-Conv module into the target detection model backbone network. The SPD layer converts the input feature map from space to depth according to the scaling factor, divides the feature map into multiple sub-maps and splices them along the channel dimension, and then uses a non-stepped convolution layer to adjust the channel dimension. This technology avoids the direct discarding of pixel information in the traditional downsampling process, while reducing the size of the feature map while maintaining complete feature information, significantly improving the model's recognition accuracy and detection stability for small target vehicles under the perspective of drone aerial photography; adopting a task-aligned dynamic detection head structure, through a dynamic feature alignment mechanism, the vehicle classification and position regression tasks can share optimized feature representations, effectively reducing the spatial inconsistency problem, and improving the accuracy of vehicle detection and positioning accuracy, especially the ability to recognize vehicles with different postures in complex aerial photography scenes.

[0054] (3) This invention establishes a mapping relationship between the image coordinate system and the camera coordinate system by combining the UAV flight control data, and uses a transformation matrix to unify the vehicle coordinates captured at different times into a geographic reference coordinate system. This technology effectively solves the problem of inconsistent vehicle trajectory coordinates when the UAV is in motion, ensures the spatial accuracy and temporal continuity of vehicle trajectory data, and provides a reliable data foundation for subsequent motion parameter calculation and trajectory analysis.

[0055] (4) The comprehensive road description model constructed by this invention integrates multi-dimensional information such as vehicle motion parameters, road travel direction, and inter-vehicle relationships. It uses a clustering algorithm to identify the dominant travel direction within the lane space and quantifies the relative distance, relative speed, and spatial relationship type between vehicles based on vehicle position information. Compared with traditional methods that rely solely on a single feature, this model can provide a more comprehensive and accurate description of road traffic conditions, significantly improving the accuracy and applicability of traffic event recognition, and providing strong support for event detection in complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a flow chart of the vehicle identification and traffic incident detection method based on the drone perspective of the present invention;

[0058] Figure 2 Schematic diagram of the SPD-Conv module of the present invention;

[0059] Figure 3 Schematic diagram of the overall structure of TADDH of the present invention;

[0060] Figure 4 Schematic diagram of the Inner-loU loss function of the present invention;

[0061] Figure 5 Schematic diagram of the coordinate system conversion process of the present invention;

[0062] Figure 6 Schematic diagram of the relative position relationship between the drone and the camera of the present invention;

[0063] Figure 7 It is a schematic diagram of the polymerization result of the present invention;

[0064] Figure 8 Schematic diagram of lane space of the present invention;

[0065] Figure 9 Schematic diagram of the road travel direction coordinate system of the present invention;

[0066] Figure 10 A schematic diagram of the relationship between vehicles of the present invention;

[0067] Figure 11 The flowchart of extracting the relationship between vehicles of the present invention;

[0068] Figure 12 This is an example diagram of extracting the relationship between vehicles of the present invention;

[0069] Figure 13 This is a rendering of the vehicle after emergency parking according to the present invention;

[0070] Figure 14 This is a diagram showing the effect of a vehicle driving in reverse or reversing according to the present invention after marking;

[0071] Figure 15 This is the effect diagram after the congested vehicles are marked according to the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0073] like Figure 1 As shown, the present invention provides a method for vehicle identification and traffic incident detection based on the perspective of a drone, comprising the following steps:

[0074] S1. Obtain UAV aerial traffic video data and UAV flight control data;

[0075] S2. Use target detection algorithm to identify the image in the video data and obtain vehicle information;

[0076] S3. Based on the identified vehicle, a target tracking algorithm is used to track the vehicle in the video data, and the identified vehicle is mapped to a unified coordinate system in combination with the flight control data to generate vehicle trajectory data;

[0077] S4. Build a comprehensive road description model based on the vehicle trajectory data and vehicle information, where the comprehensive road description model includes vehicle motion parameters, road driving directions, and relationships between vehicles;

[0078] S5. A traffic event recognition module is formed based on the comprehensive road description model, and the input aerial video is recognized according to preset traffic event recognition rules to obtain a recognition result.

[0079] This invention overcomes the limited coverage of traditional roadside monitoring equipment by fusing drone aerial traffic video data with flight control data, combined with target detection and tracking algorithms. It can achieve accurate vehicle identification and trajectory tracking over a wider range. Through coordinate system-mapping technology, it effectively solves the problem of vehicle positioning accuracy when the drone is in motion, significantly improving the stability and accuracy of vehicle tracking. By integrating multi-dimensional information such as vehicle motion parameters, road direction, and inter-vehicle relationships, it significantly improves the accuracy and robustness of traffic event identification compared to traditional methods that rely solely on single features, effectively addressing the shortcomings of existing drone traffic monitoring technology in small target identification, multi-dimensional data fusion, and intelligent event detection.

[0080] Specifically, step S1 primarily uses the drone platform to acquire the basic data required for traffic monitoring. This data consists of two key components: aerial traffic video data and flight control data. The drone, equipped with a high-resolution camera, cruises over a pre-defined traffic monitoring area, continuously capturing road traffic conditions from a bird's-eye perspective. The camera's pitch angle is fixed at -90° (i.e., vertically downward), ensuring a stable vertical perspective. The aerial traffic video data is recorded at a fixed frame rate, providing a continuous image sequence for subsequent vehicle identification and trajectory tracking. The drone's flight control system records key flight parameter data at regular intervals, including the drone's three-dimensional position coordinates, flight velocity vector data, flight altitude, drone attitude data (including pitch, roll, and yaw angles), and camera attitude parameters. The aerial traffic video data and flight control data are strictly synchronized in time, ensuring that each frame of video image corresponds to the correct flight control parameters. After data acquisition is complete, the aerial traffic video data and flight control data are stored separately, providing reliable input sources for subsequent data processing and analysis.

[0081] Furthermore, step S2 specifically includes:

[0082] S21, extracting the first frame image of the video data, and preprocessing the first frame image;

[0083] S22. Input the preprocessed image into a pre-trained target detection model, analyze the extracted features through the target detection model, and output the identified vehicle information, wherein the vehicle information includes the vehicle category and vehicle bounding box data, and the vehicle bounding box data includes the vehicle center point coordinates, vehicle width, vehicle height, and vehicle rotation angle.

[0084] Among them, the target detection model of the present invention is an improved YOLOv11-OBB (You Only Look Once version11 with Oriented Bounding Box) model, which uses YOLOv11-OBB as the basic vehicle recognition algorithm. The algorithm can accurately output key information such as the length, width, and orientation of the vehicle. To further improve the recognition effect of small target vehicles, the present invention constructs a small target detection framework that integrates YOLOv11-OBB and SPD convolution. The network structure of the target detection model includes a backbone network, a neck network and a detection head, wherein the backbone network contains multiple convolutional layers and an SPD-Conv module (Space-to-DepthConvolution). The SPD-Conv module consists of an SPD layer and a non-stepped convolution layer. The SPD layer converts the input feature map from space to depth according to the scaling factor, divides the feature map into multiple sub-maps and splices them along the channel dimension. The non-stepped convolution layer uses a convolution with a step size of 1 to adjust the channel dimension; the neck network adopts a path aggregation network structure to perform top-down and bottom-up bidirectional feature fusion on the multi-scale feature map output by the backbone network; the detection head adopts a task-aligned dynamic detection head structure to perform vehicle identification and positioning on the multi-scale fusion feature map output by the neck network, which includes a task decomposition module, a classification branch and a positioning branch. The positioning branch uses deformable convolution for feature enhancement. The classification branch and the positioning branch collaborate through task interaction features, and scale adjustment layers are set separately to adjust the scale of the output.

[0085] Specifically, drone aerial videos are shot at a high altitude, and when the weather is bad, visibility may be low, resulting in unclear vehicles in the picture. Traditional convolutional neural networks, due to their limited receptive field and linear convolution operations, find it difficult to capture the global features of these targets, resulting in reduced recognition performance. SPD-Conv is a new building block for improving convolutional neural networks, mainly for performance optimization in low-resolution images and small target recognition tasks. SPD-Conv can achieve fine-grained feature learning, thereby preventing feature loss. The core principle of SPD-Conv is to replace traditional strided convolution and pooling layers with space-to-depth conversion, thereby reducing information loss and improving the precision of feature extraction. Figure 2 As shown in the figure, SPD-Conv consists of an SPD layer and a non-strided convolutional layer. The SPD layer is responsible for performing spatial-to-depth conversion, avoiding direct pixel discarding. For example, when the scaling factor is 2, the input feature map is divided into four sub-images. After concatenating them along the channel dimension, the spatial size is halved while the number of channels is quadrupled, thus preserving all information. The non-strided convolutional layer uses convolution with a stride of 1 to adjust the channel dimension, reducing computational complexity.

[0086] SPD-Conv can split the feature map to obtain sub-feature maps, achieving fine-grained feature extraction. After a convolution with a step size of 1, further optimization is achieved by compressing the number of channels of the feature map. By splitting the feature map and combining it with convolution, a downsampling effect can be achieved. Adding an SPD module to the feature extraction network for optimization can reduce the size of the spatial dimension while increasing the size of the depth dimension, which helps to reduce the computational complexity while maintaining or improving the performance of the model. Introducing SPD-Conv into the YOLOv11-OBB model can enhance the model's recognition performance in small target recognition. Especially when processing aerial images with large variations in target size and density, SPD-Conv, with its unique convolutional structure, can efficiently capture the fine-grained features of small targets, thereby overcoming the shortcomings of traditional methods in dense small target recognition.

[0087] Figure 3The overall architecture of the Task Align Dynamic Detection Head (TADDH) is shown. The left side of the figure shows the feature inputs for P3, P4, and P5. These features are derived from the multi-scale feature fusion output of the neck network. Each feature map undergoes two Conv_GN_3×3 convolutions (3×3 Convolution with GroupNormalization) for preliminary feature extraction and dimensionality rescaling. The processed features then enter the core Task Decomposition Module, which works in conjunction with the Generator Mask Offset Module to generate task-specific feature representations. The Generator Module is responsible for generating the offsets and masks required for the deformable convolutions. The features then enter the Deformable Convolutional Networks v2 (DCNV2) module for feature enhancement. DCNV2 utilizes the offsets and masks generated by the generator to adaptively spatially sample the feature maps, enhancing the model's adaptability to complex object shapes and pose variations. The features processed by the deformable convolutions are element-wise fused with the output of the Task Decomposition Module via a Multiply operation, enabling feature interaction between tasks. The fused features are fed into multiple parallel output branches: the classification branch, which includes the Conv_Cls convolution layer (classification convolution layer), responsible for vehicle classification; the regression branch, which includes the Conv_Reg convolution layer (regression convolution layer), responsible for predicting the position and size of the bounding box; and the scale adjustment layer, with each branch equipped with a scale layer (scaling layer) to normalize the output. Finally, the output of each branch is normalized using a sigmoid activation function to produce the final detection result.

[0088] TADDH first utilizes shared convolution technology to avoid the need for a large number of independent convolutional layers for each task, significantly reducing the number of model parameters. To address the issue of inconsistent object scales within each recognition head, TADDH introduces a scale layer to scale features, enabling the model to handle recognition tasks for objects of varying sizes. TADDH learns interactive features from feature extractors in multiple convolutional layers to generate joint features, enhancing inter-task collaboration and sharing information between classification and localization tasks, thereby reducing spatial inconsistency. Its localization branch utilizes DCNV2 and interactive features to generate DCNV2 offsets and masks, while the classification branch uses interactive features for dynamic feature selection. DCNV2 enhances feature representation through flexible sampling of feature maps, improving model adaptability and performance, particularly for complex objects. The generation of offsets and masks, as well as the dynamic adjustment of sampling points, are achieved through multi-level offset learning, which helps improve model accuracy. TADDH also incorporates a modulation mechanism that allows the network to dynamically adjust the location and intensity of sampling points, improving adaptability and enhancing inter-task collaboration for more accurate prediction results.

[0089] Integrating TADDH with OBB rotation annotation can significantly improve object recognition accuracy, especially when identifying objects with rotation or irregular orientation. OBB provides bounding box information for rotated objects, which helps address the inability of traditional rectangular bounding box methods to effectively handle rotated objects. Integrating TADDH with OBB rotation annotation allows the model to maintain robustness and adaptability even with limited training data or low-quality annotations.

[0090] The target detection model is optimized and trained through the loss function. In this paper, Inner-loU is proposed to assist the border to calculate IoU and improve the generalization ability, such as Figure 4 As shown, the size of the auxiliary bounding box is controlled by the ratio of the scale factor ratio.

[0091]

[0092] Where b1, b r 、b t 、b b The left, right, top and bottom coordinates of the auxiliary prediction box; are the left, right, top and bottom coordinates of the auxiliary real box; x c and y c Respectively represent the x and y coordinate values of the center point of the auxiliary prediction box; and They represent the x and y coordinate values of the center point of the auxiliary real frame respectively; w represents the width value of the auxiliary prediction frame, and h represents the height value of the auxiliary prediction frame; wgt is the width of the auxiliary real box, h gt is the height of the auxiliary real box; ratio is the scale factor, usually in the range of [0.5, 1.5].

[0093] The calculation formula of the loss function is as follows:

[0094]

[0095] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inner

[0096]

[0097] Where, L FII is the total loss function value, IOU inner is the ratio of the intersection and union of the auxiliary prediction box and the auxiliary real box scaled by the ratio; union is the union area of the auxiliary prediction box and the auxiliary real box; inner is the intersection area of the auxiliary prediction box and the auxiliary real box; k is a hyperparameter used to adjust the ratio of IOU inner the extent of contribution of the loss component; is a penalty term that calculates the sum of the normalized differences between the predicted box and the true box in their original width and height. It encourages the auxiliary predicted boxes generated by the model to be consistent with the auxiliary true boxes in overall size, even if their internal intersection is high at a specific ratio. ∈ is a very small positive number used to prevent the denominator from being zero and increase numerical stability.

[0098] When the ratio is 1, the size of the auxiliary border is equal to the size of the actual border. The value range of Inner-IoU loss is also [0,1]. The auxiliary border and the actual border only differ in scale, and the loss function is calculated in the same way. Compared with the IoU loss, when the ratio is less than 1, the auxiliary border size is smaller than the actual border, and its effective range of regression is smaller than the IoU loss, but its absolute value of the gradient is greater than the gradient obtained by the IoU loss, which can accelerate the convergence of high IoU samples; when the ratio is greater than 1, the larger-scale auxiliary border expands the effective range of regression, which has some benefits for the regression of low IoU. After introduction, when the ratio is less than 1, the ratio k Smaller, resulting in Larger, thus amplifying (1-IOU inner );When ratio>1, ratio k Larger, resulting in Smaller, thus reducing (1-IOUinner ). This modulation mechanism can more finely control the IOU under different ratio settings. inner The weight of the loss makes the model training more sensitive or stable to changes in ratio.

[0099] Furthermore, step S3 specifically includes:

[0100] S31. Based on the identified vehicles, a multi-target tracking algorithm is used to perform data association and cross-frame tracking on vehicle targets in consecutive frames of the traffic video data, and a motion trajectory of each vehicle in the image coordinate system is generated.

[0101] S32. Establishing a coordinate mapping relationship between the image coordinate system and the camera coordinate system based on the UAV flight control data. Specifically, the steps include: mapping the image coordinate system of the first frame to the camera coordinate system based on the coordinate mapping relationship, and using the camera coordinate system of the first frame as the unified coordinate system; mapping the image coordinate system of the nth frame to the camera coordinate system based on the coordinate mapping relationship, and projecting the camera coordinate system of the nth frame onto the unified coordinate system.

[0102] S33. Convert the vehicle motion trajectory in the pixel coordinate system to the geographic reference coordinate system through a coordinate mapping relationship to generate vehicle trajectory data in a unified coordinate system.

[0103] Specifically, to extract the trajectory from a moving drone video, the coordinates of the vehicle captured by the drone at all times are unified into a single coordinate system. The scale required for this unified coordinate system must be the scale of the real world. In this paper, the camera coordinate system at the time of capturing the first frame is used as the unified coordinate system.

[0104] The flight control data can be used to obtain the position, speed, altitude, attitude and other parameters recorded at fixed time intervals during the flight of the drone, as well as the camera's attitude data. Through the above data, the vehicle coordinates in each frame of the drone video can be projected into a unified coordinate system.

[0105] Taking the example of projecting the vehicle coordinates in the image coordinate system of the nth frame to the drone coordinate system when shooting the first frame, we will further explain how to project the vehicle coordinates in any frame to the drone coordinate system when shooting the first frame. First, project the coordinates from the image coordinate system of the nth frame to the camera coordinate system when shooting the nth frame; then project from the camera coordinate system when shooting the nth frame to the drone coordinate system when shooting the nth frame; finally, project from the camera coordinate system when shooting the nth frame to the camera coordinate system when shooting the first frame. The relative relationship between the coordinate systems during the conversion process is as follows: Figure 5 shown.

[0106] The conversion between the camera coordinate system and the image coordinate system only requires the camera's intrinsic parameter matrix. The conversion between the camera coordinate system and the drone coordinate system depends on the relative relationship between the drone and the camera. The present invention focuses on the recognition of traffic events from a bird's-eye view, so the camera pitch angle (rotation angle around the drone's horizontal axis) is fixed to -90° (i.e. vertically downward), and its rotation coordinate system is defined as follows: Figure 6 As shown in the figure, the relative positional relationship between the drone and its camera remains unchanged. The transformation matrix between the drone coordinate system and the camera coordinate system is a constant matrix, whose value remains unchanged throughout a flight. The transformation process requires three matrices: the transformation matrices for the drone coordinate system when capturing the nth frame and the first frame; the other two matrices are constant matrices.

[0107] Because the drone's sensors record its position and attitude data during flight, the transfer matrix between the drone's coordinate system at the time of the nth frame and the first frame is calculated based on the drone's position at the time of the nth frame and the first frame. The rotation matrix between the drone's coordinate system at the time of the nth frame and the first frame is calculated based on the drone's attitude data at the time of the nth frame and the first frame's position. Ultimately, the transformation matrix between the drone's coordinate system at the time of the nth frame and the first frame is obtained.

[0108] The final conversion formula is as follows:

[0109]

[0110] Where, is the vehicle coordinate when shooting the first frame, The vehicle coordinates when the nth frame is captured are homogeneous coordinates (x, y, z, 1) with a length of 4. T ;K -1 is the inverse matrix of the camera intrinsic parameter matrix, T n_1 The transformation matrix (i.e., the drone motion matrix) from the camera coordinate system when shooting the nth frame to the camera coordinate system when shooting the first frame is composed of the translation matrix and rotation matrix calculated from the drone displacement and rotation data between the shooting of the nth frame and the first frame; It is the transformation matrix from the camera coordinate system to the drone coordinate system when shooting the nth frame. It is a constant matrix, which is composed of the translation matrix and rotation matrix calculated by the relative translation and rotation between the camera coordinate system and the drone coordinate system; R n_1 is the rotation matrix, t n_1 is the translation vector; K n is the camera intrinsic parameter matrix; f x is the focal length in the x-axis direction of the image; f yis the focal length in the y-axis direction of the image; c x is the x-coordinate of the image principal point (projection of the optical center on the image plane); c y is the y coordinate of the image principal point; s is the tilt factor of the pixel coordinate axis.

[0111] Since the present invention is a traffic event recognition from a bird's-eye view, the pitch angle of the camera on the drone (the angle of rotation around the horizontal axis of the drone) is fixed at -90°. It can be assumed that the relative position of the drone position sensor and the optical center of the camera will not change. Therefore, it is assumed that the drone and the camera as a whole are regarded as rigid bodies and cannot be deformed. From the manifestation of the rigid body, we can know the transformation matrix X of a point a on the rigid body from time t1 to time t2, which means that the transformation matrix of other points on the rigid body except point a from time t1 to t2 is also X. Therefore, the spatial transformation matrix of the drone at any two moments can be regarded as the spatial transformation matrix of the optical center of the camera, and vice versa. Therefore, the process of projecting the coordinates in the image coordinate system of the nth frame to the drone coordinate system when the first frame was taken can be simplified as follows:

[0112] The image coordinate system of the nth frame is projected onto the camera coordinate system when the nth frame was taken;

[0113] The camera coordinate system when shooting the nth frame is projected onto the camera coordinate system when shooting the first frame.

[0114] The first step only requires knowing the camera's intrinsic parameter matrix. The second step uses the drone's position and attitude data when capturing the nth frame and the drone's position and attitude data when capturing the first frame to calculate the drone coordinate system transformation matrix at the two moments as the camera coordinate system transformation matrix at the two moments. The above transformation process can be simplified as follows: the camera coordinate system is originally described in the drone coordinate system, and the camera coordinate system transformation is described in the drone coordinate system. Under the premise that the relative position of the drone and camera remains unchanged, the displacement of the drone position sensor is considered to be the displacement of the camera's optical center. Therefore, the camera displacement is directly described in the world coordinate system, which reduces the transformation matrix to the conversion between the camera coordinate system and the drone coordinate system.

[0115] Among them, the final calculation formula of the simplified coordinate mapping process is as follows:

[0116]

[0117] In the above formula, only the UAV motion matrix T n_1is a variable, and the data in the drone's motion matrix can be obtained from the drone's flight control data. Therefore, after obtaining the drone's flight control data, the vehicle coordinates in the nth frame can be projected into the camera coordinate system at the time of the first frame, completing the subsequent road description modeling. By unifying the vehicle coordinates at any moment to the camera coordinate system of the first frame, parameters such as vehicle length and inter-vehicle distance can be accurately calculated based on real-world scale.

[0118] Furthermore, step S4 specifically includes:

[0119] S41. Calculating motion parameters of each vehicle based on the vehicle trajectory data and vehicle information, where the motion parameters include at least real-time speed and acceleration;

[0120] S42, extracting the road driving direction and spatial range of the lane space based on the vehicle trajectory data, vehicle information and motion parameters;

[0121] S43. Analyze and quantify inter-vehicle relationships based on vehicle position information in a road travel direction coordinate system, where the inter-vehicle relationships include at least relative distances and relative speeds between the vehicles.

[0122] S44. Construct a comprehensive road description model of the vehicle based on the motion parameters, the road driving direction, and the relationship between vehicles.

[0123] Specifically, a comprehensive road description model is established using vehicle position and trajectory data, which covers motion parameters, road driving directions and relationships between vehicles.

[0124] (1) Motion parameters: including dynamic information such as speed and acceleration.

[0125] By extracting each frame of video data, we can obtain the position information of all identified vehicles in the current image coordinate system at the time of capture. The vehicle position information of each frame constitutes the vehicle's spatiotemporal point (including frame and center point position information). The spatiotemporal point of the same vehicle is the discrete vehicle trajectory of the vehicle. The fields included in the vehicle trajectory data are shown in Table 1:

[0126] Table 1 Vehicle trajectory field table

[0127] Field Name Field meaning id Vehicle number frame Frame Index x Vehicle center point x-coordinate y Vehicle center point y-coordinate w Vehicle width h Vehicle length angle Vehicle heading angle (equivalent to counterclockwise rotation in the positive direction of the x-axis)

[0128] The instantaneous speed of all vehicles on the road at a specific moment can be calculated based on vehicle trajectory data. The instantaneous speed uses a time span of one-third of a second. The specific calculation formula is as follows:

[0129]

[0130] Where, is the vehicle speed at the nth frame; is the position of the vehicle at the nth frame; fps is the frame rate, that is, the number of frames contained in 1 second of the video; is the position vector of the vehicle at the (n-Δn)th frame; Δn is the frame difference corresponding to the time span.

[0131] After obtaining the vehicle's trajectory and speed data, the vehicle speed is aggregated by direction, thereby dividing the vehicle into different lane spaces. Each lane space has an independent vehicle data set. Then, the road driving direction and spatial range of each lane space are extracted. The vehicle data includes the vehicle's trajectory, speed, size, and direction data. The road driving direction and spatial range of the lane space are extracted from the above vehicle data. Figure 7 As shown in the figure, circles of different colors represent different lane spaces.

[0132] (2) Driving direction clustering: Through cluster analysis, the vehicle driving trend is revealed. The process of extracting the road driving direction in the lane space includes: determining the spatial distribution pattern of vehicle movement based on vehicle trajectory data, identifying the spatial boundary range of the lane in combination with vehicle information, calculating the speed vector of each vehicle and normalizing it, using a clustering algorithm to perform cluster analysis on the road driving direction vector, identifying the dominant driving flow direction in the lane space, eliminating abnormal driving direction vectors through statistical methods, and determining the road driving direction in the lane space.

[0133] Specifically, in this embodiment, the DBSCAN clustering algorithm is selected to perform cluster analysis on road direction vectors. DBSCAN is a density-based clustering algorithm. If a point has a sufficient number of points in its neighborhood, it is considered a core point, and points with similar density are clustered together. If a point does not have a sufficient number of points in its neighborhood, it is considered noise.

[0134] The input for clustering is normalized velocity vector data. The velocity vector describes the direction and speed of a vehicle. Normalizing the velocity vector eliminates velocity magnitude from affecting the clustering results, focusing only on the direction. Normalized velocity vectors are equivalent to points on a unit circle, and the distances between these points can be used to cluster road travel directions.

[0135] The comprehensive formula for clustering is as follows:

[0136]

[0137] Where v represents the velocity vector; d represents the velocity direction; |v| represents the modulus of the velocity vector; v x represents the component of the velocity vector in the x direction; v y Represents the component of the velocity vector in the y direction; dist(p,q) represents the distance between any velocity direction p and velocity direction q; represents the component of velocity direction p in the x direction; represents the component of velocity direction q in the x direction; represents the component of velocity direction p in the y direction; represents the component of velocity direction q in the y direction; N c (p) represents the number of speed directions in the neighborhood of speed direction p. If the distance between p and q is within the threshold range, p and q are considered to be in their respective areas. MinPts represents the threshold of the number of speed directions in the area. If it exceeds the threshold, it is considered that an aggregation result is found.

[0138] Most of the time, a vehicle's direction of travel is roughly aligned with or deviates from the road's direction by a small angle. Only during situations such as overtaking or lane changing does a vehicle's direction of travel differ significantly from the road's direction. Therefore, statistical methods are used to eliminate these few instances of vehicle directions that significantly deviate from the road's direction. The remaining vehicle directions can then be used to calculate the road's direction of travel. Data that differs by two standard deviations from the mean is considered outliers, so the mean and standard deviation of the speed direction are calculated.

[0139] First, the velocity vectors of all vehicles are normalized. The result is called the vehicle travel vector, so that the vector is only used to describe the vehicle's travel direction:

[0140]

[0141] Where, represents the vehicle velocity vector; Represents the vehicle's travel vector.

[0142] The vehicle driving vectors are summed and then normalized. The result is called the average driving vector, which is used to quantify the average driving direction of the vehicle.

[0143]

[0144] Where, represents the average driving vector, and N represents the number of vehicles.

[0145] Calculate the angle between the vehicle's driving vector and the average driving vector for each vehicle. The angle represents the deviation between the vehicle's driving direction and the average driving direction. The standard deviation of the vehicle's driving direction is calculated from the deviation between the vehicle's driving direction and the average driving direction:

[0146]

[0147] Where σ is the standard deviation.

[0148] After obtaining the mean and standard deviation of the speed direction, the speed directions that are more than twice the standard deviation away from the mean are eliminated, and then the remaining speed directions are averaged to obtain the road driving direction in the current lane space.

[0149] After extracting the road driving direction of the current lane space, the spatial range is a rectangle with the bottom edge parallel to the road driving direction of the current lane space. The rectangle is described by four variables x1, x2, y1, and y2. The lane space is as follows: Figure 8 As shown in the figure, the orange area represents the lane space. x1, x2, y1, and y2 are the maximum and minimum values in the x and y directions of the vehicle boundary points within the lane space, respectively. It should be noted that a lane space is a road with no noticeable deformation and the collection of vehicles within it. A road is composed of several lane spaces. If a section of road does not fall within any lane space, it means that there are no vehicles on that section of road. Therefore, no traffic events based on the direction of travel or the relationship between vehicles will occur.

[0150] (3) Relationships between vehicles: Analyze the relative positions and interactions between vehicles. Based on vehicle identification and tracking, the relationships between vehicles are extracted and quantified, specifically including the matching of the front, rear, left front, left rear, right front, and right rear vehicles. This allows for quantification of the relative position and speed of the vehicle and its surrounding vehicles. Extracting the positional relationships between vehicles facilitates efficient description and quantification of whether the driving behavior between vehicles is compliant, thereby identifying traffic incidents involving illegal driving.

[0151] The relationship between vehicles is established based on the road travel direction coordinate system. The road travel direction coordinate system is as follows: Figure 9 As shown in the figure, the positive direction of the x-axis of this coordinate system is the direction of the road, and the positive direction of the y-axis is the direction after rotating 90° counterclockwise from the positive direction of the x-axis. The distance in the x-axis direction is referred to as the longitudinal distance below, and the distance in the y-axis direction is referred to as the lateral distance below. Based on this, the definition of the positional relationship between vehicles is given as follows:

[0152] a. The preceding vehicle of a vehicle is the vehicle with the smallest longitudinal distance to the preceding vehicle among the vehicles whose lateral distance to the preceding vehicle is less than or equal to the lateral distance threshold and whose vehicle center x-coordinate is greater than the preceding vehicle's x-coordinate. If the smallest longitudinal distance is greater than the longitudinal distance threshold, the preceding vehicle is considered to have no preceding vehicle. The same applies to the following vehicles.

[0153] b. The left front vehicle of a vehicle is the vehicle with the smallest longitudinal distance to the vehicle in the lane space where the lateral distance to the vehicle is greater than the lateral distance threshold and the vehicle center x-coordinate is greater than the vehicle's x-coordinate. If the smallest longitudinal distance is greater than the longitudinal distance threshold, the vehicle is considered to have no left front vehicle. The definitions of the left rear vehicle, right front vehicle, and right rear vehicle are similar.

[0154] The meaning of the longitudinal distance threshold is that when the distance between a vehicle and the vehicle in front of it exceeds this value, the position of the vehicle in front of it no longer affects the driving behavior of the vehicle, and it is equivalent to considering that there is no vehicle in front of the vehicle. In addition, the lateral distance threshold is used to distinguish the front vehicle, the left front vehicle, the left rear vehicle, the rear vehicle, the left rear vehicle, and the right rear vehicle. When the absolute value of the lateral distance between the front and rear vehicles is less than the lateral distance threshold, the two vehicles are considered to be the front and rear vehicles, otherwise they are the left front vehicle / left rear vehicle. Whether it is the left front vehicle / left rear vehicle or the right front vehicle / right rear vehicle depends on the size relationship of the y-coordinates of the two vehicles, that is, whether the center point of the front vehicle is on the left or right side of the center point of the rear vehicle. In this embodiment, the lateral distance threshold is taken as 0.75 times the width of the vehicle.

[0155] The relationship between vehicles is as follows Figure 10 As shown in the figure: s1 represents the longitudinal distance between the center point coordinates of cars A and B. s1 is less than the longitudinal distance threshold, so A and B form a front-and-rear relationship. Furthermore, car A is to the left of car B, making it the left rear vehicle of car B. s2 represents the longitudinal distance between car B and cars C and D. s3 represents the lateral distance between the center point coordinates of car B and car C. s4 represents the lateral distance between the center points of car B and car D. Because s2 is less than the longitudinal distance threshold, s3 is greater than the lateral distance threshold, and car C is to the left of car B, car C is the left front vehicle of car B. s4 is less than the lateral distance threshold, so car D is the front vehicle of car B.

[0156] Further, if Figure 11 As shown, step S43 specifically includes the following steps:

[0157] S431, obtaining the trajectory data and vehicle information of all vehicles in the current frame, and initializing the current vehicle index to the first vehicle;

[0158] S432, taking the current vehicle as a benchmark, sequentially selecting other vehicles as target vehicles for relationship analysis;

[0159] S433, calculate the Euclidean distance between the current vehicle and the target vehicle, and determine whether the distance is less than a preset distance threshold. If not, select the next target vehicle and return to step S432; if so, proceed to the next step;

[0160] S434: Calculate the relative angle between the vehicles based on the current vehicle orientation vector and the vector connecting the two vehicles, and determine the type of spatial relationship between the vehicles based on the relative angle range;

[0161] S435: Calculate and record the relative distance and relative speed between vehicles that determine the spatial relationship type, and store the relationship type between the vehicles in a vehicle relationship matrix;

[0162] S436: Determine whether all vehicles have been traversed. If not, update the counter and continue traversing. If completed, output the relationship between vehicles.

[0163] Figure 11 The flowchart for extracting vehicle relationships from a collection of lane space vehicle position data and lane travel directions is shown. As shown in the figure, if the distance between the vehicle being calculated for surrounding vehicle relationships and the preceding vehicle exceeds a lateral distance threshold, the two vehicles are deemed to no longer affect each other's driving behavior. Since the vehicle data is already sorted by distance along the x-axis, forward traversal is discontinued. Following vehicles no longer affect the driving behavior of the vehicle being calculated for surrounding vehicle relationships. At this point, the search for the preceding vehicle is discontinued, and the current vehicle is deemed to have no preceding vehicle. Except in this case, the search continues forward to find all preceding vehicles of all types.

[0164] In summary, first, the position coordinates of all vehicles are projected into the lane travel coordinate system, with the lane travel direction being the positive direction of the x-axis. Then, the vehicle data is sorted from smallest to largest according to the vehicle's x-axis coordinate, ensuring that the vehicles with the highest index in the vehicle data are also at the front of the x-axis. Then, the vehicle data is traversed. Due to the sorting, the vehicles after the traversed vehicle are the front vehicles of the traversed vehicle in the x-axis direction. It is only necessary to determine whether it is the front vehicle, the left front vehicle, or the right front vehicle by the y-axis distance from the traversed vehicle. The vehicles behind the currently traversed vehicle are traversed in a loop until the front vehicle, the left front vehicle, and the right front vehicle of the currently traversed vehicle are found. At the same time, when the front vehicle of the traversed vehicle is found, the traversed vehicle is marked as the rear vehicle of its front vehicle. In this way, after traversing all vehicles, the surrounding vehicle relationships of all vehicles can be updated.

[0165] Next, Figure 12 Take the above as an example to further illustrate the technical solution of the present invention. Figure 12The diagram shows an example of extracting relationships between vehicles. In the diagram, vehicles A, B, C, and D are sorted by the size of their positions in the x-axis direction. When updating the relationship between vehicle A and its surrounding vehicles, the index of the traversed vehicle is 0 for the currently processed vehicle. The traversal index for the vehicle immediately preceding the currently traversed vehicle, i.e., the vehicle following A, is initialized to 1. The distance between vehicles A and B in the y-axis direction indicates that vehicle B is the right front vehicle of vehicle A. Vehicle B is marked as the right front vehicle of vehicle A, and vehicle A is marked as the left rear vehicle of vehicle B. The traversal index then increments, and the value of the preceding traversed vehicle becomes 2, pointing to vehicle C. The distance between vehicles A and C in the y-axis direction indicates that vehicle C is the right front vehicle of vehicle A. However, at this point, a vehicle has already been edited as the right front vehicle of vehicle A, so vehicle A is only marked as the left rear vehicle of vehicle C. Similarly, continuing the traversal will determine that vehicles D and E are the preceding and left preceding vehicles of vehicle A, respectively. Vehicle A is then marked as the following and right following vehicles of vehicles D and E, respectively. This completes the update of vehicle A's surrounding vehicle relationships. The current vehicle is incremented to 2, and the calculation of vehicle B's surrounding vehicle relationships begins. Because the preceding vehicles have already calculated their respective preceding vehicles and marked themselves as the preceding vehicles' following vehicles before calculating vehicle B's surrounding relationships, all of vehicle B's following vehicle relationships have already been calculated. Furthermore, a vehicle can have multiple following vehicles. For example, in the figure below, vehicle D has two right following vehicles, B and C. However, due to the update order, vehicle B's left preceding vehicle is calculated first, marking vehicle D's right following vehicle. Then, vehicle C's left preceding vehicle is calculated as vehicle D, marking vehicle C's right following vehicle. Finally, vehicle D's right following vehicle is vehicle C. After calculating all preceding vehicles, the following vehicles are also calculated, thus determining the surrounding vehicle relationships for all vehicles.

[0166] Furthermore, step S5 specifically includes:

[0167] S51. Constructing a traffic event recognition module based on the comprehensive road description model, wherein the traffic event recognition module includes preset traffic event recognition rules;

[0168] S52: Input the aerial video to a traffic event recognition module, which analyzes and processes the input data based on traffic event recognition rules;

[0169] S53. When the analysis result meets the conditions set by the discrimination rule, determine that a corresponding traffic event occurs and output a recognition result; the type of the traffic event includes at least one of vehicle congestion, abnormal parking, illegal lane change and traffic accident.

[0170] The aforementioned drone aerial video processing process is encapsulated into a standalone traffic event recognition module, which exposes a simple interface for easy operation. It supports user-defined recognition rules for the following four types of traffic events: traffic congestion, abnormal parking, illegal lane change, and traffic accidents.

[0171] That is, the present invention also provides a vehicle identification and traffic incident detection system based on the perspective of a drone, including a data acquisition module and a traffic identification module. The data acquisition module is responsible for obtaining drone aerial traffic video data and drone flight control data, providing a data basis for subsequent vehicle identification and traffic incident detection; the traffic identification module includes a vehicle identification unit, a vehicle trajectory extraction unit, a road description model unit and an event identification unit. These units work together to complete the entire process from raw video data to traffic event identification results.

[0172] The vehicle identification unit is used to extract vehicle information of each vehicle in the traffic video data. The unit includes an identification frame processing submodule and a target tracking submodule; the vehicle trajectory extraction unit includes a vehicle information storage submodule and a video processing submodule; the road description model unit is used to integrate vehicle trajectory data and vehicle information and construct a comprehensive road description model, including a road driving coordinate system vehicle submodule; the event recognition unit is used to construct a traffic event recognition module based on the comprehensive road description model, and recognize the input aerial video according to preset traffic event recognition rules, including an event processing submodule and a traffic event type submodule.

[0173] (1) Recognition frame processing submodule

[0174] The recognition frame processing submodule is used to store recognition frame information. When creating a recognition frame object, it passes in the rotation frame data in the length, width, height and angle format. It also provides internal conversion between different rotation frame formats. Specifically, it includes the following interfaces:

[0175] Get the object-oriented obb format data of the rotation box, and the return value is (x1, y1, x2, y2, x3, y3, x4, y4);

[0176] Get the upper left and lower right angle format data of the rotation box, and the return value is (x1, y1, x2, y2, angle);

[0177] Get the center length, width and angle format data of the rotation box. The return value is (x, y, w, h, angle).

[0178] (2) Traffic event type submodule

[0179] The traffic event type submodule is an enumeration class that contains the types of traffic events. Users can add traffic event types as needed.

[0180] (3) Vehicle information storage submodule

[0181] The vehicle information storage submodule is used to store basic vehicle information and historical vehicle trajectories. The basic information includes the vehicle's id, category id, length, and width. The vehicle category id is the type identified when the vehicle is first identified, and the length and width of the vehicle are the average of the length and width of all identified identification frames of the vehicle. Historical trajectory data is stored using a hash table trajectory frame dictionary, with the frame index as the key and the trajectory (x, y coordinates of the center point, vehicle orientation angle) as the value, thereby storing the position and orientation of the vehicle in each frame in history. The trajectory data stored in the vehicle information storage submodule is used to calculate speed, draw vehicle trajectory and other data on the original image, and extract traffic events based on the vehicle's own movement. The vehicle information storage submodule contains the following public interfaces:

[0182] Add the historical trajectory data of the vehicle. The input parameters are in the same format as the trajectory frame dictionary stored in the vehicle information storage submodule. The key of the trajectory dictionary is the frame index, and the value is the trajectory data (x, y coordinates of the center point, vehicle heading angle). In addition, when adding trajectory data, the average length and width of the vehicle in the incremental trajectory will be calculated. If the length and width of the current vehicle are not set, they will be set to the average length and width of the incremental trajectory; otherwise, the length and width of the vehicle will be updated by weighted average of the number of frames in the current incremental trajectory.

[0183] Get the traffic events of the vehicle on the target frame. The return value is a list of traffic events based on the vehicle's own motion on the target frame;

[0184] Get the vehicle's outer frame on the specified frame. Note: The length and width of the vehicle in the return value is not the length and width of the vehicle recognized in the target frame, but the average length and width of all the recognized vehicle frames;

[0185] Get the vehicle's velocity vector at the target frame. The input time span is the time span for calculating the velocity, expressed in seconds. For example, a time span of 0.5 indicates that when calculating the vehicle's velocity vector at the target frame, the displacement difference between the target frame and 0.5 seconds before the target frame / 0.5 seconds is used as the vehicle's velocity at the target frame. The position 0.5 seconds before the target frame requires converting seconds to frames, using the frame time ratio field. The frame time ratio stores the time span equivalent to one frame. Therefore, the input time span divided by the frame time ratio, rounded down, is the starting frame for velocity calculation.

[0186] Determines whether the vehicle appears on the target frame. If the vehicle trajectory contains the target frame, it returns true, otherwise it returns false.

[0187] This method is a class method and does not rely on instance data from the Vehicle Information Storage submodule. It is used to calculate the modulus of the velocity vector, that is, the magnitude of the velocity. The input velocity vector is a velocity vector. This method is a class method and does not rely on instance data from the Vehicle Information Storage submodule. It is used to calculate the angle of the velocity vector. The input velocity list is a list of velocity vectors, and the return value is a list of velocity vector angles.

[0188] (4) Road driving coordinate system vehicle submodule

[0189] The road driving coordinate system vehicle submodule stores the vehicle's information in the road driving direction coordinate system: vehicle center point coordinates (position), orientation angle (angle), speed (speed), and also includes the vehicle's surrounding relationships: the vehicle's front vehicle (pre_car), left front vehicle (left_pre_car), right front vehicle (right_pre_car), rear vehicle (behind_car), left rear vehicle (left_behind_car), and right rear vehicle (right_behind_car).

[0190] The function of the road driving coordinate system vehicle submodule is to extract traffic events based on the relationship between the vehicle and its surroundings. The road driving coordinate system vehicle submodule contains the following public interfaces:

[0191] Gets the distance and relative speed between the vehicle and the target vehicle. The input parameter is the target vehicle, and the return value is the distance and relative speed between the vehicle and the target vehicle. Calling this interface can easily calculate the distance between the vehicle and its surrounding vehicles.

[0192] Gets a list of traffic events between vehicles around the vehicle. The return value is a list of traffic event instance objects.

[0193] (5) Target tracking submodule

[0194] The target tracking submodule encapsulates YOLO and DeepSORT, providing recognition and tracking functions. The target tracking submodule encapsulates the functions of image recognition and continuous image tracking. It contains the YOLO instance object yolo, the DeepSORT instance object tracker, the category name list class_names, and the confidence threshold conf_threshold for recognition; in addition, there is a specified category ID for recognition: when the specified category ID has a value, only the category specified by the specified category ID will be recognized, that is, objects of other recognized types will be filtered out. When the specified category ID is not assigned a value, objects of all categories will be recognized; category counter class_counters, the mapping of tracker ID to vehicle ID, and the tracking to object mapping are used to calculate the independent ID for each recognized object.

[0195] The target tracking submodule contains the following public interfaces:

[0196] This interface generates a unique ID for an object, the object ID, and contains classification information. The object ID indicates the object's rank in the class to which the current object belongs. In other words, the object is uniquely identified by the class minus the object ID. The tracking ID is the tracking ID, which can also uniquely identify the object but does not distinguish between classes. The specific implementation of this method is:

[0197] The first step is to determine whether the category ID is in the category counter. If not, the category ID is added to the category counter with a value of 0. The second step is to calculate the independent ID of each identified object. The ID calculation rule is: category ID × 1000 + the value corresponding to the category ID in the category counter. The value corresponding to the category ID in the category counter is incremented by one. The third step is to return the object ID.

[0198] Identify and track the input image frame, and the return value is a list of recognition frame instance objects. Each recognition frame instance object represents the recognition frame of a recognized object. It should be noted that the call to the prediction and tracking frame method is stateful, that is, when the prediction and tracking frame method is called, the object of the current frame will be tracked based on the tracking result of the previous frame. If the object of the current frame overlaps with the object of the previous frame, it is considered to be the same object, otherwise it is considered to be a new object. Therefore, the call to the prediction and tracking frame method needs to ensure the continuity of the frame and cannot skip frames. The internal implementation of this interface is:

[0199] The first step is to call the detection method of its own detection network to identify the input image frame and obtain the recognition result; the second step is to construct the tracking input from the recognition output: since the recognition box of the input deepsort is not the original recognition box in the case of obb, a mapping between the recognition box of the input deepsort and the original recognition box is established, the key is the recognition box sequence number of the input deepsort, and the value is the original recognition box; the third step is to call the update method of its own tracker to track the input recognition box and obtain the tracking result; the fourth step is to obtain the original recognition box from the tracking box, and call its own tracking id to object id in turn to obtain the object id for each successfully tracked object - the id that uniquely represents the object and the category id are encapsulated and returned together.

[0200] (6) Video processing submodule

[0201] The video processing submodule provides a video processing interface and internally encapsulates the target tracking submodule instance object. The target tracking submodule is used for vehicle recognition in images and vehicle tracking in continuous images. When creating a video processing submodule object, the construction function requires the path to the YOLO model's weight file, the recognition confidence threshold, and the specified category ID for identifying only specific categories (for details, see the meaning of the specified category ID field in the target tracking submodule). In addition, there is a video capture instance object capture for video data processing functions such as reading videos. It also stores basic video information: video frame width frame_width, video frame height frame_height, video frame rate fps, and total video frame count frame_count.

[0202] The video processing submodule provides the following public interfaces:

[0203] Gets the image frame of a specified frame in a video. The return value is a NumPy array representing the image frame of the specified frame in the video. The input parameters are the bounding box, the drone motion matrix motion_matrix, and the camera intrinsic parameter matrix camera_matrix. This method projects the coordinates of the bounding box into the coordinate system described by the drone motion matrix and the camera intrinsic parameter matrix. The return value is a bounding box instance object representing the projected bounding box.

[0204] The input parameters are the frame index range for extracting the trajectory and the drone motion matrix and camera intrinsic parameter matrix within this range. The function of this method is to extract the trajectory data within the frame index range from the start frame to the end frame from the video, and call its own coordinate projection (identification box, motion matrix, camera matrix) for coordinate projection on the recognition result. The return value is a hash table with the frame index as the key and the vehicle recognition box list as the value. It should be noted that: in order to ensure the continuity of the trajectory, the frame index range of the input parameters should also be continuous when calling this interface continuously. If the frame range of the trajectory obtained by calling this interface twice is discontinuous, the trajectory will also be discontinuous, which may cause the trajectory of the same vehicle to be recognized as the trajectory of two vehicles.

[0205] (7) Event processing submodule

[0206] The event processing submodule provides all interfaces that are ultimately exposed to the user, and internally encapsulates the process of video processing and processing traffic information extracted from the video into traffic events. When creating it, you need to provide a video processor instance object for extracting traffic events, a file for storing drone motion data and the camera's intrinsic parameter matrix. If the event is obtained online, no file input is passed in. The event processing submodule includes the road driving direction of the lane space (lane direction, hash table, frame index as key, direction vector as value, lazy calculation, cache after calculation), vehicle collection (vehicle, hash table, vehicle id as key, vehicle information storage submodule instance object as value), video processor, and a private field trajectory frame count, which is used to record the range of frames that have been extracted for the current trajectory.

[0207] The event processing submodule provides the following private interfaces:

[0208] Updates the vehicle trajectory data within the frame index range [start frame to end frame]. This method internally calls the video processor's GetTrajectory(start frame, end frame, motion matrix list, camera matrix) method to obtain the vehicle recognition box list to extract the vehicle trajectory and save the trajectory data to the current vehicle dictionary.

[0209] Updates the vehicle trajectory data for the incoming image frame. This method internally calls the prediction and tracking frame (image) method of the target tracking submodule of the video processor to obtain a list of vehicle recognition frames. It then calls the coordinate projection (recognition frame, motion matrix, camera matrix) of the video processor to project the coordinates to the specified frame, thereby extracting the vehicle trajectory and saving the trajectory data to the current vehicle dictionary.

[0210] Get the vehicle collection on the target frame, and the return value is a list of vehicle information storage submodule instance objects.

[0211] Calculate the road driving direction on the target frame, and the return value is the road driving direction vector.

[0212] The specific implementation of this method is:

[0213] The first step is to determine whether the road driving direction of the target frame exists in its own lane direction dictionary. If so, the road driving direction vector is returned directly; if not, continue to execute the following steps; the second step is to call its own Get Frame Vehicle (target frame) method to obtain the vehicle set on the target frame; the third step is to obtain the speed vectors of all vehicles and normalize the vehicle speed so that the speed does not affect the clustering result, thereby satisfying the clustering for speed direction; the fourth step is to use statistical methods to eliminate speed vectors that obviously deviate from the lane direction in the speed set of each group after clustering. Specifically, the speed vector is normalized and averaged to describe the average speed direction, the angle between each speed vector and the average speed direction is calculated, the standard deviation of the angle is calculated, and the speed vector with an angle greater than 2 times the standard deviation is eliminated; the fifth step is to use the direction of the vector sum of the speed vectors of each group of vehicles as the lane direction, store the lane direction vector in its own lane direction dictionary, and return the lane direction vector.

[0214] Get all microscopic traffic events in the lane space on the target frame, and return a hash table with the vehicle ID as the key and the traffic event list as the value. The specific implementation of this method is:

[0215] The first step is to call its own Get Frame Vehicle (target frame) method to obtain the vehicle set on the target frame; the second step is to traverse the vehicle set in all lane spaces of the current frame, and call the Get Traffic Event (target frame) method of the vehicle information storage submodule to obtain the traffic event list of the vehicle on the target frame; the third step is to traverse the vehicle set in all lane spaces of the current frame, extract the road driving direction of the target frame, project the vehicle set to the road driving coordinate system of the road target frame, sort the projected vehicle set from near to far according to the road driving direction distance, traverse the sorted vehicles, extract the vehicle relationship between each vehicle to obtain the road driving coordinate system vehicle list, traverse the road driving coordinate system vehicle list, and call the Get Traffic Event method of the road driving coordinate system vehicle to obtain the traffic event list based on the vehicles around the vehicle; the fourth step is to obtain the traffic event list based on the road driving rules; the fifth step is to merge the above three traffic event lists, that is, all traffic events in the lane space on the target frame, where the return value is a hash table with the key being the vehicle id and the value being the traffic event list, and one element representing all traffic events triggered by one vehicle.

[0216] Get all macro traffic events in the lane space on the target frame. The return value is a traffic event list.

[0217] In addition to the above private methods, the event handling submodule also provides the following public interfaces:

[0218] Get all traffic events in the video within the frame index range of [start frame to end frame]. If the start frame is empty, extract the traffic events of the entire video. The return value is a hash table. The hash table contains two keys: micro events and macro events, corresponding to micro traffic events and macro traffic events respectively. The value corresponding to the micro event is also a hash table that stores all micro traffic events triggered by the vehicle and records the frame index range triggered by the traffic event. Specifically, the vehicle id is used as the key, and the hash table consisting of the traffic event instance object and the frame index list where the traffic event occurs is used as the value. The value corresponding to the macro event is also a hash table that stores all macro traffic events, where the key is the traffic event instance object and the value is the frame list where the traffic event occurs.

[0219] The specific implementation of this method is as follows: the first step is to read the drone operation matrix within the frame index range of [start frame to end frame] from its own drone file to form a motion matrix list, and read the camera intrinsic parameter matrix camera matrix; the second step is to call its own update trajectory (start frame, end frame) method to update the vehicle trajectory data within the frame index range of [start frame to end frame]; the third step is to identify a traffic event every one-third of a second from the start frame (after conversion, it is every second / 3 frames of its own frame) and integrate the traffic event into the return value of the identified traffic event until the end frame. The identification process is: a) call its own get frame micro traffic event (target frame) method to obtain the micro traffic event of the current frame, and summarize the traffic event triggered by the vehicle in the current frame into all traffic events triggered by the vehicle; b) call its own get frame macro traffic event (target frame) method to obtain the macro traffic event of the current frame, and summarize the macro traffic event triggered by the current frame into all macro traffic events;

[0220] Images are continuously transmitted over the network and added to a message queue. A thread consumes messages in the queue, which then calls its own updated online trajectory (image, motion matrix, camera matrix) to update the trajectory data before identifying traffic events. This process is essentially the same as offline traffic event acquisition. It's important to note that when calling this API, the continuously transmitted images must be consecutive frames from the video; otherwise, the vehicle trajectory will be discontinuous, and traffic events will not be identified.

[0221] The traffic event data of the lane space is saved into a comma-separated value file, and the vehicle trajectory data in the comma-separated value file is extracted.

[0222] Customize traffic event recognition rules

[0223] (1) Define the recognition rules for macro traffic events: Users need to define the rules for events of interest in the macro traffic event (target frame) method on the acquisition frame of the event processing submodule class. All historical data of all vehicles are stored in the event processing submodule class. Users can define the recognition rules for triggering macro traffic events based on these macro traffic data.

[0224] (2) Define the identification rules of micro-traffic events based on the vehicle itself: The user needs to define the event in the method of obtaining traffic events (target frames) in the vehicle information storage submodule class. The vehicle information storage submodule class stores data related to the vehicle's own movement, vehicle size, etc. The user can define whether the vehicle triggers a traffic event based on the vehicle's own movement based on this data.

[0225] (3) Define the identification rules of micro-traffic events based on vehicle and road driving rules and the relationship between vehicles and their surroundings

[0226] Users need to define event rules in the Get Traffic Event (Target Frame) method of the Road Driving Coordinate System Vehicle class. The Road Driving Coordinate System Vehicle stores vehicle data projected onto the corresponding frame, as well as the relative relationships between vehicles. Based on this data, users can define traffic event recognition rules that trigger based on vehicle and road driving rules and the relationship between vehicles and their surroundings.

[0227] After the user completes the definition of the traffic event recognition rules of their interest, they only need to create an event processing sub-module instance object and call the event processing sub-module's offline acquisition of traffic events (start frame, end frame) or the event processing sub-module's online acquisition of traffic events (picture, motion matrix, camera matrix) to realize offline or online recognition of traffic events.

[0228] Specific traffic event recognition implementation

[0229] (1) Emergency stop: It is defined as an abnormal traffic behavior in which a vehicle suddenly takes extreme braking measures under normal driving conditions, causing the vehicle speed to drop to zero in a very short time and stay in the lane. The danger of emergency stop is that it instantly breaks the continuity of traffic flow, causing the rear vehicles to cause a series of rear-end collisions due to insufficient reaction time. At the same time, the vehicle may make an emergency stop because there is an object in front that should not be on the road and needs to be avoided urgently. The traffic event recognition module proposed by the present invention is used to complete the recognition of emergency stop events. It is only necessary to define the event in the method of obtaining traffic events (target frames) in the vehicle information storage submodule class. The judgment rule is: if the current frame speed of the vehicle is 0, and the total deceleration of the vehicle within 0.5 seconds of the current time exceeds a certain threshold, it is considered that the vehicle has made an emergency stop. In this process, it is necessary to first convert the time span in seconds into the time span in frames. The vehicle information storage submodule class stores the conversion ratio of each frame and time width in the frame time ratio field. It is only necessary to divide the time span in seconds by the frame time ratio to obtain the time span in frames. Figure 13 The figure shows the recognition effect of emergency parking in the video. The yellow part in the figure is the effect after the emergency parking vehicle is marked.

[0230] (2) Going against traffic or reversing: The direction of the vehicle is opposite to the direction of travel specified by the road. The danger of going against traffic or reversing is that it violates the road driving rules and is extremely sudden and unpredictable for other vehicles. It is difficult for normally traveling vehicles to react in time to going against traffic or reversing, which can easily lead to serious collision accidents. Moreover, due to the high relative speed, the consequences of the collision are often more serious and may result in the destruction of the vehicle and the death of people. The traffic event recognition module proposed by the present invention is used to complete the recognition of the going against traffic or reversing events. It is only necessary to define the event in the method of obtaining traffic events (target frames) of the road driving coordinate system vehicle class. The road driving coordinate system vehicle stores the coordinates, speed and other data of the vehicle projected to the current frame road driving coordinate system. Therefore, the vehicle's velocity vector is already in the road driving direction coordinate system. If the vehicle's velocity angle exceeds 90°, it is considered that the vehicle's driving direction is no longer consistent with the road driving direction, and therefore it is considered that a going against traffic or reversing event has occurred. Figure 14 The image shows the marking effect of wrong-way / reversing vehicles. After obtaining the time frame and vehicle data of the wrong-way vehicle, the vehicle can be marked in the video and the continuous frames can be spliced into one picture.

[0231] (3) Congestion: The average headway between vehicles traveling in the same direction on the road is significantly reduced to below a certain critical value, resulting in a traffic state in which the traffic flow loses its free-flow characteristics and vehicles are forced to frequently slow down or stagnate. The traffic event recognition module proposed in the present invention is used to complete the identification of congestion events. It is only necessary to define the event in the macro traffic event (target frame) method on the acquisition frame of the event processing submodule class. The specific judgment method is that when 60% of the vehicles in a road space have a headway of less than 5m, congestion is considered to have occurred. Figure 15 The congestion identification effect is shown, that is, the congestion in the video is identified and the congested lane space is framed.

[0232] Event recognition effectiveness evaluation

[0233] In order to verify the effectiveness of the traffic event recognition system proposed in this invention, a performance evaluation test was conducted on the system. Since a traffic event consists of a set of continuous frames, the recognition effect of a traffic event must evaluate not only whether the category is accurate, but also whether the frame range of the recognized traffic event is accurate. Considering that the number of traffic events is small but the impact is large, the loss caused by missed recognition of traffic events is greater than the loss caused by incorrect recognition. Therefore, this paper selects the category recall rate and the frame range recall rate to quantify the effect of the frame range of the recognized traffic event. The calculation formula of the frame range recall rate is as follows:

[0234]

[0235] R Fl Indicates the frame range recall rate, Fl pred Indicates the event frame range of recognition, Fl true represents the frame range of the real event, and |Fl| represents the number of frames used to calculate the frame range F1.

[0236] By testing multiple UAV aerial video sequences, the evaluation results are shown in Table 2.

[0237] Table 2 Event recognition evaluation results

[0238] Event Type Category recall Frame range recall Emergency Stop 93.00％ 63.34％ Reverse driving / reversing 100.00％ 93.40％ congestion 83.70％ 90.30％ comprehensive 92.23％ 82.34％

[0239] The results show an overall class recall rate of 92.23%, demonstrating that the proposed system can effectively identify various types of traffic events. For congestion events, the class recall rate was 83.70%. This relatively low rate is due to the fact that congestion is determined by calculating the percentage of inter-vehicle distances below a threshold. When a video contains many vehicles and is in the initial stages of congestion, congestion may not be identified. An improved approach is to optimize the threshold selection based on local traffic conditions to improve class recall.

[0240] The overall frame-wide recall rate was 82.34%. This was partially influenced by the accuracy of the vehicle recognition algorithm, as a very small number of frames with continuously moving vehicles were either not recognized or identified as another vehicle. The lowest frame-wide recall rate was 63.34% for emergency stops. This is because the emergency stop rule determines whether the total deceleration within a 0.5-second interval exceeds a threshold. Therefore, all frames within 0.5 seconds during continuous emergency braking where the deceleration did not exceed the threshold were excluded.

[0241] The recognition effect of wrong-way / reversing events is the best, with a category recall rate of 100% and a frame range recall rate of 93.40%, indicating that the vehicle speed angle judgment method based on the road driving coordinate system has high accuracy.

[0242] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A vehicle identification and traffic incident detection method based on drone perspective, characterized in that: The following steps are involved: S1. Obtain UAV aerial traffic video data and UAV flight control data; S2. Use target detection algorithm to identify the image in the video data and obtain vehicle information; S3. Based on the identified vehicle, a target tracking algorithm is used to track the vehicle in the video data, and the identified vehicle is mapped to a unified coordinate system in combination with the flight control data to generate vehicle trajectory data; S4. Build a comprehensive road description model based on the vehicle trajectory data and vehicle information, where the comprehensive road description model includes vehicle motion parameters, road driving directions, and relationships between vehicles; S5. A traffic event recognition module is formed based on the comprehensive road description model, and the input aerial video is recognized according to preset traffic event recognition rules to obtain a recognition result.

2. The method for vehicle identification and traffic incident detection based on drone perspective according to claim 1, characterized in that: Step S2 specifically includes: S21, extracting the first frame image of the video data, and preprocessing the first frame image; S22. Input the preprocessed image into a pre-trained target detection model, analyze the extracted features through the target detection model, and output the identified vehicle information, wherein the vehicle information includes the vehicle category and vehicle bounding box data, and the vehicle bounding box data includes the vehicle center point coordinates, vehicle width, vehicle height, and vehicle rotation angle.

3. The method for vehicle identification and traffic incident detection based on drone perspective as claimed in claim 2, characterized in that: The target detection model is an improved YOLOv11-OBB model. The network structure of the target detection model includes: The backbone network includes multiple convolutional layers and an SPD-Conv module. The SPD-Conv module consists of an SPD layer and a non-strided convolutional layer. The SPD layer converts the input feature map from space to depth according to the scaling factor, divides the feature map into multiple sub-maps and concatenates them along the channel dimension. The non-strided convolutional layer uses a convolution with a stride of 1 to adjust the channel dimension. The neck network adopts a path aggregation network structure to perform top-down and bottom-up bidirectional feature fusion on the multi-scale feature maps output by the backbone network; The detection head adopts a task-aligned dynamic detection head structure to perform vehicle identification and positioning on the multi-scale fusion feature map output by the neck network. It includes a task decomposition module, a classification branch, and a positioning branch. The positioning branch uses deformable convolution for feature enhancement. The classification branch and the positioning branch collaborate through task interaction features, and respectively set a scale adjustment layer to adjust the scale of the output.

4. The method for vehicle identification and traffic incident detection based on drone perspective as claimed in claim 2, characterized in that: Step S3 specifically includes: S31. Based on the identified vehicles, a multi-target tracking algorithm is used to perform data association and cross-frame tracking on vehicle targets in consecutive frames of the traffic video data to generate a motion trajectory of each vehicle in the image coordinate system; S32, establishing a coordinate mapping relationship between the image coordinate system and the camera coordinate system according to the UAV flight control data; S33. Convert the vehicle motion trajectory in the pixel coordinate system to the geographic reference coordinate system through a coordinate mapping relationship to generate vehicle trajectory data in a unified coordinate system.

5. The method for vehicle identification and traffic incident detection based on drone perspective as claimed in claim 4, characterized in that: Step S32 includes: According to the coordinate mapping relationship, the image coordinate system of the first frame is mapped to the camera coordinate system, and the camera coordinate system of the first frame is used as the unified coordinate system; Map the image coordinate system of the nth frame to the camera coordinate system based on the coordinate mapping relationship, and project the camera coordinate system of the nth frame to the unified coordinate system; The coordinate mapping process uses the following formula: Where, is the vehicle coordinate when shooting the first frame, is the vehicle coordinate when shooting the nth frame, K -1 is the inverse matrix of the camera intrinsic parameter matrix, T n_1 The transformation matrix of the camera coordinate system when shooting the nth frame is converted to the camera coordinate system when shooting the first frame, R n_1 is the rotation matrix, t n_1 is the translation vector; K n is the camera intrinsic parameter matrix; f x is the focal length in the x-axis direction of the image; f y is the focal length in the y-axis direction of the image; c x is the x-coordinate of the image principal point (projection of the optical center on the image plane); c y is the y coordinate of the image principal point; s is the tilt factor of the pixel coordinate axis.

6. The method for vehicle identification and traffic incident detection based on drone perspective as claimed in claim 2, characterized in that: Step S4 specifically includes: S41. Calculating motion parameters of each vehicle based on the vehicle trajectory data and vehicle information, where the motion parameters include at least real-time speed and acceleration; S42, extracting the road driving direction and spatial range of the lane space based on the vehicle trajectory data, vehicle information and motion parameters; S43. Analyze and quantify inter-vehicle relationships based on vehicle position information in a road travel direction coordinate system, where the inter-vehicle relationships include at least relative distances and relative speeds between the vehicles. S44. Construct a comprehensive road description model of the vehicle based on the motion parameters, the road driving direction, and the relationship between vehicles.

7. The method for vehicle identification and traffic incident detection based on drone perspective according to claim 6, characterized in that: The process of extracting the road driving direction in the lane space in step S42 includes: determining the spatial distribution pattern of vehicle movement based on vehicle trajectory data, identifying the spatial boundary range of the lane in combination with vehicle information, calculating the velocity vector of each vehicle and normalizing it, performing cluster analysis on the road driving direction vectors using a clustering algorithm, identifying the dominant driving flow direction in the lane space, eliminating abnormal driving direction vectors through statistical methods, and determining the road driving direction in the lane space.

8. The method for vehicle identification and traffic incident detection based on drone perspective according to claim 6, characterized in that: Step S43 specifically includes: S431, obtaining the trajectory data and vehicle information of all vehicles in the current frame, and initializing the current vehicle index to the first vehicle; S432, taking the current vehicle as a benchmark, sequentially selecting other vehicles as target vehicles for relationship analysis; S433, calculate the Euclidean distance between the current vehicle and the target vehicle, and determine whether the distance is less than a preset distance threshold. If not, select the next target vehicle and return to step S432; if so, proceed to the next step; S434: Calculate the relative angle between the vehicles based on the current vehicle orientation vector and the vector connecting the two vehicles, and determine the type of spatial relationship between the vehicles based on the relative angle range; S435: Calculate and record the relative distance and relative speed between vehicles that determine the spatial relationship type, and store the relationship type between the vehicles in a vehicle relationship matrix; S436: Determine whether all vehicles have been traversed. If not, update the counter and continue traversing. If completed, output the relationship between vehicles.

9. The method for vehicle identification and traffic incident detection based on drone perspective as claimed in claim 1, characterized in that: Step S5 specifically includes: S51. Constructing a traffic event recognition module based on the comprehensive road description model, wherein the traffic event recognition module includes preset traffic event recognition rules; S52: Input the aerial video to a traffic event recognition module, which analyzes and processes the input data based on traffic event recognition rules; S53. When the analysis result meets the conditions set by the discrimination rule, determine that a corresponding traffic event has occurred and output a recognition result; the type of the traffic event includes at least one of vehicle congestion, abnormal parking, illegal lane change, wrong-way driving, and potential traffic accident.

10. A vehicle identification and traffic incident detection system based on drone perspective, characterized in that: Data acquisition module, used to collect traffic video data and drone flight control data; Traffic identification module, including: a vehicle identification unit for extracting vehicle information of each vehicle in traffic video data; The vehicle trajectory extraction unit is used to associate and track the identified vehicles, and map the vehicle positions to a unified coordinate system in combination with the UAV flight control data, thereby generating the target vehicle's trajectory data; A road description model unit, used to integrate vehicle trajectory data and vehicle information and build a comprehensive road description model; The event recognition unit is used to build a traffic event recognition module based on the comprehensive road description model, recognize the input aerial video according to the preset traffic event recognition rules, and obtain the recognition result.

Images