Traffic violation detection method and device based on UV-KGNN network, equipment and medium

Abstract

The application discloses a traffic violation detection method and device based on a UV-KGNN network, equipment and a medium, relates to the field of intelligent transportation, and comprises the following steps: collecting multi-modal traffic-related data based on a plurality of sensors carried by a UAV, and determining target physical quantities related to the operation state of the UAV and vehicles; performing target detection and feature extraction based on a preset initial network and the target physical quantities, constructing a UV-KGNN network by using corresponding vehicle detection frames and deep visual features, and predicting the uncertain trajectory of the vehicle by using the UV-KGNN network; classifying and determining potential traffic violation behaviors based on the trajectory prediction result and a preset classifier; and analyzing and processing a strategy based on the size of the target confidence corresponding to each target potential traffic violation behavior in the behavior determination result, so as to determine a target traffic violation detection result. The application realizes high-precision, physically consistent, uncertainty-quantized and environment-adaptive traffic violation detection.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a traffic violation detection method, device, equipment and medium based on a UV-KGNN network. Background Art

[0002] Existing drone traffic violation detection solutions face many challenges in complex traffic environments: (1) Perception limitations in complex dynamic environments lead to insufficient robustness and accuracy in target detection and tracking; (2) Motion trajectory prediction relies heavily on pure data-driven models, lacking effective modeling of the vehicle's real dynamic characteristics and traffic physics, resulting in the predicted trajectory possibly not being consistent with the actual physical laws, and lacking quantitative assessment of the uncertainty of the prediction results; (3) Ignoring multi-target interactions and the coupled motion of drones and vehicles, resulting in deviations in relative motion estimation, which in turn leads to detection errors. These defects together lead to a lag in the determination of violations, and high false alarm and missed alarm rates, making it difficult to meet the needs of intelligent law enforcement.

[0003] Therefore, how to achieve drone traffic violation detection with high precision, physical consistency, uncertainty quantification and strong environmental adaptability is an urgent problem that needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a traffic violation detection method, device, equipment, and medium based on a UV-KGNN network, which can effectively achieve high-precision, physically consistent, uncertainty quantified, and environmentally adaptable drone traffic violation detection, thereby reducing the false alarm rate and missed alarm rate, improving the accuracy, authenticity, foresight, and reliability of the detection results, and thus laying a solid foundation for achieving a closed loop of fully automated intelligent law enforcement. The specific solution is as follows:

[0005] In a first aspect, the present application provides a traffic violation detection method based on a UV-KGNN network, comprising:

[0006] Collecting multimodal traffic-related data through multiple sensors carried by the drone, and determining multiple target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data;

[0007] Target detection and feature extraction are performed based on the preset initial network and the target physical quantity. A UV-KGNN network is constructed using the corresponding vehicle detection frame and deep visual features. The UV-KGNN network is then used to predict the vehicle's uncertain trajectory to obtain a trajectory prediction result. The UV-KGNN network is a kinematic graph neural network that couples the UAV and vehicle.

[0008] classify and determine a current or future potential traffic violation behavior based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier, to determine a behavior determination result;

[0009] analyze a processing strategy corresponding to each target potential traffic violation behavior based on a size of a target confidence corresponding to each target potential traffic violation behavior in the behavior determination result, to determine a target traffic violation detection result.

[0010] Optionally, the plurality of sensors carried by the unmanned aerial vehicle collect multi-modal traffic-related data, and a plurality of target physical quantities related to the operating state of the unmanned aerial vehicle and the vehicle are determined based on the corresponding multi-modal traffic-related data, including:

[0011] The plurality of sensors carried by the unmanned aerial vehicle collect multi-modal traffic-related data; the multi-modal includes image modal, video modal, and point cloud modal;

[0012] The multi-modal traffic-related data is preprocessed to determine a preprocessing result;

[0013] A first preset state estimation algorithm and the preprocessing result are used to estimate the position, speed, and attitude of the vehicle in a world coordinate system, to determine a plurality of first target physical quantities related to the operating state of the vehicle;

[0014] A second preset state estimation algorithm and the preprocessing result are used to estimate the position, speed, and attitude of the unmanned aerial vehicle in a world coordinate system, in combination with the unmanned aerial vehicle position information provided by the preset positioning system and the unmanned aerial vehicle acceleration information and unmanned aerial vehicle angular velocity information provided by the inertial measurement component, to determine a plurality of second target physical quantities related to the operating state of the unmanned aerial vehicle.

[0015] Optionally, the target detection and feature extraction based on the preset initial network and the target physical quantities include:

[0016] Multi-scale deformable small target detection is performed based on a preset initial network and the target physical quantities, to determine a vehicle detection frame; the preset initial network is a network constructed based on a YOLOv8 network, in combination with a preset deformable convolution network and a dynamic feature fusion pyramid network;

[0017] Deep features of different scales and deformations are captured based on the preset deformable convolution network in the preset initial network and the target physical quantities, and feature enhancement is performed using the dynamic feature fusion pyramid network in the preset initial network, to determine deep visual features.

[0018] Optionally, the method of constructing a UV-KGNN network using the corresponding vehicle detection frame and deep visual features, and using the UV-KGNN network to perform uncertain trajectory prediction of the vehicle to obtain a trajectory prediction result includes:

[0019] Target recognition is performed based on the corresponding vehicle detection frame and depth vision, and the identity of the same target in different frames is associated to complete the multi-target tracking operation and determine the multi-target tracking results;

[0020] Constructing a UV-KGNN network based on the multi-target tracking results and a preset vehicle dynamics model;

[0021] Determine the real-time kinematic parameters corresponding to each vehicle by using the UV-KGNN network in combination with a preset improved recurrent neural network, the target physical quantity, and the deep visual features;

[0022] Based on the target graph attention layer in the UV-KGNN network and the real-time kinematic parameters, the future trajectory of each vehicle is predicted, and uncertainty analysis of the predicted trajectory is performed to obtain a trajectory prediction result; the target graph attention layer is a graph attention layer used to fuse the physical related information corresponding to the drone and the vehicle into the graph node features.

[0023] Optionally, the classifying and determining current or future potential traffic violations based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier includes:

[0024] Based on a preset classifier, the trajectory prediction result, and the historical trajectory information corresponding to the vehicle output by the UV-KGNN network, classify and determine the vehicle's current or future potential traffic violations, and determine an initial confidence level for each potential traffic violation;

[0025] Calibrate the initial confidence of each of the potential traffic violation behaviors based on the confidence correction network in the preset classifier to determine a target confidence of each of the potential traffic violation behaviors;

[0026] The behavior determination result is determined based on the target confidence, the threshold information corresponding to each preset traffic violation behavior category and the trajectory prediction result.

[0027] Optionally, the analyzing, based on the target confidence level corresponding to each target potential traffic violation in the behavior determination result, the processing strategies corresponding to each target potential traffic violation include:

[0028] Determining target behavior labels corresponding to each target potential traffic violation based on preset label generation rules and the target confidence corresponding to each target potential traffic violation in the behavior determination result; the target behavior labels include high-confidence violation labels and low-confidence violation labels;

[0029] Determine a handling strategy for each of the target potential traffic violations based on the corresponding target behavior label and the preset violation handling strategy;

[0030] A target traffic violation detection result is determined based on each of the target potential traffic violation behaviors and the corresponding processing strategy.

[0031] Optionally, the method further includes:

[0032] Based on the elastic weight solidification algorithm, the preset task perception selection mechanism and the preset online incremental learning algorithm, the UV-KGNN network and the preset classifier are lightweight parameter updated in combination with the current traffic-related data, so as to complete the traffic violation detection operation using the updated UV-KGNN network and the updated preset classifier.

[0033] In a second aspect, the present application provides a traffic violation detection device based on a UV-KGNN network, comprising:

[0034] a physical quantity determination module, configured to collect multimodal traffic-related data through a plurality of sensors carried by the drone, and determine a plurality of target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data;

[0035] A trajectory prediction module is used to perform target detection and feature extraction based on a preset initial network and the target physical quantity, construct a UV-KGNN network using the corresponding vehicle detection box and deep visual features, and use the UV-KGNN network to predict the vehicle's uncertain trajectory to obtain a trajectory prediction result; the UV-KGNN network is a kinematic graph neural network that couples the UAV and vehicle;

[0036] a violation behavior determination module, configured to classify and determine current or future potential traffic violations based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier to determine a behavior determination result;

[0037] The detection result determination module is used to analyze the processing strategies corresponding to each target potential traffic violation behavior based on the target confidence level corresponding to each target potential traffic violation behavior in the behavior judgment result to determine the target traffic violation detection result.

[0038] In a third aspect, the present application provides an electronic device, comprising:

[0039] Memory, used to store computer programs;

[0040] A processor is used to execute the computer program to implement the steps of the aforementioned traffic violation detection method based on the UV-KGNN network.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of the aforementioned traffic violation detection method based on the UV-KGNN network.

[0042] It can be seen that in this application, multimodal traffic-related data are collected by multiple sensors carried by the drone, and based on the corresponding multimodal traffic-related data, multiple target physical quantities related to the operating status of the drone and the vehicle are determined; target detection and feature extraction are performed based on the preset initial network and the target physical quantities, and a UV-KGNN network is constructed using the corresponding vehicle detection frame and deep visual features, and the UV-KGNN network is used to predict the uncertainty trajectory of the vehicle to obtain a trajectory prediction result; the UV-KGNN network is a kinematic graph neural network coupled with the drone and the vehicle; based on the trajectory prediction result, the preset classifier and the confidence correction network in the preset classifier, the current or future potential traffic violations are classified and judged to determine the behavior judgment result; based on the size of the target confidence corresponding to each target potential traffic violation in the behavior judgment result, the processing strategy corresponding to each target potential traffic violation is analyzed to determine the target traffic violation detection result. That is to say, in this application, the multimodal traffic-related data collected by the drone is first processed to determine multiple target physical quantities related to the operating status of the drone and the vehicle. After that, target detection and feature extraction are performed through the preset initial network and target physical quantities to construct a UV-KGNN network using the corresponding vehicle detection frame and deep visual features, and the UV-KGNN network is used to determine the trajectory prediction result. Then, based on the trajectory prediction result, the preset classifier and the confidence correction network therein, the current or future potential traffic violations are classified and determined. The corresponding processing strategy is determined based on the target confidence corresponding to each target potential traffic violation in the behavior judgment result. In this way, it is possible to effectively achieve drone traffic violation detection with high precision, physical consistency, uncertainty quantification and strong environmental adaptability, thereby reducing the false alarm rate and missed alarm rate, improving the accuracy, authenticity, foresight and reliability of the detection results, and thus laying a solid foundation for achieving a closed loop of fully automated intelligent law enforcement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flow chart of a traffic violation detection method based on UV-KGNN network provided in this application;

[0045] Figure 2 A schematic diagram of the structure of a traffic violation detection device based on a UV-KGNN network provided in this application;

[0046] Figure 3 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] Existing drone traffic violation detection solutions face many challenges in complex traffic environments: (1) Perception limitations in complex dynamic environments lead to insufficient robustness and accuracy in target detection and tracking; (2) Motion trajectory prediction relies heavily on pure data-driven models, lacking effective modeling of the vehicle's real dynamic characteristics and traffic physics, resulting in the predicted trajectory possibly not being consistent with the actual physical laws, and lacking quantitative assessment of the uncertainty of the prediction results; (3) Ignoring multi-target interactions and the coupled motion of drones and vehicles, resulting in deviations in relative motion estimation, which in turn leads to detection errors. These defects together lead to a lag in the determination of violations, and high false alarm and missed alarm rates, making it difficult to meet the needs of intelligent law enforcement.

[0049] To this end, this application provides a traffic violation detection solution based on the UV-KGNN network, which can effectively achieve high-precision, physical consistency, uncertainty quantification and strong environmental adaptability of drone traffic violation detection, thereby reducing the false alarm rate and missed alarm rate, and improving the accuracy, authenticity, foresight and reliability of the detection results, thereby laying a solid foundation for realizing the closed loop of fully automated intelligent law enforcement.

[0050] See also Figure 1As shown, the embodiment of the present invention discloses a traffic violation detection method based on a UV-KGNN network, comprising:

[0051] Step S11: Collect multimodal traffic-related data through multiple sensors carried by the drone, and determine multiple target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data.

[0052] In this embodiment, multiple sensors onboard the drone, such as visible light cameras, infrared sensors, and lidar sensors, first collect multimodal traffic data in real time, including images, videos, and point clouds. The collected data is preprocessed to accurately estimate the drone's motion state and key physical quantities such as the vehicle's real-time speed and attitude. Specifically, the drone's multiple sensors collect multimodal traffic-related data, including image, video, and point cloud modalities. The multimodal traffic-related data is preprocessed to determine a preprocessing result. The vehicle's position, velocity, and attitude in a world coordinate system are estimated based on a first preset state estimation algorithm and the preprocessing result to determine multiple first target physical quantities related to the vehicle's operating state. Finally, the drone's position, velocity, and attitude in a world coordinate system are estimated based on a second preset state estimation algorithm and the preprocessing result, combined with the drone's position information provided by a preset positioning system and the drone's acceleration and angular velocity information provided by an inertial measurement unit to determine multiple second target physical quantities related to the drone's operating state. This ensures that subsequent trajectory predictions conform to actual physical laws.

[0053] Furthermore, regarding the second target physical quantity related to the UAV's own motion state, in this embodiment, an extended Kalman filter (EKF) is used to fuse high-precision position information provided by the RTK-GPS (Real-Time Kinematic Global Positioning System) with acceleration and angular velocity information provided by the IMU (Inertial Measurement Unit) to estimate the UAV's position, velocity, and attitude in the world coordinate system in real time. The EKF's prediction and update steps combine the UAV's nonlinear motion model and sensor measurement models. Accurate UAV motion information is key to achieving "UAV-vehicle coupling" in the subsequent UV-KGNN (UAV vehicle coupled—Kinematic Graph Neural Network). It allows for accurate decoupling of the vehicle's real-world motion from the UAV's perspective, avoiding errors introduced by the motion of the observation platform, especially when the UAV is performing complex maneuvers.

[0054] The calculation formula of the EKF prediction step is as follows:

[0055] ;

[0056] Where, is the drone state vector (including position ,speed , attitude (quaternion or Euler angle), etc.); f() is the nonlinear motion model of the drone; is the control input at time k; P is the state covariance matrix; F is the Jacobian matrix (state transfer matrix); Q is the process noise covariance; k represents a specific time point in the time series; represents the posterior state covariance matrix at time k-1, which is obtained after the prediction covariance matrix at time k-1 is measured and updated. It represents the best estimate of the uncertainty of the state at time k-1 after combining the measurement information at time k-1; represents the drone state vector at time k predicted based on the drone state vector at time k-1; represents the Jacobian matrix at time k-1; represents the state covariance matrix at time k-1; represents the covariance matrix of the process noise at time k-1; represents the posterior state estimate at time k-1; T represents the matrix transpose.

[0057] The calculation formula of the EKF update step is as follows:

[0058] ;

[0059] Where, is the sensor measurement value at time k (GPS position, IMU angular velocity / acceleration); h() is the measurement model; H is the measurement model Jacobian matrix; R is the measurement noise covariance; K is the Kalman gain; I is the identity matrix; is the actual measurement value (observation value) collected by the sensor at time k; is the predicted state at time k based on time k The measured values ​​predicted by the measurement model h() are: is the predicted covariance of the measurement at time k; for The first derivative of ; is the final posterior estimate of the state at time k after measurement update; is the posterior estimated covariance matrix of the state at time k after measurement update; is the Kalman gain at time k; is the Jacobian matrix of the measurement model at time k; T represents the matrix transpose; is the measurement noise covariance at time k.

[0060] Step S12: Target detection and feature extraction are performed based on the preset initial network and the target physical quantity, a UV-KGNN network is constructed using the corresponding vehicle detection frame and deep visual features, and the UV-KGNN network is used to predict the vehicle's uncertain trajectory to obtain a trajectory prediction result; the UV-KGNN network is a kinematic graph neural network that couples the UAV and the vehicle.

[0061] In this embodiment, after determining the target physical quantity, multi-scale deformable small target detection and deep feature extraction are performed based on the target physical quantity to accurately obtain the vehicle detection frame and deep visual features. That is, multi-scale deformable small target detection is performed based on a preset initial network and the target physical quantity to determine the vehicle detection frame; the preset initial network is a network based on the YOLOv8 network, combined with a preset deformable convolutional network and a dynamic feature fusion pyramid network; based on the preset deformable convolutional network in the preset initial network and the target physical quantity, deep features of different scales and deformations are captured, and feature enhancement is performed using the dynamic feature fusion pyramid network in the preset initial network to determine the deep visual features.

[0062] It should be understood that for multi-scale deformable small target detection and deep feature extraction, in this embodiment, YOLOv8 is combined with DCNv3 (Deformable Convolutional Networks v3, the third version of the deformable convolutional network) and DFFPN (a feature pyramid network (FPN) that introduces dynamic feature fusion technology (DFF), referred to as dynamic feature fusion pyramid network). Among them, DCNv3 is optimized: to address the problem of drastic changes in vehicle size and deformation from the perspective of drones, DCNv3 learns the sampling offset and modulation scalar , which enables the convolution kernel to adaptively adjust its receptive field shape and the contribution of each sampling point, more flexibly capturing target features of different scales and deformations. This optimization improves the recall rate and positioning accuracy of small target detection, providing the UV-KGNN network with more accurate initial vehicle observations. The convolution operation can be expressed as:

[0063] ;

[0064] Where, is the current position on the output feature map; Offset to predefined standard grid; is the weight; x is the input feature map; is a learnable modulation scalar, ; is the learnable sampling offset, , represents the set of real numbers.

[0065] Enhancements to DFFPN: Traditional FPN may suffer from semantic misalignment and information redundancy when fusing features at different scales. DFFPN dynamically adjusts the fusion weights of feature maps at different scales by introducing spatial attention and channel attention fusion modules. SCAFM (Spatial Channel Attention Fusion Module) first learns channel attention in the channel dimension through global average pooling and fully connected layers, then learns spatial attention in the spatial dimension through convolutional layers, and finally combines the two attentions to achieve adaptive feature fusion at the pixel and channel levels. This enhancement ensures that the key features of small targets are not diluted during the multi-scale fusion process, providing the UV-KGNN network with richer and more discriminative visual features, thereby more accurately estimating the vehicle's kinematic parameters.

[0066] ;

[0067] Where, are features from low-level (fine-grained) feature maps, are features from high-level (fine-grained) feature maps, is the fusion feature, the superscript Indicates a pyramid level.

[0068] Furthermore, this embodiment performs multi-target tracking based on vehicle detection frames and deep vision features to maintain target identities. Based on this, a UV-KGNN network is constructed (this network integrates vehicle dynamics models, multi-target spatial interactions, and drone-coupled motion information to achieve high-precision, physically consistent predictions of future vehicle trajectories and simultaneously quantify prediction uncertainty). The constructed UV-KGNN network is then used to predict vehicle trajectories. Specifically, target recognition is performed based on the corresponding vehicle detection frames and deep vision, and the identities of the same target in different frames are associated to complete multi-target tracking operations and determine multi-target tracking results. A UV-KGNN network is constructed based on the multi-target tracking results and a preset vehicle dynamics model. The UV-KGNN network is used to determine the real-time kinematic parameters corresponding to each vehicle, using a preset improved recurrent neural network, the target physical quantities, and the deep vision features. Future trajectories of each vehicle are predicted based on the target graph attention layer in the UV-KGNN network and the real-time kinematic parameters, and uncertainty analysis of the predicted trajectories is performed to obtain trajectory prediction results. The target graph attention layer is a graph attention layer used to integrate physical information corresponding to the drone and vehicle into graph node features.

[0069] It is important to understand that regarding the construction of the UV-KGNN network and the uncertain trajectory prediction of vehicles, in this embodiment, the core of the UV-KGNN lies in its Physical Information Fusion Graph Attention Network (P-GAT). This layer goes beyond simple data-driven prediction. Instead, it deeply integrates the vehicle's nonlinear dynamics model (such as the nonlinear dual-track model) as a strong physical prior, and the drone's high-precision motion state as an observation platform constraint, into the feature updates of graph nodes. This deep integration of the vehicle dynamics model ensures that the predicted trajectory not only conforms to the data distribution but also conforms to physical laws, significantly improving prediction accuracy, especially during complex maneuvers. Furthermore, the graph neural network effectively captures the spatial influence between vehicles, making the prediction closer to real-world traffic flow and enabling multi-objective interactive modeling. Furthermore, by integrating the drone's own motion, it accurately decouples relative motion, improving prediction accuracy. Furthermore, the model simultaneously predicts the mean and covariance matrix of the trajectory, thereby quantifying the uncertainty of the prediction. This provides the basis for subsequent high-confidence violation determinations and achieves risk perception.

[0070] Vehicle kinematic parameter estimation: The real-time kinematic parameters of the vehicle are estimated using a GRU (Gated Recurrent Unit) network (a type of recurrent neural network) from historical visual features and observation states (information directly obtained from sensors or detection results at each time step that describes the vehicle's state in the sensor coordinate system or image coordinate system, including the position of the vehicle detection frame). Historical visual features, referred to as the aforementioned deep visual features, contain information such as the vehicle's texture, outline, and color from different perspectives, and can capture subtle deformations, posture changes, and specific pixel representations of the vehicle in the image. The observation state refers to information directly obtained from sensors or detection results at each time step that describes the vehicle's state in the sensor coordinate system or image coordinate system. This information is used to reflect the vehicle's projection or preliminary positioning in the observation space, including the position of the vehicle detection frame (e.g., the position of its center coordinates in the image), its size (e.g., width and height), or its preliminary estimated position in the world coordinate system (i.e., the first target physical quantity obtained above).

[0071] ;

[0072] Where, is the estimated longitudinal velocity of vehicle i at time t; is the estimated front wheel steering angle of vehicle i at time t; is the estimated longitudinal acceleration of vehicle i at time t; For vehicle i at time visual characteristics; For vehicle i at time Observation status; It refers to the length of the historical observation time window on which the vehicle kinematic parameter estimation depends.

[0073] Vehicle nonlinear dual-track model (Bicycle Model) prediction: Predict the future state of the vehicle based on the estimated kinematic parameters.

[0074] ;

[0075] Where, is the position of vehicle i at time t; is the heading angle of vehicle i at time t; is the longitudinal velocity of vehicle i at time t; is the acceleration of vehicle i at time t; is the time step; is the distance from the vehicle's front wheelbase to the center of gravity; is the distance from the vehicle's rear wheelbase to the center of gravity; is the front wheel steering angle of vehicle i at time t; is the sideslip angle of vehicle i at time t; is the longitudinal velocity of vehicle i at time t+1; is the position of vehicle i at time t+1; is the heading angle of vehicle i at time t+1.

[0076] Physical Information Fusion Graph Attention Layer (P-GAT) Update:

[0077] ;

[0078] Where LeakyReLU represents the leaky linear rectification function; is the node (representing vehicle) in the Characteristics of the layer; Node (representing vehicle) j in the Characteristics of the layer; is the set of neighbor nodes of node i; For the The attention weight of the layer measures the importance of neighbor j to i; For the The learnable weight matrix of the layer; For the The learnable linear mapping matrix of the layer transforms the motion increments predicted by the physical model Mapping to the feature space of graph nodes; The future prediction of vehicle j by the dual-track model (i.e., the t+ physical state at the moment; For vehicle j in the The current (i.e., time t) physical state encoded in the layer features.

[0079] Trajectory prediction and uncertainty quantification:

[0080] ;

[0081] Where, is the future trajectory sequence predicted for vehicle i at time t (i.e., the current moment); is the mean of the predicted position of vehicle i in the kth step in the future; is the covariance matrix of the predicted position of vehicle i in the kth step in the future, which is used to quantify the uncertainty of the prediction; is the visual feature of vehicle i at time t; a sequence of visual features that represent history; It represents the length of trajectory prediction or the number of predicted future time steps; UV represents multiple target physical quantities related to the operating status of drones and vehicles; KGNN stands for Knowledge Graph Neural Network, knowledge graph attention network.

[0082] Step S13: classify and judge the current or future potential traffic violation based on the trajectory prediction result, the preset classifier, and the confidence correction network in the preset classifier to determine the behavior judgment result.

[0083] In this embodiment, the vehicle's refined historical trajectory, future predicted trajectory, and predicted uncertainty information output by the UV-KGNN are input into an uncertainty-aware prospective classifier (UAPC). The UAPC uses multimodal fusion and a confidence correction network to perform high-confidence classification and judgment of current or future potential traffic violations, thereby achieving forward-looking warnings. Specifically, based on a preset classifier, the trajectory prediction results, and the vehicle's historical trajectory information output by the UV-KGNN network, the vehicle's current or future potential traffic violations are classified and judged, and an initial confidence level is determined for each potential traffic violation. The confidence correction network in the preset classifier calibrates the initial confidence level of each potential traffic violation to determine a target confidence level for each potential traffic violation. The behavior judgment result is determined based on the target confidence level, threshold information corresponding to each preset traffic violation category, and the trajectory prediction results.

[0084] Specifically, regarding uncertainty-aware forward-looking violation classification and determination, the core of the UAPC in this embodiment lies in its Confidence Correction Network (CRN). This network leverages the trajectory prediction uncertainty (covariance matrix) and historical observation consistency output by the UV-KGNN to dynamically correct and calibrate the initial confidence level of the violation determination. This ensures that the final violation determination confidence level more accurately reflects the actual probability of a violation, enabling more reliable forward-looking triggering (i.e., issuing warnings before a violation occurs) and significantly reducing the false alarm rate. Furthermore, by combining high-precision maps with expert rules, the determination process can adapt to complex traffic scenarios (such as varying road speed limits and specific lane regulations), avoiding misjudgments and supporting the determination of multiple traffic violation types, achieving contextual adaptability.

[0085] Final judgment confidence correction:

[0086] ;

[0087] in, is the final calibrated violation judgment confidence, i.e., the target confidence; The initial violation confidence output by the classifier; The trajectory covariance matrix predicted by UV-KGNN (indicating the uncertainty of trajectory prediction); To measure the consistency between historical observations and the current predicted trajectory, residuals or Gaussian goodness of fit between the trajectory and the historical observations are typically calculated to quantify the accuracy and reliability of the model's predictions. If the historical observations closely match the model's current predicted trajectory, the model's predictions are reliable. Conversely, if there are significant discrepancies, this may indicate a decline in the model's predictive power at the current moment, necessitating revisions or adjustments to its confidence level.

[0088] Example of forward-looking violation determination logic (Note: Unified reference , that is, the final violation judgment confidence after confidence correction):

[0089] Running a red light (forward-looking judgment): The judgment is made based on the vehicle's predicted trajectory and the status of the intersection's stop line and traffic lights.

[0090] ;

[0091] Where, represents the judgment of vehicle i’s red light running behavior; is the predicted position of vehicle i at k moments in the future; Represents the model's final probability output of whether the vehicle violates traffic regulations; is the stop line position; is the distance threshold; Traffic light status; is a forward-looking trigger threshold; Indicates that the initial position of vehicle i at time t is behind the stop line; The length of the future time period covered when the model makes trajectory predictions, or the upper limit of the number of future time steps predicted.

[0092] Speeding (forward-looking judgment) is based on the instantaneous speed of the vehicle at the current time t and the predicted future speed, which are compared with the current road speed limit provided by the HD map.

[0093] ;

[0094] Where, represents the speeding behavior judgment of vehicle i; is the instantaneous speed of vehicle i at the current time t; is the predicted speed of vehicle i at the next k moments; The speed limit of the current road (provided by the HD map); The speed threshold is allowed. Exceeding the speed limit plus this threshold is considered speeding. Represents the model's final probability output of whether the vehicle violates traffic regulations; is a forward-looking trigger threshold; The length of the future time period covered when the model makes trajectory predictions, or the upper limit of the number of future time steps predicted.

[0095] Illegal lane change (forward-looking judgment): judged based on the interaction between the vehicle trajectory and lane lines (provided by high-precision maps).

[0096] ;

[0097] in, represents the judgment of illegal lane change behavior of vehicle i; is the predicted trajectory of vehicle i at time t; both solid lane lines and lane lines (including solid and dashed lines) can be provided by HD maps; CountCrossing() is a function that counts the number of times a trajectory crosses a specific lane line; Nthreshold is the crossing count threshold; Crossing() is a function that determines whether the trajectory crosses a lane line; NoTurnSignalObserved is a logical judgment, indicating that the vehicle's turn signal was not detected when it changed lanes; is a forward-looking trigger threshold; Right now , which represents the final probability output of the model for whether the vehicle violates traffic regulations.

[0098] Step S14: Based on the target confidence corresponding to each target potential traffic violation behavior in the behavior determination result, the processing strategy corresponding to each target potential traffic violation behavior is analyzed to determine the target traffic violation detection result.

[0099] In this embodiment, based on the UAPC's determination results, high-confidence violations are automatically confirmed and filed, while low-confidence suspicious events are prompted for manual review. Multi-level outputs, including real-time warnings, detailed reports, and visual interfaces, are generated to fully support intelligent law enforcement. Specifically, based on preset label generation rules and the target confidence levels corresponding to each target potential traffic violation in the behavior determination results, target behavior labels corresponding to each target potential traffic violation are determined. These target behavior labels include high-confidence violation labels and low-confidence violation labels. Based on the corresponding target behavior labels and the preset violation handling strategies, a handling strategy for each target potential traffic violation is determined. Finally, a target traffic violation detection result is determined based on each target potential traffic violation and the corresponding handling strategy. It is understood that the confidence level can be determined by setting a threshold or selecting the highest N values.

[0100] Furthermore, this embodiment can also incorporate an online incremental learning algorithm to perform lightweight parameter updates on the UV-KGNN and UAPC based on actual deployment data, continuously improving the model's adaptability and recognition accuracy in dynamic environments and preventing catastrophic forgetting. Specifically, based on the elastic weight solidification algorithm, a preset task perception selection mechanism, and a preset online incremental learning algorithm, the UV-KGNN network and the preset classifier are lightweight parameter updates combined with current traffic-related data to complete traffic violation detection using the updated UV-KGNN network and the updated preset classifier. Actual deployment data refers to the traffic-related data collected and continuously flowing in real time after the drone traffic violation detection solution proposed in this embodiment is deployed in actual traffic enforcement scenarios. This data represents a new data stream encountered in real-world application environments that reflects the current traffic conditions. Its core purpose is to continuously adjust and optimize the model, enabling it to adapt to the ever-changing external environment and traffic regulations, thereby maintaining high accuracy and robustness.

[0101] Specifically, regarding online learning, this embodiment utilizes Elastic Weight Consolidation (EWC) combined with task-aware selection. EWC technology protects old task knowledge during incremental learning, preventing catastrophic forgetting. Furthermore, a task-aware selection mechanism is introduced to intelligently select and update the subset S of model parameters most relevant to the new task based on the characteristics of the new sample, rather than blindly updating all parameters. This significantly improves the efficiency of incremental learning and computing resource utilization while maintaining model performance. This mechanism ensures that the UV-KGNN and its predictions and judgments can maintain high accuracy and robustness in the face of ever-changing real-world traffic environments, possessing the ability to self-evolve, reducing manual maintenance costs and extending the system lifecycle.

[0102] ;

[0103] wherein, is the total loss function; is the loss of the new task (e.g., the violation detection sample under the new scene); S is a subset of parameters related to the old task selected according to the new task data, for example, only update the feature extraction layer related to a specific vehicle type or light condition; is the regularization strength, balancing new and old task learning; is the parameter The importance of the old task loss is estimated by the Fisher information matrix; is the current i-th model parameter; is the i-th parameter value when the old model training is completed.

[0104] In this way, by deeply integrating the traffic violation detection scheme described in the embodiment into the intelligent traffic law enforcement system, seamless connection from violation detection to automatic fine generation and pushing can be realized, and key evidence tamper-proof management is performed through technologies such as blockchains, and the automation, fairness and credibility of traffic management are comprehensively improved.

[0105] In summary, the embodiment provides a forward-looking traffic violation detection scheme based on unmanned aerial vehicle-vehicle coupled kinematic graph neural network (UV-KGNN) for traffic violation behaviors such as red light running, overspeed driving, and illegal lane changing, which can realize high-precision, physically consistent prediction and uncertainty quantification of multi-target future trajectories in traffic flow, conduct high-confidence forward-looking violation judgment and risk assessment, and finally through an automated intelligent law enforcement closed loop, significantly improve the reliability and efficiency of intelligent law enforcement, and solve the problems of physical inconsistency of trajectory prediction, uncertainty quantification, multi-target interaction neglect, and high violation judgment lag and misjudgment rate in the prior art. Finally, combined with online incremental learning and intelligent decision system, a full-automatic intelligent law enforcement closed loop is realized, and the reliability and forward-looking nature of traffic violation detection are significantly improved.

[0106] It can be seen that in this application, the multimodal traffic-related data collected by the drone is first processed to determine multiple target physical quantities related to the operating status of the drone and the vehicle. After that, target detection and feature extraction are performed through the preset initial network and target physical quantities to construct a UV-KGNN network using the corresponding vehicle detection frame and deep visual features, and the UV-KGNN network is used to determine the trajectory prediction result. Then, based on the trajectory prediction result, the preset classifier and the confidence correction network therein, the current or future potential traffic violations are classified and determined. The corresponding processing strategy is determined based on the target confidence corresponding to each target potential traffic violation in the behavior judgment result. In this way, it is possible to effectively achieve drone traffic violation detection with high precision, physical consistency, uncertainty quantification and strong environmental adaptability, thereby reducing the false alarm rate and missed alarm rate, improving the accuracy, authenticity, foresight and reliability of the detection results, and thus laying a solid foundation for realizing a closed loop of fully automated intelligent law enforcement.

[0107] See also Figure 2 As shown, the embodiment of the present application also discloses a traffic violation detection device based on the UV-KGNN network, including:

[0108] a physical quantity determination module 11 for collecting multimodal traffic-related data through a plurality of sensors carried by the drone, and determining a plurality of target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data;

[0109] The trajectory prediction module 12 is configured to perform target detection and feature extraction based on a preset initial network and the target physical quantity, construct a UV-KGNN network using the corresponding vehicle detection frame and deep visual features, and use the UV-KGNN network to perform uncertain vehicle trajectory prediction to obtain a trajectory prediction result; the UV-KGNN network is a kinematic graph neural network that couples the UAV and the vehicle;

[0110] a violation behavior determination module 13, configured to classify and determine current or future potential traffic violations based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier to determine a behavior determination result;

[0111] The detection result determination module 14 is used to analyze the processing strategies corresponding to each target potential traffic violation behavior based on the target confidence level corresponding to each target potential traffic violation behavior in the behavior determination result to determine the target traffic violation detection result.

[0112] In some specific embodiments, the physical quantity determination module 11 can be specifically used to: collect multimodal traffic-related data based on multiple sensors carried by the drone; the multimodality includes image mode, video mode and point cloud mode; preprocess the multimodal traffic-related data to determine a preprocessing result; estimate the position, speed and attitude of the vehicle in the world coordinate system based on a first preset state estimation algorithm and the preprocessing result to determine a plurality of first target physical quantities related to the vehicle's operating state; estimate the position, speed and attitude of the drone in the world coordinate system based on a second preset state estimation algorithm and the preprocessing result, and in combination with the drone position information provided by a preset positioning system and the drone acceleration information and drone angular velocity information provided by an inertial measurement component to determine a plurality of second target physical quantities related to the drone's operating state.

[0113] In some specific embodiments, the trajectory prediction module 12 can be specifically used to: perform multi-scale deformable small target detection based on a preset initial network and the target physical quantity to determine a vehicle detection frame; the preset initial network is a network based on the YOLOv8 network, and is combined with a preset deformable convolutional network and a dynamic feature fusion pyramid network to construct the network; based on the preset deformable convolutional network and the target physical quantity in the preset initial network, deep features of different scales and deformations are captured, and feature enhancement is performed using the dynamic feature fusion pyramid network in the preset initial network to determine deep visual features.

[0114] In some specific embodiments, the trajectory prediction module 12 can be specifically used to: perform target recognition based on the corresponding vehicle detection frame and depth vision and associate the identity of the same target in different frames to complete multi-target tracking operations and determine multi-target tracking results; construct a UV-KGNN network based on the multi-target tracking results and a preset vehicle dynamics model; use the UV-KGNN network, and combine it with a preset improved recurrent neural network and the target physical quantity and the deep vision features to determine the real-time kinematic parameters corresponding to each vehicle; predict the future trajectory of each vehicle based on the target graph attention layer and the real-time kinematic parameters in the UV-KGNN network, and perform uncertainty analysis of the predicted trajectory to obtain a trajectory prediction result; the target graph attention layer is a graph attention layer used to fuse the physical related information corresponding to the drone and the vehicle into the graph node features.

[0115] In some specific embodiments, the violation determination module 13 can be specifically used to: classify and determine the vehicle's current or future potential traffic violations based on a preset classifier, the trajectory prediction result, and the historical trajectory information corresponding to the vehicle output by the UV-KGNN network, and determine the initial confidence of each potential traffic violation; calibrate the initial confidence of each potential traffic violation based on the confidence correction network in the preset classifier to determine the target confidence of each potential traffic violation; determine the behavior determination result based on the target confidence, the threshold information corresponding to each preset traffic violation category, and the trajectory prediction result.

[0116] In some specific embodiments, the detection result determination module 14 can be specifically used to: determine the target behavior label corresponding to each target potential traffic violation based on the preset label generation rules and the target confidence corresponding to each target potential traffic violation in the behavior judgment result; the target behavior label includes a high-confidence violation behavior label and a low-confidence violation behavior label; based on the corresponding target behavior label and the preset violation behavior processing strategy, determine the processing strategy for each target potential traffic violation; determine the target traffic violation detection result based on each target potential traffic violation and the corresponding processing strategy.

[0117] In some specific embodiments, the traffic violation detection device based on the UV-KGNN network can also be used to: based on the elastic weight solidification algorithm, the preset task perception selection mechanism and the preset online incremental learning algorithm, combine the current traffic-related data to perform lightweight parameter updates on the UV-KGNN network and the preset classifier, so as to use the updated UV-KGNN network and the updated preset classifier to complete the traffic violation detection operation.

[0118] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0119] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the traffic violation detection method based on the UV-KGNN network disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0120] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0121] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0122] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of implementing the traffic violation detection method based on the UV-KGNN network and executed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs capable of completing other specific tasks.

[0123] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned UV-KGNN network-based traffic violation detection method. The specific steps of this method can be referred to the corresponding content disclosed in the aforementioned embodiments and will not be repeated here.

[0124] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0125] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0127] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0128] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A traffic violation detection method based on UV-KGNN network, characterized in that: include: Collecting multimodal traffic-related data through multiple sensors carried by the drone, and determining multiple target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data; Target detection and feature extraction are performed based on the preset initial network and the target physical quantity. A UV-KGNN network is constructed using the corresponding vehicle detection frame and deep visual features. The UV-KGNN network is then used to predict the vehicle's uncertain trajectory to obtain a trajectory prediction result. The UV-KGNN network is a kinematic graph neural network that couples the UAV and vehicle. Classifying and judging current or future potential traffic violations based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier to determine a behavior judgment result; Analyzing the processing strategies corresponding to each target potential traffic violation behavior based on the target confidence level corresponding to each target potential traffic violation behavior in the behavior determination result to determine the target traffic violation detection result; The method of constructing a UV-KGNN network using the corresponding vehicle detection frame and deep visual features, and using the UV-KGNN network to perform uncertainty trajectory prediction of the vehicle to obtain a trajectory prediction result includes: Target recognition is performed based on the corresponding vehicle detection frame and deep visual features, and the identity of the same target in different frames is associated to complete the multi-target tracking operation and determine the multi-target tracking results; Constructing a UV-KGNN network based on the multi-target tracking results and a preset vehicle dynamics model; Determine the real-time kinematic parameters corresponding to each vehicle by using the UV-KGNN network in combination with a preset improved recurrent neural network, the target physical quantity, and the deep visual features; Based on the target graph attention layer in the UV-KGNN network and the real-time kinematic parameters, the future trajectory of each vehicle is predicted, and uncertainty analysis of the predicted trajectory is performed to obtain a trajectory prediction result; the target graph attention layer is a graph attention layer used to fuse the physical related information corresponding to the drone and the vehicle into the graph node features.

2. The traffic violation detection method based on UV-KGNN network according to claim 1 is characterized in that: The multimodal traffic-related data is collected by multiple sensors carried by the drone, and based on the corresponding multimodal traffic-related data, multiple target physical quantities related to the operating status of the drone and the vehicle are determined, including: Collect multimodal traffic-related data based on multiple sensors carried by drones; the multimodal data includes image modality, video modality, and point cloud modality; Preprocessing the multimodal traffic-related data to determine a preprocessing result; estimating the position, velocity, and posture of the vehicle in a world coordinate system based on a first preset state estimation algorithm and the preprocessing result to determine a plurality of first target physical quantities related to the vehicle operating state; Based on the second preset state estimation algorithm and the preprocessing result, and in combination with the drone position information provided by the preset positioning system and the drone acceleration information and drone angular velocity information provided by the inertial measurement component, the position, velocity and attitude of the drone in the world coordinate system are estimated to determine multiple second target physical quantities related to the drone's operating state.

3. The traffic violation detection method based on UV-KGNN network according to claim 1 is characterized in that: The target detection and feature extraction based on the preset initial network and the target physical quantity includes: Perform multi-scale deformable small target detection based on a preset initial network and the target physical quantity to determine a vehicle detection frame; the preset initial network is a network constructed based on a YOLOv8 network and combined with a preset deformable convolutional network and a dynamic feature fusion pyramid network; Based on the preset deformable convolutional network in the preset initial network and the target physical quantity, deep features of different scales and deformations are captured, and the dynamic feature fusion pyramid network in the preset initial network is used to perform feature enhancement to determine deep visual features.

4. The traffic violation detection method based on UV-KGNN network according to claim 1 is characterized in that: The classification and determination of current or future potential traffic violations based on the trajectory prediction result, the preset classifier, and the confidence correction network in the preset classifier include: Based on a preset classifier, the trajectory prediction result, and the historical trajectory information corresponding to the vehicle output by the UV-KGNN network, classify and determine the vehicle's current or future potential traffic violations, and determine an initial confidence level for each potential traffic violation; Calibrate the initial confidence of each of the potential traffic violation behaviors based on the confidence correction network in the preset classifier to determine a target confidence of each of the potential traffic violation behaviors; The behavior determination result is determined based on the target confidence, the threshold information corresponding to each preset traffic violation behavior category and the trajectory prediction result.

5. The traffic violation detection method based on UV-KGNN network according to claim 1 is characterized in that: The analyzing, based on the target confidence level corresponding to each target potential traffic violation in the behavior determination result, the processing strategy corresponding to each target potential traffic violation, includes: Determining target behavior labels corresponding to each target potential traffic violation based on preset label generation rules and the target confidence corresponding to each target potential traffic violation in the behavior determination result; the target behavior labels include high-confidence violation labels and low-confidence violation labels; Determine a handling strategy for each of the target potential traffic violations based on the corresponding target behavior label and the preset violation handling strategy; A target traffic violation detection result is determined based on each of the target potential traffic violation behaviors and the corresponding processing strategy.

6. The traffic violation detection method based on UV-KGNN network according to claim 1 is characterized in that: Also includes: Based on the elastic weight solidification algorithm, the preset task perception selection mechanism and the preset online incremental learning algorithm, the UV-KGNN network and the preset classifier are lightweight parameter updated in combination with the current traffic-related data, so as to complete the traffic violation detection operation using the updated UV-KGNN network and the updated preset classifier.

7. A traffic violation detection device based on UV-KGNN network, characterized in that: include: a physical quantity determination module, configured to collect multimodal traffic-related data through a plurality of sensors carried by the drone, and determine a plurality of target physical quantities related to the operating status of the drone and the vehicle based on the corresponding multimodal traffic-related data; A trajectory prediction module is used to perform target detection and feature extraction based on a preset initial network and the target physical quantity, construct a UV-KGNN network using the corresponding vehicle detection box and deep visual features, and use the UV-KGNN network to predict the vehicle's uncertain trajectory to obtain a trajectory prediction result; the UV-KGNN network is a kinematic graph neural network that couples the UAV and vehicle; a violation behavior determination module, configured to classify and determine current or future potential traffic violations based on the trajectory prediction result, a preset classifier, and a confidence correction network in the preset classifier to determine a behavior determination result; a detection result determination module, configured to analyze, based on the target confidence level corresponding to each target potential traffic violation in the behavior determination result, a processing strategy corresponding to each target potential traffic violation to determine a target traffic violation detection result; Among them, the trajectory prediction module is used to: perform target recognition based on the corresponding vehicle detection frame and deep visual features and associate the identity of the same target in different frames to complete multi-target tracking operations and determine the multi-target tracking results; construct a UV-KGNN network based on the multi-target tracking results and a preset vehicle dynamics model; use the UV-KGNN network, and combine it with the preset improved recurrent neural network and the target physical quantity and the deep visual features to determine the real-time kinematic parameters corresponding to each vehicle; predict the future trajectory of each vehicle based on the target graph attention layer and the real-time kinematic parameters in the UV-KGNN network, and perform uncertainty analysis of the predicted trajectory to obtain a trajectory prediction result; the target graph attention layer is a graph attention layer used to fuse the physical related information corresponding to the drone and the vehicle into the graph node features.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the traffic violation detection method based on the UV-KGNN network as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the traffic violation detection method based on the UV-KGNN network as described in any one of claims 1 to 6.

Images