Expressway dynamic vehicle behavior trajectory tracking, sensing and monitoring system

Abstract

The invention discloses an expressway dynamic vehicle behavior trajectory tracking induction monitoring system, which relates to the technical field of intelligent traffic monitoring, and comprises a multi-source data acquisition module used for responding to a trigger signal of a vehicle entering a monitoring area, acquiring multi-source data, and sending the multi-source data to the monitoring area; the multi-source data comprises three-dimensional point cloud data, a thermal infrared image, a multi-view visible light image, vehicle speed and lane data and a license plate image; and the preprocessing and calibration module is used for registering and fusing the thermal infrared image and the visible light image. According to the expressway dynamic vehicle behavior trajectory tracking induction monitoring system provided by the invention, through multi-source sensor fusion and intelligent algorithm innovation, high-precision tracking and real-time monitoring of vehicle behavior trajectories are realized, road safety and traffic efficiency are effectively improved, and the expressway dynamic vehicle behavior trajectory tracking induction monitoring system is suitable for popularization and application. Multi-dimensional sensors such as a laser radar, an infrared thermal inductance instrument and a visible light camera are adopted for cooperative work, and the capability of capturing vehicle information in a weak light environment is enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic monitoring technology, specifically to a dynamic vehicle behavior trajectory tracking and sensing monitoring system for highways. Background Technology

[0002] As my country's expressway mileage continues to grow, its core position in regional economic linkages and the comprehensive transportation system is becoming increasingly prominent. The surge in traffic flow and the complexity of vehicle driving conditions have placed higher demands on road safety management and traffic efficiency improvement. Dynamic tracking and precise monitoring of vehicle behavior trajectories, as a core supporting technology for intelligent traffic management, can provide crucial data for traffic violation identification, accident early warning and handling, and toll collection and supervision. This helps to shift the traffic governance model from passive response to proactive prevention and control. Currently, it is being gradually promoted and applied in various traffic control scenarios, becoming an important technical means to ensure road traffic order and safety.

[0003] Existing monitoring technologies mostly rely on single sensors or simple combinations for data collection and analysis, leaving room for optimization in practical applications. Some video-based detection solutions are susceptible to changes in lighting and inclement weather, with reduced accuracy in low-light and foggy conditions. Single-sensor-based acquisition methods struggle to comprehensively capture multi-dimensional data such as vehicle 3D information and motion status, resulting in limited information coverage. Furthermore, insufficient efficiency in time synchronization, spatial calibration, and feature fusion of multi-source data hinders the accuracy of vehicle identification and trajectory continuity tracking, and necessitates further enhancement in the comprehensiveness of identifying complex abnormal behaviors such as lane changes across solid lines and close following. These factors make it difficult for existing technologies to fully meet the demands of all-weather, high-precision, and comprehensive highway monitoring. To address this, we propose a dynamic vehicle behavior trajectory tracking and sensing monitoring system for highways. Summary of the Invention

[0004] To address the aforementioned technical issues, a dynamic vehicle behavior trajectory tracking and sensing monitoring system for highways is provided. This technical solution solves the problems of existing technologies that mostly employ single sensors or simple combination schemes, video detection being easily affected by lighting and weather, and single sensors being unable to comprehensively acquire multi-dimensional data; insufficient efficiency in time synchronization, spatial calibration, and feature fusion of multi-source data, resulting in the need to improve the accuracy of vehicle identity binding and trajectory tracking, and also requiring enhanced comprehensiveness in judging complex abnormal behaviors.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A dynamic vehicle behavior trajectory tracking and monitoring system for highways includes: The multi-source data acquisition module is used to respond to the trigger signal of a vehicle entering the monitoring area and acquire multi-source data, including: three-dimensional point cloud data, thermal infrared images, multi-view visible light images, vehicle speed and lane data, and license plate images. The preprocessing and calibration module is used to register and fuse the thermal infrared image and the visible light image, cluster and denoise the three-dimensional point cloud data, and uniformly calibrate the point cloud coordinate system and the image coordinate system. The target recognition module is used to process the calibrated fused data based on the target detection algorithm, and to identify and bind the vehicle's license plate, vehicle type and axle number information; The trajectory tracking module is used to predict and match the vehicle's state in consecutive frames based on the vehicle identification information and millimeter-wave radar data using a multi-target tracking algorithm, and generate a continuous driving trajectory that includes position, speed and lane. The analysis and early warning module is used to compare the driving trajectory with preset rules, identify traffic violations and abnormal events, and generate early warning information; The output application module is used to output vehicle image data, structured trajectory data, early warning reports and visualization information, and to connect with the toll collection system and traffic management platform.

[0006] Preferably, the process of responding to the trigger signal of a vehicle entering the monitoring area and collecting multi-source data specifically involves: Based on the trigger detection equipment deployed on the roadside, it monitors in real time whether vehicles enter the monitoring area; When a vehicle is detected entering, a trigger signal is generated and sent to the multi-source data acquisition module; The multi-source data acquisition module parses the trigger signal and generates a unified acquisition command. The acquisition command is broadcast to all data acquisition units to begin acquiring multi-source data.

[0007] Preferably, based on the unified acquisition command received by the multi-source data acquisition module, all data acquisition units are started synchronously, controlling the lidar, infrared thermal sensor, visible light camera group, millimeter-wave radar and license plate recognition device to start data acquisition simultaneously; A unified timestamp is applied to the raw data streams collected by each sensor, so that the 3D point cloud, thermal infrared image, multi-view visible light image, vehicle speed and lane data and license plate image are aligned in time. When the vehicle completely leaves the monitoring area, the multi-source data acquisition module sends a stop command to all data acquisition units to end the current acquisition task. The data acquisition process is completed by integrating the time-stamped raw data generated by each sensor during the acquisition process to form a time-aligned multimodal data packet.

[0008] Preferably, the preprocessing and calibration module comprises the following steps: Based on multimodal data packets, thermal infrared images and multi-view visible light images are acquired. An algorithm based on offset convolutional networks is used to spatially align and fuse their features to obtain a multispectral fused image. Acquire 3D point cloud data from LiDAR, process it using Euclidean clustering algorithm, segment out independent vehicle target point cloud clusters and filter out discrete noise points to obtain denoised point cloud clusters; A world coordinate system is established, and the rotation matrix and translation vector of the lidar to the world coordinate system, as well as the rotation matrix and translation vector of the camera to the world coordinate system, are obtained through joint calibration. Multiply the coordinates of points in the point cloud cluster by the inverse of the rotation matrix of the laser radar to the world coordinate system, and then superimpose the translation vector to obtain the coordinates of the point cloud in the world coordinate system. The image pixel coordinates are converted to coordinates in the camera coordinate system using the camera intrinsic parameter matrix, then multiplied by the inverse of the rotation matrix from the camera to the world coordinate system, and the translation vector is superimposed to obtain the coordinates of the corresponding point in the image in the world coordinate system. By integrating the multispectral fused image, the denoised point cloud clusters, and their coordinate correspondence in the world coordinate system, calibrated fused data is obtained.

[0009] Preferably, the target recognition module identifies vehicle attributes using the following method: Based on the fusion data output by the preprocessing and calibration modules, a multispectral fused image and a denoised vehicle point cloud cluster are obtained. Define the target detection model and obtain its pre-trained weight parameters and network structure; The multispectral fused image is input into the target detection model. Through forward propagation calculation, the license plate region in the image is located and its character sequence is identified, and the license plate recognition result is output. Simultaneously, the multispectral fusion image is fused with the corresponding vehicle's laser point cloud cluster at the feature level to extract the joint features of the vehicle's contour and texture. The classifier calculates the matching degree between the joint features and the preset vehicle type and axle number categories, and outputs the vehicle type and axle number recognition results. By integrating the license plate recognition results, vehicle model recognition results, and axle count recognition results, a set of attribute information for the vehicle is obtained.

[0010] Preferably, the target recognition module binds the vehicle's license plate, vehicle type, and axle count information using the following method: Based on the obtained set of attribute information of the vehicle, obtain the license plate, vehicle type, number of axles and appearance image data of all vehicles detected at the same time; Set an identity binding benchmark, using the vehicle detection box ID and spatial location output by the target detection model as the initial association basis; Calculate the association between various attribute information under the same detection box ID and assign them to the same temporary vehicle identifier; All attribute information assigned to the same temporary identifier is merged to generate a unique vehicle digital profile containing license plate, vehicle type, number of axles, and feature image, thus completing the binding of multi-dimensional information to a single vehicle entity.

[0011] Preferably, the method for predicting the vehicle motion state of the trajectory tracking module is as follows: Based on the vehicle digital archives and historical trajectory sequences output by the target recognition module, the world coordinate system position and instantaneous speed of each vehicle at the previous moment are obtained. Define the state vector and covariance matrix of the Kalman filter, and obtain its state transition matrix and observation matrix; The previous world coordinate system position and instantaneous velocity are input into the Kalman filter, and the prior estimate and covariance of the vehicle position at the current moment are obtained by calculating through the state transition matrix. By integrating prior estimated position, estimated velocity, and corresponding covariance, the predicted state set of all tracked vehicles in the current frame is obtained.

[0012] Preferably, the trajectory tracking module uses the following method for multi-target matching and trajectory generation: Based on the predicted state set and the new detected target set output by the target recognition module in the current frame, the fusion weight based on motion features and appearance features is obtained by matching the cost function; the motion feature is the Euclidean distance between the predicted position and the actual detection box, and the appearance feature is the cosine similarity of the image feature vector. The predicted state set and the new detection target set are input into the Hungarian algorithm. The matching cost between all predictions and detections is calculated through the cost function, and the optimal matching pair is solved. The state of the successfully matched detected target is fused with the speed and lane information measured in real time by the millimeter-wave radar, the state of the Kalman filter is updated, and the trajectory points containing global coordinates, instantaneous speed, acceleration and lane number are output. By integrating the continuously output trajectory points according to the time series, a complete and smooth continuous driving trajectory for each vehicle is generated.

[0013] Preferably, the preset rule is: Obtain the continuous driving trajectory of the vehicle output by the trajectory tracking module, and extract the global coordinates, instantaneous speed, acceleration and lane number sequence of each vehicle over time; Set up a traffic rule model and load predefined abnormal behavior judgment rules, including lane solid line coordinate range, allowed driving direction vector, stopping time threshold, minimum speed threshold and safe following distance threshold.

[0014] Preferably, the generation of early warning information specifically includes: The vehicle's real-time coordinates are compared with the coordinate range of the solid line of the lane. If the vehicle's trajectory point continuously crosses the coordinate range of the solid line, it is determined to be a lane change behavior that crosses the solid line. Calculate the dot product of the vehicle's driving direction vector and the lane's permitted driving direction vector. If the dot product is negative, it is determined to be a reverse driving behavior. The system detects the coordinate changes of continuous trajectory points of the vehicle. If the change is consistently lower than a set threshold and the duration exceeds the stop time threshold, it is determined to be an abnormal parking behavior. Compare the vehicle's instantaneous speed with the minimum speed threshold of the road segment. If the speed is consistently below the threshold and it is not during a period of traffic congestion, it is determined to be driving below the speed limit. Calculate the real-time coordinate difference between the vehicle in front and the vehicle behind in the same lane. If the distance is consistently less than the safe following distance threshold and both vehicles are traveling at speeds higher than the set value, it is determined to be a high-speed close following behavior. By associating all judgment results with the corresponding vehicle's digital file, extracting before-and-after image evidence of the abnormal behavior, and generating a structured early warning event containing vehicle identity, behavior type, timestamp, location coordinates, and evidence images.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The proposed highway dynamic vehicle behavior trajectory tracking and sensing monitoring system achieves high-precision tracking and real-time monitoring of vehicle behavior trajectories through multi-source sensor fusion and intelligent algorithm innovation, effectively improving road safety and traffic efficiency. It employs multi-dimensional sensors such as lidar, infrared thermal sensors, and visible light cameras working collaboratively, combined with image registration and fusion algorithms to enhance the ability to capture vehicle information in low-light environments. The application of a deep learning target detection model improves the accuracy of license plate recognition, vehicle type classification, and axle count determination. Furthermore, the combination of multi-target tracking algorithms and millimeter-wave radar data enables accurate generation and real-time tracking of continuous vehicle trajectories. This system not only automatically identifies traffic violations and abnormal events and generates timely warning information, but also supports structured data output and visualized information display, facilitating seamless integration with toll collection systems and traffic management platforms. Through technological innovation and practical application, it enhances road safety and smoothness, provides strong technical support for traffic management, and has significant social and economic benefits. Attached Figure Description

[0016] Figure 1 This is a system framework diagram of the present invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] Reference Figure 1 As shown, a dynamic vehicle behavior trajectory tracking and sensing monitoring system for highways includes: a multi-source data acquisition module, used to collect multi-source data in response to a trigger signal that a vehicle enters the monitoring area, wherein the multi-source data includes three-dimensional point cloud data, thermal infrared images, multi-view visible light images, vehicle speed and lane data, and license plate images; The specific steps for collecting multi-source data in response to the trigger signal of a vehicle entering the monitoring area are as follows: Based on the trigger detection equipment deployed on the roadside, the system monitors in real time whether vehicles have entered the monitoring area. The trigger detection equipment adopts a combination of inductive loops, millimeter-wave radar and video detection equipment. The inductive loops are buried under the center line of each lane along the direction of the entrance to the monitoring area, and the spacing between adjacent inductive loops is set to 5 meters. The millimeter-wave radar and video detection equipment are integrated and installed under the crossbeam of the roadside gantry, with an installation height of 6 meters. The horizontal distance between them and the data acquisition units such as lidar and infrared thermal sensors deployed later does not exceed 3 meters, ensuring that the trigger signal covers the entrance to the monitoring area without any blind spots. When a vehicle is detected entering, a trigger signal is generated and sent to the multi-source data acquisition module; The multi-source data acquisition module parses the trigger signal and generates a unified acquisition command. The acquisition command is broadcast to all data acquisition units to begin acquiring multi-source data; based on the unified acquisition command received by the multi-source data acquisition module, all data acquisition units are started synchronously, controlling the lidar, infrared thermal sensor, visible light camera group, millimeter-wave radar and license plate recognition device to start data acquisition simultaneously. A unified timestamp is applied to the raw data streams collected by each sensor. The time synchronization signal comes from a high-precision time reference provided by the GPS timing module. Each sensor’s built-in clock receives a calibration signal sent by the GPS timing module every 100 milliseconds. The unified time code is embedded in the frame header of the raw data stream in hexadecimal format through the built-in timestamp generator to ensure that the time alignment accuracy error of 3D point cloud, thermal infrared image, multi-view visible light image, vehicle speed and lane data and license plate image does not exceed 1 millisecond. When the vehicle completely leaves the monitoring area, the multi-source data acquisition module sends a stop command to all data acquisition units to end the current acquisition task; it integrates the time-stamped raw data generated by each sensor during the acquisition process to form a time-aligned multimodal data packet, thus completing the data acquisition process.

[0019] The preprocessing and calibration module is used to register and fuse the thermal infrared image and the visible light image, cluster and denoise the three-dimensional point cloud data, and uniformly calibrate the point cloud coordinate system and the image coordinate system. The preprocessing and calibration module comprises the following steps: Based on multimodal data packets, thermal infrared images and multi-view visible light images are acquired. Spatial alignment and feature fusion are performed using an algorithm based on a shiftable convolutional network. This shiftable convolutional network consists of 8 convolutional layers and 3 pooling layers. The first 4 convolutional layers use fixed 3×3 kernels, and the last 4 convolutional layers use shiftable 5×5 kernels. The offset is dynamically adjusted based on image pixel gradients and edge features using an adaptive learning algorithm. The network is trained using a fusion dataset of a public dataset containing 100,000 sets of multispectral images of highways under different weather and lighting conditions and a self-built labeled dataset. The loss function is a weighted combination of cross-entropy loss and mean squared error loss, with weight ratios of 0.6 and 0.4, respectively. Finally, a multispectral fused image is obtained. The three-dimensional point cloud data of the LiDAR was acquired and processed by the Euclidean clustering algorithm. The distance threshold set by the algorithm was 0.3 meters to 0.8 meters. The minimum number of points in the point cloud cluster was set to 50. Points that were more than 0.8 meters away from the nearest cluster center or that were isolated and had no other point cloud data within a 0.5-meter radius were identified as discrete noise points and filtered out. Independent vehicle target point cloud clusters were segmented to obtain the denoised point cloud clusters. A world coordinate system was established, with the physical coordinates of the roadside milestone at the entrance of the monitoring area as the origin. The x-axis extended along the road's forward direction, the y-axis was perpendicular to the road plane and pointing upwards, and the z-axis was perpendicular to the road's forward direction and pointing outwards. A black and white checkerboard calibration board with dimensions of 1.2m × 0.8m and a grid size of 0.1m was used as the calibration object in the joint calibration process. The calibration board was placed in 20 different positions and angles within the monitoring area. Calibration data under each attitude was collected simultaneously by the lidar and camera. The camera intrinsic parameter matrix was calculated using the Zhang Zhengyou calibration method. The rotation matrix and translation vector of the lidar to the world coordinate system, as well as the rotation matrix and translation vector of the camera to the world coordinate system, were solved using the singular value decomposition algorithm. Multiply the coordinates of points in the point cloud cluster by the inverse of the rotation matrix of the laser radar to the world coordinate system, and then superimpose the translation vector to obtain the coordinates of the point cloud in the world coordinate system. The image pixel coordinates are converted to camera coordinates using the camera intrinsic parameter matrix, then multiplied by the inverse of the rotation matrix from the camera to the world coordinate system, and the translation vector is superimposed to obtain the coordinates of the corresponding points in the world coordinate system. The multispectral fused image, the denoised point cloud clusters, and their coordinate correspondence in the world coordinate system are integrated to obtain the calibrated fused data.

[0020] The target recognition module is used to process the calibrated fused data based on the target detection algorithm, and to identify and bind the vehicle's license plate, vehicle type and axle number information; The target recognition module identifies vehicle attributes using the following specific method: Based on the fusion data output by the preprocessing and calibration modules, a multispectral fused image and a denoised vehicle point cloud cluster are obtained. The target detection model is set as the YOLOv8 single detector model. Its pre-training weights are derived from the optimized weights obtained by initial training on the COCO general target detection dataset and then transfer training on a labeled dataset containing 50,000 highway vehicle-specific images. The network structure includes a backbone feature extraction network, a neck feature fusion network, and a head detection output network. The multispectral fused image is input into the target detection model. Through forward propagation calculation, the license plate region in the image is located and its character sequence is identified, and the license plate recognition result is output. Simultaneously, a 64-dimensional texture feature vector is extracted from the multispectral fused image through a convolutional neural network, and a 64-dimensional geometric feature vector is extracted from the corresponding vehicle's laser point cloud cluster through a PointNet network. The two types of feature vectors are fused at the feature level using an element-wise addition method to generate a 128-dimensional joint feature vector. The classifier uses a fully connected neural network with three hidden layers, with 256, 128, and 64 neurons in the hidden layers, respectively. The ReLU function is used as the activation function, and the stochastic gradient descent algorithm is used during training. The initial learning rate is set to 0.001, decays by 10% every 10 training cycles, and is iterated for 50 cycles until the loss function converges. The classifier calculates the matching degree between the joint feature and the preset vehicle type and axle number categories, and outputs the vehicle type and axle number recognition results. By integrating the license plate recognition results, vehicle model recognition results, and axle count recognition results, a set of attribute information for the vehicle is obtained.

[0021] The specific method for binding the vehicle's license plate, vehicle type, and axle count information to the target recognition module is as follows: Based on the obtained set of attribute information of the vehicle, obtain the license plate, vehicle type, number of axles and appearance image data of all vehicles detected at the same time; Set an identity binding benchmark, using the vehicle detection box ID and spatial location output by the target detection model as the initial association basis; Calculate the association between various attribute information under the same detection box ID and assign them to the same temporary vehicle identifier; merge all attribute information assigned to the same temporary identifier to generate a unique vehicle digital file containing license plate, vehicle type, number of axles and feature image, thus completing the binding of multi-dimensional information with a single vehicle entity.

[0022] The trajectory tracking module is used to predict and match the vehicle's state in consecutive frames based on the vehicle identification information and millimeter-wave radar data using a multi-target tracking algorithm, and generate a continuous driving trajectory that includes position, speed and lane. The trajectory tracking module predicts the vehicle motion state using the following method: Based on the vehicle digital archives and historical trajectory sequences output by the target recognition module, the world coordinate system position and instantaneous speed of each vehicle at the previous moment are obtained. The state vector of the Kalman filter is set to 6-dimensional, including x-coordinate, y-coordinate, z-coordinate, x-direction velocity, y-direction velocity, and z-direction velocity. The covariance matrix is ​​a 6×6 diagonal matrix. The state transition matrix is ​​constructed based on the constant velocity motion model and is in the form of a block matrix composed of an identity matrix and a time interval. The observation matrix is ​​a 3×6 matrix, and only the x, y, and z coordinates are observed. The previous world coordinate system position and instantaneous velocity are input into the Kalman filter, and the prior estimate and covariance of the vehicle position at the current moment are obtained by calculating through the state transition matrix. By integrating prior estimated position, estimated velocity and corresponding covariance, the predicted state set of all tracked vehicles in the current frame is obtained. The process noise covariance is determined as a diagonal matrix through offline statistical analysis of highway vehicle motion data, with diagonal elements of 0.01, 0.01, 0.01, 0.1, 0.1 and 0.1 respectively. The observation noise covariance is set as a diagonal matrix based on the sensor measurement accuracy, with diagonal elements of 0.005.

[0023] The trajectory tracking module uses the following method for multi-target matching and trajectory generation: Based on the predicted state set and the new target set output by the target recognition module in the current frame, the matching cost is calculated through the matching cost function. The mathematical expression of the matching cost function is Cost = 0.6×(1 - cosθ) +0.4×(d / 5.0), where cosθ is the cosine similarity of the image feature vector, d is the Euclidean distance between the predicted position and the center of the actual detection box, and 5.0 is the preset maximum allowed matching distance. The predicted state set and the new detection target set are input into the Hungarian algorithm. A cost matrix is ​​constructed using the Cost values ​​between all predictions and detections. The threshold condition for successful matching is that the Cost value is less than 0.3. If the Cost value exceeds this threshold, it is determined as a non-match. The state of the successfully matched detected target is fused with the speed and lane information measured in real time by the millimeter-wave radar, the state of the Kalman filter is updated, and the trajectory points containing global coordinates, instantaneous speed, acceleration and lane number are output. The continuously output trajectory points are integrated in time series to generate a complete and smooth continuous driving trajectory for each vehicle.

[0024] The analysis and early warning module is used to compare the driving trajectory with preset rules, identify traffic violations and abnormal events, and generate early warning information; The preset rule is as follows: Obtain the continuous driving trajectory of the vehicle output by the trajectory tracking module, and extract the global coordinates, instantaneous speed, acceleration and lane number sequence of each vehicle over time; A traffic rule model is set up, and predefined abnormal behavior judgment rules are loaded, including the coordinate range of solid lane lines, the permissible driving direction vector, the stopping time threshold, the minimum speed threshold, and the safe following distance threshold. The coordinate range of solid lane lines is determined by obtaining the original data by scanning road markings with LiDAR and then calibrating the coordinates in conjunction with road design drawings. The stopping time threshold is set to 3 seconds, the minimum speed threshold is set to 50% of the road section speed limit, and the safe following distance threshold is calculated according to the formula D = v×0.8 + v² / (2×0.7×9.8), where v is the instantaneous speed of the vehicle, 0.8 is the driver's reaction time, 0.7 is the road adhesion coefficient, and 9.8 is the gravitational acceleration.

[0025] The specific steps for generating the early warning information are as follows: The vehicle's real-time coordinates are compared with the coordinate range of the solid line of the lane. If the vehicle's trajectory point continuously crosses the coordinate range of the solid line for more than 0.5 seconds, it is judged as a lane change behavior that crosses the solid line. By analyzing whether the vehicle turns on its turn signal and whether the lane change speed is stable during the lane change process, normal lane change and illegal lane change can be distinguished. Calculate the dot product of the vehicle's driving direction vector and the lane's permitted driving direction vector. If the dot product is negative, it is determined to be a reverse driving behavior. The system detects the coordinate changes of continuous trajectory points of the vehicle. If the change is consistently lower than a set threshold and the duration exceeds the stop time threshold, it is determined to be an abnormal parking behavior. Compare the instantaneous speed of the vehicle with the minimum speed threshold of the road segment. If the speed is consistently below the threshold and the average speed of multiple vehicles in the same lane is detected to determine that it is not a period of traffic congestion, then it is determined to be driving below the speed limit. The interference of special situations such as emergency avoidance is eliminated by analyzing the rate of change of vehicle acceleration. Calculate the real-time coordinate difference between the vehicle in front and the vehicle behind in the same lane. If the distance is consistently less than the safe following distance threshold and both vehicles are traveling at speeds higher than the set value, it is determined to be a high-speed close following behavior. By associating all judgment results with the corresponding vehicle's digital file, image data from 3 seconds before and after the occurrence of abnormal behavior is extracted. The image resolution is set to 1920×1080. Multi-view panoramic images are synthesized using image stitching technology as evidence images, generating a structured early warning event that includes vehicle identity, behavior type, timestamp, location coordinates, and evidence images.

[0026] The output application module is used to output vehicle image data, structured trajectory data, early warning reports and visualization information, and to connect with the toll collection system and traffic management platform. Image data is stored and transmitted using the H.265 compression standard. Structured trajectory data includes fields such as vehicle ID, timestamp, x-coordinate, y-coordinate, z-coordinate, speed, acceleration, lane number, and driving direction. The data types are string, timestamp type, floating point, floating point, floating point, floating point, floating point, integer, and floating point, respectively. Visualization information is achieved by overlaying the trajectory data onto a high-precision electronic map provided by the Gaode Map API, using different colors to distinguish different vehicle trajectories, and supporting trajectory playback and real-time dynamic refresh. The system interfaces with toll collection systems and traffic management platforms using a RESTful API protocol. Data transmission is encrypted using SSL / TLS and verified for data integrity using the MD5 hash algorithm, ensuring secure and reliable data transmission.

[0027] The hardware selection requirements for the entire system are as follows: The lidar should be a 128-line lidar with a detection range of not less than 200 meters, an angular resolution of 0.1°×0.1°, and a data output frame rate of not less than 10Hz; The infrared thermal sensor has a resolution of 640×512 pixels, a detection temperature range of -40℃ to 150℃, and a temperature measurement accuracy error of no more than ±0.5℃. The visible light camera group includes four high-definition cameras with a frame rate of no less than 30fps and a resolution of 1920×1080 pixels, supporting low-light shooting and automatic exposure adjustment. The millimeter-wave radar has a detection range of no less than 150 meters, a speed measurement range of 0-250 km / h, and a speed measurement accuracy error of no more than ±1 km / h. The computing device uses a GPU server equipped with four NVIDIA A100 graphics cards, an Intel Xeon Platinum 8470C CPU, 128 GB of memory, and a storage capacity of no less than 20 TB.

[0028] The software system adopts a microservice architecture, divided into data acquisition, preprocessing, identification, tracking, early warning, and output services. These services interact via the gRPC communication protocol, and the data flow format is JSON. Adaptive measures for the system under low visibility conditions, such as severe weather or nighttime, include: an adaptive weighted fusion algorithm for sensor data compensation, dynamically adjusting the weights of each sensor's data based on weather conditions; enhanced weighting of infrared thermal imager data in foggy scenarios; and improved exposure compensation and gain adjustment of the visible light camera in nighttime scenarios. The identification model employs transfer learning, fine-tuned on low-visibility datasets, optimizing the feature extraction network to improve identification performance in complex environments.

[0029] The system's testing and verification methods are as follows: Tests are conducted in multi-lane scenarios with varying traffic volumes, covering 2 to 8 lanes, with traffic volumes ranging from 500 pcu / h to 3000 pcu / h. Recognition accuracy evaluation indicators include a license plate recognition accuracy of no less than 99.5%, a vehicle type recognition accuracy of no less than 98%, and an axle count recognition accuracy of no less than 97%. Trajectory tracking accuracy evaluation indicators are a position error of no more than 0.5 meters and a speed error of no more than 1 km / h. The false alarm rate for warning information is no more than 0.5%, and the missed alarm rate is no more than 0.1%. The testing process combines real-world testing with simulation testing to ensure the system operates stably and reliably in various application scenarios.

[0030] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dynamic vehicle behavior trajectory tracking and monitoring system for highways, characterized in that, include: The multi-source data acquisition module is used to respond to the trigger signal of a vehicle entering the monitoring area and acquire multi-source data, including: three-dimensional point cloud data, thermal infrared images, multi-view visible light images, vehicle speed and lane data, and license plate images. The preprocessing and calibration module is used to register and fuse the thermal infrared image and the visible light image, cluster and denoise the three-dimensional point cloud data, and uniformly calibrate the point cloud coordinate system and the image coordinate system. The target recognition module is used to process the calibrated fused data based on the target detection algorithm, and to identify and bind the vehicle's license plate, vehicle type and axle number information; The trajectory tracking module is used to predict and match the vehicle's state in consecutive frames based on the vehicle identification information and millimeter-wave radar data using a multi-target tracking algorithm, and generate a continuous driving trajectory that includes position, speed and lane. The analysis and early warning module is used to compare the driving trajectory with preset rules, identify traffic violations and abnormal events, and generate early warning information; The output application module is used to output vehicle image data, structured trajectory data, early warning reports and visualization information, and to connect with the toll collection system and traffic management platform.

2. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 1, characterized in that, The specific steps for collecting multi-source data in response to the trigger signal of a vehicle entering the monitoring area are as follows: Based on the trigger detection equipment deployed on the roadside, it monitors in real time whether vehicles enter the monitoring area; When a vehicle is detected entering, a trigger signal is generated and sent to the multi-source data acquisition module; The multi-source data acquisition module parses the trigger signal and generates a unified acquisition command. The acquisition command is broadcast to all data acquisition units to begin acquiring multi-source data.

3. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 2, characterized in that, Based on the unified acquisition command received by the multi-source data acquisition module, all data acquisition units are started synchronously, controlling the lidar, infrared thermal sensor, visible light camera group, millimeter-wave radar and license plate recognition device to start data acquisition at the same time. A unified timestamp is applied to the raw data streams collected by each sensor, so that the 3D point cloud, thermal infrared image, multi-view visible light image, vehicle speed and lane data and license plate image are aligned in time. When the vehicle completely leaves the monitoring area, the multi-source data acquisition module sends a stop command to all data acquisition units to end the current acquisition task. The data acquisition process is completed by integrating the time-stamped raw data generated by each sensor during the acquisition process to form a time-aligned multimodal data packet.

4. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 3, characterized in that, The preprocessing and calibration module comprises the following steps: Based on multimodal data packets, thermal infrared images and multi-view visible light images are acquired. An algorithm based on offset convolutional networks is used to spatially align and fuse their features to obtain a multispectral fused image. Acquire 3D point cloud data from LiDAR, process it using Euclidean clustering algorithm, segment out independent vehicle target point cloud clusters and filter out discrete noise points to obtain denoised point cloud clusters; A world coordinate system is established, and the rotation matrix and translation vector of the lidar to the world coordinate system, as well as the rotation matrix and translation vector of the camera to the world coordinate system, are obtained through joint calibration. Multiply the coordinates of points in the point cloud cluster by the inverse of the rotation matrix of the laser radar to the world coordinate system, and then superimpose the translation vector to obtain the coordinates of the point cloud in the world coordinate system. The image pixel coordinates are converted to coordinates in the camera coordinate system using the camera intrinsic parameter matrix, then multiplied by the inverse of the rotation matrix from the camera to the world coordinate system, and the translation vector is superimposed to obtain the coordinates of the corresponding point in the image in the world coordinate system. By integrating the multispectral fused image, the denoised point cloud clusters, and their coordinate correspondence in the world coordinate system, calibrated fused data is obtained.

5. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 4, characterized in that, The target recognition module identifies vehicle attributes using the following specific method: Based on the fusion data output by the preprocessing and calibration modules, a multispectral fused image and a denoised vehicle point cloud cluster are obtained. Define the target detection model and obtain its pre-trained weight parameters and network structure; The multispectral fused image is input into the target detection model. Through forward propagation calculation, the license plate region in the image is located and its character sequence is identified, and the license plate recognition result is output. Simultaneously, the multispectral fusion image is fused with the corresponding vehicle's laser point cloud cluster at the feature level to extract the joint features of the vehicle's contour and texture. The classifier calculates the matching degree between the joint features and the preset vehicle type and axle number categories, and outputs the vehicle type and axle number recognition results. By integrating the license plate recognition results, vehicle model recognition results, and axle count recognition results, a set of attribute information for the vehicle is obtained.

6. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 5, characterized in that, The specific method for binding the vehicle's license plate, vehicle type, and axle count information to the target recognition module is as follows: Based on the obtained set of attribute information of the vehicle, obtain the license plate, vehicle type, number of axles and appearance image data of all vehicles detected at the same time; Set an identity binding benchmark, using the vehicle detection box ID and spatial location output by the target detection model as the initial association basis; Calculate the association between various attribute information under the same detection box ID and assign them to the same temporary vehicle identifier; All attribute information assigned to the same temporary identifier is merged to generate a unique vehicle digital profile containing license plate, vehicle type, number of axles, and feature image, thus completing the binding of multi-dimensional information to a single vehicle entity.

7. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 6, characterized in that, The trajectory tracking module predicts the vehicle motion state using the following method: Based on the vehicle digital archives and historical trajectory sequences output by the target recognition module, the world coordinate system position and instantaneous speed of each vehicle at the previous moment are obtained. Define the state vector and covariance matrix of the Kalman filter, and obtain its state transition matrix and observation matrix; The previous world coordinate system position and instantaneous velocity are input into the Kalman filter, and the prior estimate and covariance of the vehicle position at the current moment are obtained by calculating through the state transition matrix. By integrating prior estimated position, estimated velocity, and corresponding covariance, the predicted state set of all tracked vehicles in the current frame is obtained.

8. The highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 7, characterized in that, The trajectory tracking module uses the following method for multi-target matching and trajectory generation: Based on the predicted state set and the new detected target set output by the target recognition module in the current frame, the fusion weight based on motion features and appearance features is obtained by matching the cost function; the motion feature is the Euclidean distance between the predicted position and the actual detection box, and the appearance feature is the cosine similarity of the image feature vector. The predicted state set and the new detection target set are input into the Hungarian algorithm. The matching cost between all predictions and detections is calculated through the cost function, and the optimal matching pair is solved. The state of the successfully matched detected target is fused with the speed and lane information measured in real time by the millimeter-wave radar, the state of the Kalman filter is updated, and the trajectory points containing global coordinates, instantaneous speed, acceleration and lane number are output. By integrating the continuously output trajectory points according to the time series, a complete and smooth continuous driving trajectory for each vehicle is generated.

9. A highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 8, characterized in that, The preset rule is as follows: Obtain the continuous driving trajectory of the vehicle output by the trajectory tracking module, and extract the global coordinates, instantaneous speed, acceleration and lane number sequence of each vehicle over time; Set up a traffic rule model and load predefined abnormal behavior judgment rules, including lane solid line coordinate range, allowed driving direction vector, stopping time threshold, minimum speed threshold and safe following distance threshold.

10. A highway dynamic vehicle behavior trajectory tracking and sensing monitoring system according to claim 9, characterized in that, The specific steps for generating the early warning information are as follows: The vehicle's real-time coordinates are compared with the coordinate range of the solid line of the lane. If the vehicle's trajectory point continuously crosses the coordinate range of the solid line, it is determined to be a lane change behavior that crosses the solid line. Calculate the dot product of the vehicle's driving direction vector and the lane's permitted driving direction vector. If the dot product is negative, it is determined to be a reverse driving behavior. The system detects the coordinate changes of continuous trajectory points of the vehicle. If the change is consistently lower than a set threshold and the duration exceeds the stop time threshold, it is determined to be an abnormal parking behavior. Compare the vehicle's instantaneous speed with the minimum speed threshold of the road segment. If the speed is consistently below the threshold and it is not during a period of traffic congestion, it is determined to be driving below the speed limit. Calculate the real-time coordinate difference between the vehicle in front and the vehicle behind in the same lane. If the distance is consistently less than the safe following distance threshold and both vehicles are traveling at speeds higher than the set value, it is determined to be a high-speed close following behavior. By associating all judgment results with the corresponding vehicle's digital file, extracting before-and-after image evidence of the abnormal behavior, and generating a structured early warning event containing vehicle identity, behavior type, timestamp, location coordinates, and evidence images.

Images