High-speed traffic flow detection method and system based on deep learning

Abstract

The invention relates to the technical field of intelligent traffic, and discloses a deep learning-based expressway traffic flow detection method and system, and the method comprises the steps: obtaining multi-modal traffic data and environment data in real time from each key road section of an expressway, and carrying out the preprocessing of the multi-modal traffic data obtained in real time, including the time-space alignment of the multi-modal traffic data; inputting the preprocessed multi-modal traffic data and environment data into a pre-trained deep fusion network model, and outputting various vehicle parameters for representing expressway traffic flow characteristics; inputting the various vehicle parameters into a pre-trained high-speed traffic flow detection model, and outputting traffic flow grade information; dynamically updating the deep fusion network model at the key road section by adopting a model fusion mode according to the vehicle parameter change condition at each key road section within the preset time; and updating the high-speed traffic flow detection model according to external conditions introduced by the traffic flow grade information in actual use. And the accuracy and real-time performance of high-speed traffic flow detection are improved by combining deep learning.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a high-speed vehicle flow detection method and system based on deep learning. Background Art

[0002] In the modern transportation system, highway traffic flow monitoring is extremely important. It is not only related to the real-time control of traffic conditions by the traffic management department so that timely diversion measures can be taken, but also affects the driver's travel planning, helping them choose the best route and save travel time. However, the existing highway traffic flow detection methods have many shortcomings.

[0003] Traditional detection technologies, such as geomagnetic induction coil detection, require laying a large number of induction coils under the road surface. This is not only difficult and costly to construct, but also extremely inconvenient to maintain in the later stage. Once the coil fails, the maintenance process will cause serious interference to the normal traffic on the highway. At the same time, the detection accuracy of the geomagnetic induction coil is easily affected by the road conditions, such as road deformation, water accumulation, etc., which may cause deviations in the detection results. Although microwave detection technology is relatively easy to install, its detection effect is greatly reduced in complex high-speed traffic environments. In multi-lane and densely trafficked sections, microwave signals are prone to interfere with each other, resulting in frequent misjudgments and missed judgments. Moreover, in severe weather conditions, such as heavy rain, fog, and heavy snow, the propagation of microwaves is hindered and the detection accuracy will be significantly reduced.

[0004] Video-based traffic flow detection methods have received widespread attention in recent years. However, most existing video-based detection algorithms directly apply general object detection networks. In high-speed scenarios, vehicles travel at extremely high speeds, which places extremely high demands on the real-time performance of the algorithm. At the same time, vehicles traveling at high speeds are prone to motion blur and large scale changes in video images. In addition, changes in lighting conditions, such as strong light reflections on sunny days, glare from car lights at night, and different weather conditions, can seriously affect the accuracy of detection. In addition, general object detection networks usually have complex structures, large parameters, and high computing resource requirements. They are difficult to implement real-time detection on edge devices and cannot meet the actual needs of high-speed traffic flow detection. Summary of the invention

[0005] The present invention aims to provide a high-speed vehicle flow detection method and system based on deep learning. By optimizing the deep learning model, improving the data processing process and innovating the system architecture, the accuracy and real-time performance of high-speed vehicle flow detection can be improved, the consumption of computing resources can be reduced, and the complex environmental factors in high-speed scenarios can be effectively responded to, providing reliable support for traffic management and public travel.

[0006] The basic solution provided by the present invention is: a high-speed vehicle flow detection method based on deep learning, the method comprising: Acquire multimodal traffic data and environmental data in real time from key sections of highways, where multimodal traffic data includes vehicle video images, vehicle kinematic data, and vehicle 3D point cloud data; Preprocessing the multimodal traffic data acquired in real time, wherein the preprocessing includes the spatiotemporal alignment of the multimodal traffic data; The pre-processed multi-modal traffic data and environmental data are input into the pre-trained deep fusion network model to perform vehicle detection, tracking and traffic flow statistics, and output a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow; Input various vehicle parameters into a pre-trained high-speed traffic flow detection model and output traffic flow level information; According to the changes in vehicle parameters at each key section within the preset time, the deep fusion network model at the key section is dynamically updated by model fusion; the high-speed traffic flow detection model is updated according to the external conditions introduced by the traffic flow level information during actual use; The deep fusion network model at the key section is dynamically updated by using model fusion. At preset intervals, multiple vehicle parameters of each key section of the highway are compared separately; data analysis algorithms are used to determine whether vehicle parameters have changed significantly; when changes in vehicle parameters at the key section are detected, the update interval of the multimodal traffic data at the key section is adjusted, and the newly collected multimodal traffic data is gradually integrated into the training data set using online learning, the model is updated and a new model is obtained; by comparing the performance of the new model and the original model on the verification data set, the weights of the new model and the original model for fusing and outputting multiple vehicle parameters are determined; in the online learning process of the model, only part of the newly collected multimodal traffic data is integrated into the training set each time, and part of the model parameters are updated.

[0007] The present invention implements a high-speed vehicle flow detection method based on deep learning, and also provides a high-speed vehicle flow detection system based on deep learning, the system comprising: A data acquisition module, used to acquire multimodal traffic data and environmental data from each key stage of the expressway, wherein the multimodal traffic data includes vehicle video images, vehicle kinematic data and vehicle three-dimensional point cloud data; A data processing module, used for preprocessing the acquired multimodal traffic data, wherein the preprocessing includes time-space alignment of the multimodal traffic data; Model building module, used to build deep fusion network model and traffic flow detection model; An execution module is used to input the pre-processed multi-modal traffic data and environmental data into a pre-trained deep fusion network model to perform vehicle detection, tracking and traffic flow statistics, and output a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow at corresponding key sections; it is also used to input a variety of vehicle parameters into a pre-trained traffic flow detection model to output traffic flow level information at corresponding key sections; The update module is used to dynamically update the deep fusion network model at each key section by model fusion according to the changes in vehicle parameters at the key section within a preset time; it is also used to update the high-speed traffic flow detection model according to the changes in traffic flow level information during actual use.

[0008] The working principle and advantages of the present invention are: 1. The present invention aims to comprehensively and accurately obtain highway traffic information by constructing a multimodal data collection system, thereby significantly improving the accuracy and reliability of vehicle flow detection. The system integrates vehicle video images, vehicle kinematic data and vehicle three-dimensional point cloud data. The system can more comprehensively capture the dynamic characteristics of traffic flow and provide a solid foundation for subsequent analysis and decision-making. 2. In the data preprocessing stage, special attention is paid to the spatiotemporal alignment of multimodal traffic data. This process is crucial to eliminating data inconsistencies caused by time delays or position deviations between sensors. It not only improves the data quality, but also makes subsequent feature extraction and pattern recognition more accurate.

[0009] 3. Based on the ResNet and Transformer architecture, through the data fusion input layer, feature extraction layer, multi-scale feature interaction module, spatiotemporal attention module and prediction output layer, it has a simple structure and small amount of calculation. At the same time, it can better adapt to the complex traffic scenes of highways, effectively extract vehicle features, and synchronously output multiple traffic parameters to provide richer data support for traffic management. In the process of model training and optimization, the corresponding technology is adopted to accelerate the convergence speed of the model, prevent the model from overfitting, and improve the generalization ability and detection accuracy of the model. 4. In the process of determining the traffic level information, the present invention not only relies on the basic parameters of the vehicle (such as vehicle type, speed, etc.), but also introduces external conditions (such as weather conditions, holidays, special events, etc.). These external factors have a significant impact on traffic flow, so taking them into consideration can make the judgment of traffic level information more consistent with the current real traffic situation. In this way, traffic level information can not only reflect the real-time traffic status, but also assist in making more accurate traffic control decisions and improve road traffic efficiency.

[0010] 5. Taking the idea of ​​global traffic and local sections, the present invention firstly uniformly constructs a basic model for all sections of the entire traffic. However, considering the specific characteristics of different sections, a dynamic adjustment mechanism based on the data update and fusion model of each key section is designed, so that the corresponding model of each key section can adapt to the changes in the traffic flow of the section in real time, and perform self-fine-tuning of the model of each key section. In this way, it is possible to effectively cope with the local volatility of traffic flow, maintain a high detection accuracy of each key section, and not affect the operation of other sections and the entire system. Through continuous iterative improvements, efficient operation in the short term is guaranteed, and a solid foundation is laid for long-term stable operation, providing strong support for intelligent traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a flow chart of a high-speed vehicle flow detection method based on deep learning provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a deep fusion network model provided by an embodiment of the present invention; Figure 3 A schematic diagram of a dynamic adjustment process of a deep fusion network model provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a high-speed vehicle flow detection system based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following is a further detailed description through specific implementation methods: The embodiment is basically as shown in the attached Figure 1 As shown: A high-speed vehicle flow detection method based on deep learning, the method includes: Multimodal traffic data and environmental data are acquired in real time from key sections of the highway, wherein the multimodal traffic data includes vehicle video images, vehicle kinematic data and vehicle three-dimensional point cloud data.

[0013] Specifically, in key sections of highways, such as near traffic hubs, accident-prone sections, intersections of sections, ramp entrances and exits, etc., high-definition cameras, millimeter-wave radars, laser radars and other multimodal sensors are reasonably installed according to the terrain and traffic characteristics of the road. High-definition cameras are used to obtain vehicle video images. Products with high resolution and stable frame rate are selected. They can be installed on the gantry above the road to ensure that their field of view can cover all lanes, and the shooting angle is precisely adjusted to clearly capture the appearance characteristics, license plate number and other information of the vehicle. Millimeter-wave radars are used to obtain vehicle kinematic data, including vehicle speed, distance and angle information, and can be installed near the edge of the lane. The installation position and angle of the laser radar are precisely calibrated to ensure that the three-dimensional point cloud data of the vehicle can be accurately obtained, and the shape and position of the vehicle can be described in detail. At the same time, environmental monitoring equipment such as light sensors and weather stations are equipped to collect environmental data such as light intensity and weather conditions (sunny, rainy, snowy, foggy, etc.) in real time.

[0014] Develop a detailed data collection plan, and continuously collect multimodal traffic data according to different time periods (peak and off-peak hours on weekdays, and different time periods on weekends and holidays), different environmental data (weather conditions and light intensity), and different road sections. For example, on weekdays, data is collected during the morning peak (7:00-9:00), evening peak (17:00-19:00), and off-peak hours (10:00-16:00); on weekends and holidays, select representative time periods for collection based on different travel patterns. In terms of weather conditions, multimodal data is continuously collected in sunny, rainy, snowy, foggy and other weather conditions, and the data collected under each weather condition is not less than 12 hours. In terms of light intensity, data is collected under strong light (direct sunlight), weak light (cloudy or evening) during the day, and at night with lighting (good street lighting) and without lighting (street lamp failure or remote road sections).

[0015] The multimodal traffic data acquired in real time is preprocessed, wherein the preprocessing includes the spatiotemporal alignment of the multimodal traffic data.

[0016] Specifically, for vehicle video images, video processing software is used to sample at a preset frame rate, such as 25 frames per second, to obtain an image sequence. The bilateral filtering algorithm is used to denoise the image, and the denoising effect and edge preservation effect are balanced by adjusting the filtering parameters. The histogram equalization technology is used to enhance the contrast of the image and improve the clarity of the image. The image pixel values ​​are mapped to the [0, 1] interval for normalization processing to make the features between different images comparable. For the vehicle kinematic data and vehicle 3D point cloud data collected by millimeter wave radar and lidar, filtering and denoising are performed to remove abnormal data points and coordinate conversion. Finally, the vehicle video image, vehicle kinematic data, and vehicle 3D point cloud data are synchronized in time and unified in spatial coordinates to ensure the consistency of the fusion of radar acquired data and vehicle video image data in time and space, which is convenient for subsequent comprehensive analysis. The preprocessed multimodal traffic data is input into a pre-trained deep fusion network model, which outputs a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow at the corresponding key sections.

[0017] Specifically, a deep fusion network model is used for vehicle detection, tracking and traffic flow statistics.

[0018] like Figure 2 As shown in the figure, based on the architecture combining ResNet and Transformer, a deep fusion network model (HS-DFNet model) is constructed. The deep fusion network model includes a data fusion input layer, a feature extraction layer, a multi-scale feature interaction module, a spatiotemporal attention module and a prediction output layer. It can better adapt to the complex traffic scenes of highways, effectively extract the features that characterize the highway traffic flow, and output a variety of vehicle parameters at the same time.

[0019] Among them, in the data fusion input layer, a specific fusion convolution layer is designed to extract and fuse features of image data, millimeter-wave radar data, and lidar data. First, for image data, traditional convolution operations are used to extract its visual features, such as the appearance, color, shape, and other information of the vehicle. At the same time, for millimeter-wave radar data, since its data format may be one-dimensional distance-speed data, it is converted into a two-dimensional feature representation (for example, radar data of different time steps are arranged in a two-dimensional matrix), and convolution operations are used to extract features related to vehicle motion, such as the speed change trend, distance distribution, and angle change trend of the vehicle. For lidar data, it is usually presented in the form of a three-dimensional point cloud, which is projected onto a two-dimensional plane, and features are extracted using a deep convolutional neural network, including the three-dimensional contour information and position distribution features of the vehicle. The fusion convolution layer adopts a parallel convolution structure to perform preliminary feature extraction on the three types of data respectively, and then fuses the features from different sources through feature splicing methods such as concatenation or addition, so that the network can simultaneously consider the complementary information of different sensor data.

[0020] The feature extraction layer uses a multi-layer ResNet module, and optimizes the feature extraction capability by adjusting the number of network layers, convolution kernel size and weight. The multi-layer ResNet module plays an important role in the feature extraction process. In traffic flow detection, for shallow ResNet modules, smaller convolution kernels (such as 3x3) are used to extract local detail features of vehicles, such as features of license plates, windows and other parts, which help to distinguish different types of vehicles. As the number of network layers increases, larger convolution kernels (such as 5x5 or 7x7) are gradually used to expand the receptive field and extract more macro features, such as the overall shape of the vehicle, the characteristics of the vehicle queue, etc. At the same time, the residual connection of the ResNet module can effectively solve the gradient disappearance problem, so that the network can be stably trained when the number of layers is deepened. In this application scenario, as the network deepens, more complex feature representations of vehicles and traffic scenes can be learned, such as vehicle occlusion, vehicle appearance characteristics under different lighting conditions, and the impact of different weather (sunny, rainy, foggy) on vehicle appearance.

[0021] The multi-scale feature interaction module uses pyramid pooling and dilated convolution techniques to obtain feature maps of different scales. The use of pyramid pooling and dilated convolution techniques enables the model to obtain feature maps of different scales, which is crucial for traffic flow detection. Pyramid pooling allows the network to observe traffic scenes from different spatial scales. In traffic flow detection, information at different scales is very important. At the local scale, the information of a single vehicle can be detected, such as the specific location and category of a vehicle; at the medium scale, the spacing between vehicles and lane information can be observed; at the large scale, the traffic flow of the entire road can be considered, such as the degree of road congestion, the gathering area of ​​vehicles, etc. Dilated convolution can expand the receptive field without increasing the size of the convolution kernel, which helps to better understand the spatial layout of vehicles without losing detailed information. For example, it can help accurately judge the distribution of vehicles on the road and the relative position relationship between vehicles. Especially in the case of dense vehicles, it can better handle the occlusion problem between vehicles and identify partially occluded vehicles.

[0022] The spatiotemporal attention module combines time series information and spatial position information, and through the spatiotemporal attention mechanism, allows the model to automatically focus on the vehicle target. The spatiotemporal attention module combines time series information and spatial position information, and is very effective for dynamic information processing in traffic flow detection. In terms of space, through the spatial attention mechanism, the model automatically focuses on the vehicle target instead of background information. For example, in a complex road scene, there are many elements such as vehicles, pedestrians, and road signs. Spatial attention can guide the network to focus on the location of the vehicle and exclude other interfering elements. In terms of time, combining time series information enables the model to better understand the vehicle's motion trajectory and speed changes. By processing multiple consecutive frames of images and radar data, the vehicle's driving direction and speed can be predicted. For example, by observing the position changes of a vehicle in several consecutive frames, its speed and driving direction can be accurately calculated, and abnormal behaviors such as sudden lane changes and sudden braking can be detected.

[0023] The prediction output layer uses the convolutional layer to predict the vehicle's location, category, speed, and direction of travel based on the fused feature map. For the prediction of the vehicle's location, the output can be the coordinates relative to the image or radar detection range to accurately locate the vehicle on the road, facilitating subsequent traffic monitoring and management. For the vehicle category, the prediction of the vehicle is different types such as cars, trucks, buses, motorcycles, etc., providing a basis for detailed statistics of traffic flow. The prediction of vehicle speed and direction of travel helps traffic management departments to monitor and control road traffic flow in real time. For example, based on the vehicle's speed and direction of travel, it can be predicted whether there will be traffic congestion or whether the time of the traffic light needs to be adjusted.

[0024] The comprehensive use of data from multiple sensors avoids the limitations of a single sensor. In practical applications, it can be deployed in intelligent transportation systems to accurately detect and monitor traffic flow on different types of highways.

[0025] Conduct model training, obtain multimodal traffic data for training in different time periods, different environmental data and different road sections, and use data enhancement strategies to expand the data and construct a training data set; in the training data set, the vehicle video images are annotated based on the vehicle kinematic data and vehicle three-dimensional point cloud data, and the annotation content is related to the output vehicle parameters.

[0026] Specifically, since the vehicle speed in the high-speed traffic flow detection application environment targeted by this solution is relatively high, it meets the highway driving speed requirements and is different from the ordinary urban traffic flow detection statistics. Therefore, in addition to the existing vehicle video images, vehicle kinematic data and vehicle 3D point cloud data, environmental data (such as visibility, rain and snow conditions, wind speed, etc.) are also included. These factors will significantly affect the high-speed driving conditions and traffic flow distribution. For example, heavy rain or dense fog may cause a decrease in traffic flow on some sections of the road, or a decrease in vehicle speed.

[0027] In the time dimension, more refined sampling data is obtained, such as increasing the sampling rate from once every minute to once every second, in order to capture the instantaneous changes in traffic flow on the highway. In the spatial dimension, the monitoring of highway sections is more refined, for example, the original monitoring in kilometers is refined to units of hundreds of meters, which helps to more accurately analyze the traffic flow characteristics of different sections.

[0028] The data enhancement strategy is to rotate, scale and crop the vehicle video images in the training data set to obtain a number of random vehicle video images, and add noise and data jitter to the vehicle kinematic data to obtain a number of random vehicle kinematic data. For example, the image is randomly rotated (rotation angle range is - 10° to 10°), scaled (scaling ratio range is 0.8 - 1.2), cropped (cropping ratio range is 0.7 - 0.9), and the radar data is added with noise (Gaussian noise, salt and pepper noise, etc.), data jitter (randomly adjust data values ​​within a certain range) and other operations to expand the diversity of data and enhance the generalization ability of the model.

[0029] Labeling: You can use professional labeling tools (such as LabelBox) to accurately label image data, marking the vehicle's position (bounding box coordinates), vehicle type (car, bus, truck, special vehicle, etc.), driving direction (forward, reverse), and other information related to the output vehicle parameters.

[0030] After labeling, the samples are divided into training set, validation set and test set in a ratio of 8:1:1. During the model training process, an adaptive learning rate adjustment algorithm, such as the Adagrad algorithm, is used to dynamically adjust the learning rate according to the update of the model parameters to accelerate the convergence of the model. The initial learning rate is set to 0.001. The loss function uses a weighted multi-task loss function, including target detection loss (combined with bounding box regression loss and classification loss, such as CIoULoss and cross entropy loss functions), speed prediction loss (using the mean square error loss function to calculate the difference between the predicted speed and the true speed) and direction prediction loss (by calculating the loss of the angle difference between the predicted direction and the true direction). According to the importance of the task, the weights for the bounding box regression loss, classification loss, speed prediction loss and direction prediction loss are set to 1.2, 1.0, 0.8 and 0.6 respectively. Through the back-propagation algorithm, the parameters of the model are continuously adjusted to minimize the loss function value. During the training process, the accuracy, recall, mean average precision (mAP), speed prediction error and direction prediction accuracy of the model are evaluated on the validation set every 10 epochs. Through a large amount of training data and reasonable training strategies, the model parameters are continuously optimized. When the mean average precision (mAP) of the model on the validation set reaches more than 92%, the speed prediction error is less than 5km / h, the direction prediction accuracy reaches more than 95%, and there is no obvious improvement for 5 consecutive epochs, stop training, save the trained model weights, and ensure the high performance of the model. After obtaining vehicle detection parameters, i.e., vehicle position, vehicle speed, vehicle type, and vehicle driving direction, vehicle tracking and traffic flow statistics are performed through the model. During the statistical process, a virtual detection area is set (e.g., a rectangular area is defined in the image to represent a section of the highway road), and vehicles traveling in the same direction and passing through the detection area are counted. A multi-target tracking algorithm based on Kalman filtering is used to continuously track vehicles. The Kalman filter algorithm continuously adjusts the vehicle state estimation through two steps: prediction and update. In the prediction step, the current state is predicted based on the vehicle's previous state and motion model; in the update step, the prediction result is corrected using new detection data to improve the tracking accuracy. When a vehicle enters the detection area for the first time, it is counted and the vehicle ID is recorded. In subsequent frames, the vehicle's motion trajectory and ID are matched to avoid repeated counting. At the same time, the speed information provided by the millimeter-wave radar and lidar is combined with the position change of the vehicle in the video image to further verify the vehicle's motion state and ensure the accuracy of traffic flow statistics. For example, if the radar data shows that the vehicle speed is abnormal, and the position change of the vehicle in the video image does not match the speed, the system will conduct further analysis and judgment to avoid incorrect counting. The total number of vehicles passing through the detection area per unit time can be obtained to judge the traffic flow; based on other vehicle parameters and the length of the detection area, lanes and other data, regional traffic flow information and single-lane traffic flow information can also be obtained.

[0031] In this process, the information of millimeter-wave radar and lidar in multimodal data is also used, combined with the position change of the vehicle in the video image, to more accurately estimate the actual speed of the vehicle. By processing the radar data, the speed information of the vehicle can be directly obtained. At the same time, by analyzing the position change of the vehicle in the continuous frames in the video image, combined with the time interval, the visual speed of the vehicle can be calculated. Fusion of the two can obtain a more accurate actual speed of the vehicle. For example, at a certain moment, the millimeter-wave radar measured the vehicle speed as 80km / h, and the visual speed obtained by video image analysis was 78km / h. According to the preset fusion algorithm (such as the weighted average method, weights are set according to the reliability of radar data and visual data), the actual speed of the vehicle is calculated to be 79km / h. Finally, after the preprocessed multimodal traffic data is input into the pre-trained deep fusion network model, a variety of vehicle parameters including traffic flow, vehicle location, vehicle speed, vehicle type and vehicle driving direction are output, providing richer data support for traffic management.

[0032] Input multiple traffic parameters into a pre-trained high-speed traffic flow detection model and output traffic flow level information; Specifically, based on multiple historical traffic parameters and historical traffic flow level information, as well as deep learning, a traffic flow detection model is constructed to characterize the mapping relationship between multiple traffic parameters and traffic flow level information.

[0033] Historical traffic flow level information includes standard traffic flow level divisions determined based on the design capacity of the expressway, historical traffic flow data, and the actual needs of the local traffic management department, as well as adjustment information on traffic flow level judgments based on historical external conditions.

[0034] First, the traffic flow level is determined based on the comparison between the traffic flow in various traffic parameters and the standard traffic flow level; then, it is determined whether there are external conditions. If so, the traffic flow level determined above is adjusted, and finally accurate traffic flow level information after adjustment based on external conditions is output.

[0035] Standard traffic flow level classification, for example, the conventional method is to set the first threshold (congestion level) to 300 vehicles passing every 5 minutes, the second threshold (busy level) to 200 vehicles passing every 5 minutes, the third threshold (normal level) to 100 vehicles passing every 5 minutes, and the fourth threshold (low traffic level) to 50 vehicles passing every 5 minutes. If the above result is 240 vehicles passing every 5 minutes, it is determined that the traffic flow level information is below the congestion level, above the busy level, and biased towards the busy level.

[0036] Although the threshold for dividing the traffic flow level is determined according to the requirements of relevant standards, the traffic flow level can be determined. However, when traffic control is carried out based on the determined traffic flow level, there is a dynamic adjustment of the traffic flow level identification. Such level identification adjustment is in line with the current actual traffic conditions and is conducive to accurately supporting subsequent traffic management. For example, the normal level is busy, but when a traffic accident occurs in front of the detection area, traffic management adopts the congestion level method to control the relevant sections. For example, during holidays, due to the large overall traffic volume, a certain key section may be generally congested during normal time, and it is necessary to handle traffic during holidays with Class C severe congestion. Therefore, at this time, the traffic flow level needs to be adjusted according to the introduction of external conditions, so that the confirmation of the traffic flow level information is not fixed in the standard, but is adaptive to the actual traffic conditions, which can effectively respond to the introduction of external conditions and improve the utilization value of the traffic flow level information. The model is constructed by learning the way of adjusting the traffic flow level information after the introduction of historical external conditions through deep learning technology, and the specific technology used is not limited.

[0037] At the same time, the external conditions introduced by the actual use of the traffic flow level information and the dynamic adjustment information are continuously obtained to update the high-speed traffic flow detection model, so that the output of the high-speed traffic flow level information is more accurate and more in line with the current actual traffic conditions.

[0038] like Figure 3 As shown, according to the changes in vehicle parameters at each key section within a preset time, the deep fusion network model at the key section is dynamically updated by using model fusion.

[0039] Specifically, at preset intervals, multiple vehicle parameters of key sections of the highway are compared separately. Data analysis algorithms (such as cluster analysis, trend analysis, etc.) are used to determine whether vehicle parameters have changed significantly. Vehicle parameter changes include traffic volume growth or decline exceeding the threshold requirements, and the emergence of a new peak in traffic volume. A new peak can be understood as a sudden increase in traffic volume by several times. In the online learning process of the model, an incremental learning algorithm is used to integrate only part of the newly collected multimodal traffic data into the training set each time, and to update some model parameters. The updated parameters include the weights of the convolution kernel of the feature extraction layer and the parameters of the attention mechanism to reduce the amount of calculation and training time. At the same time, in order to ensure the stability of the model during the update process, the model fusion method is used to weightedly fuse the newly trained model with the original model. By comparing the performance of the new model and the original model on the validation set, the weights of the new model and the original model for fusing and outputting multiple traffic parameters are determined, so that the updated model can adapt to the new data mode while retaining the good performance of the original model on stable data. For example, for sections with large changes in traffic volume, the data update interval is shortened by a certain period of time, such as 4-8 minutes; for sections with relatively stable traffic volume, the data update interval is appropriately extended, such as 2-5 minutes. On a certain section of highway, after data analysis, it was found that the traffic volume during the morning rush hour has continued to grow in the past week and exceeded 20%, so the data update interval of this section was shortened from the original 15 minutes to 10 minutes. In subsequent online learning, 50 newly collected data are selected each time to be added to the training set, and the weights of the convolution kernels of the feature extraction layer are fine-tuned. After multiple updates, by comparing the performance of the new model and the original model on the validation set, it is determined that the weight of the new model is 0.7 and the weight of the original model is 0.3. After fusion, the final model is obtained and applied to output various vehicle parameters of this key section.

[0040] like Figure 4 As shown, this embodiment also provides a high-speed vehicle flow detection system based on deep learning, which executes the high-speed vehicle flow detection method based on deep learning. The system includes: A data acquisition module, used to acquire multimodal traffic data and environmental data from each key stage of the expressway, wherein the multimodal traffic data includes vehicle video images, vehicle kinematic data and vehicle three-dimensional point cloud data; A data processing module, used for preprocessing the acquired multimodal traffic data, wherein the preprocessing includes time-space alignment of the multimodal traffic data; Model building module, used to build deep fusion network model and traffic flow detection model; An execution module is used to input the pre-processed multi-modal traffic data and environmental data into a pre-trained deep fusion network model to perform vehicle detection, tracking and traffic flow statistics, and output a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow at corresponding key sections; it is also used to input a variety of vehicle parameters into a pre-trained traffic flow detection model to output traffic flow level information at corresponding key sections; The update module is used to dynamically update the deep fusion network model at each key section by model fusion according to the changes in vehicle parameters at the key section within a preset time; it is also used to update the high-speed traffic flow detection model according to the changes in traffic flow level information during actual use.

[0041] It can be understood that the above system can fully execute the above method and obtain the corresponding effect. The specific process is as described in the above method and will not be repeated here.

[0042] The high-speed vehicle flow detection method and system based on deep learning provided in this embodiment can comprehensively and accurately obtain the traffic information of the highway by constructing a multimodal data collection system, thereby improving the accuracy and reliability of vehicle flow detection. Based on the ResNet and Transformer architecture, through the data fusion input layer, feature extraction layer, multi-scale feature interaction module, spatiotemporal attention module and prediction output layer, it can better adapt to the complex traffic scenes of the highway, effectively extract vehicle features, and output multiple traffic parameters at the same time, providing richer data support for traffic management. At the same time, in the process of model training and optimization, the corresponding technology is adopted to accelerate the convergence speed of the model, prevent the model from overfitting, and improve the generalization ability and detection accuracy of the model. In the process of determining the vehicle flow level information based on vehicle parameters, by introducing external conditions, the judgment of the vehicle flow level information is more in line with the current real traffic situation, which is conducive to improving the accuracy of the vehicle flow level information in assisting the designated traffic control decision. The data update and model dynamic adjustment mechanism enable the system to adapt to the changes in traffic flow in real time, maintain a high detection accuracy, and ensure its long-term stable operation by continuously improving the performance of the system.

[0043] Embodiment 2 Different from the first embodiment, a comprehensive influence coefficient I is constructed by taking into account the influence of multiple factors such as traffic environment data and historical congestion data on vehicle flow. Assuming that the influence factor of traffic environment data is W (comprehensive quantification of factors such as visibility and weather conditions), and the influence factor of historical congestion data is H (quantified according to the congestion degree of similar historical time periods and sections), then:

[0044] in, , are weight coefficients, which respectively represent the importance of environmental data and historical congestion data on traffic flow. is the correction constant.

[0045] Assume that the initial estimated value of the vehicle detection parameter group originally calculated by the model based on multimodal traffic data (vehicle video images, kinematic data, and 3D point cloud data) is F 0 , then the vehicle detection parameter group F after comprehensive consideration of multiple factors is:

[0046] In the multi-scale feature interaction module, a multi-scale feature fusion method based on the attention mechanism is adopted. Assume that the feature maps of different scales are F 1 , F 2 , F 3 …F n , the corresponding attention weight w 1 , w 2 , w 3 …w n , the attention weight is calculated as follows:

[0047] in, It is achieved by transforming the feature map F i The score value obtained by performing linear transformation and activation function operation.

[0048] For example, ,in and are the learnable weight matrix and bias vector respectively.

[0049] The fused features are:

[0050] This fusion feature is input into the subsequent model to obtain the estimated value of the vehicle detection parameter group , combined with the vehicle parameter calculation under the above comprehensive influencing factors, the final vehicle detection parameter group for:

[0051] The high-speed vehicle flow detection method and system based on deep learning provided in this embodiment comprehensively considers multiple factors when constructing the deep fusion network model, improves the accuracy of the vehicle detection parameter data output by the model, provides a more accurate data source for vehicle flow statistics, and ultimately improves the statistical accuracy of vehicle flow.

[0052] Embodiment 3 Different from the first and second embodiments, after the vehicle detection parameters are calculated by the deep fusion network model to obtain the current traffic flow, a time series prediction method, such as the autoregressive integrated moving average model (ARIMA (p, d, q)), is used to predict the traffic flow Rt+k at the future time t+k.

[0053] First, the d-order difference is performed on the traffic flow time series data Yt to make it stable and obtain the stable series Yt.

[0054] Autoregressive part (AR(p)):

[0055] in, is the autoregressive coefficient, is a white noise sequence.

[0056] Moving average part (MA (q)):

[0057] in, is the mean, is the moving average coefficient.

[0058] The predicted future stationary series value Perform the inverse difference operation to obtain the predicted traffic flow value Rt+k. Combined with the above comprehensive influencing factors and the traffic flow calculation after feature fusion, the final predicted future traffic flow R predict for:

[0059] in, It is the comprehensive impact coefficient at the prediction time t+k, which is calculated based on the predicted value of future traffic meteorological data and the rules of historical congestion data.

[0060] The high-speed vehicle flow detection method and system based on deep learning provided in this embodiment, based on optimizing the deep fusion network model to calculate the vehicle flow, further proposes a prediction method for subsequent moments, and provides a more comprehensive process suitable for high-speed vehicle flow detection, thereby improving the applicability of this solution and being more conducive to guiding subsequent traffic management.

[0061] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field are aware of all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement the scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A high-speed vehicle flow detection method based on deep learning, characterized in that: The method comprises: Acquire multimodal traffic data and environmental data in real time from key sections of the highway, where the multimodal traffic data includes vehicle video images, vehicle kinematic data, and vehicle 3D point cloud data; Preprocess the multimodal traffic data acquired in real time; The pre-processed multimodal traffic data and environmental data are input into the pre-trained deep fusion network model to perform vehicle detection, tracking and traffic flow statistics, and output a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow at the corresponding key sections; Input various vehicle parameters into the pre-trained high-speed traffic flow detection model, and output the traffic flow level information at the corresponding key sections; According to the changes in vehicle parameters at each key section within the preset time, the deep fusion network model at the key section is dynamically updated by model fusion; the high-speed traffic flow detection model is updated according to the external conditions introduced by the traffic flow level information during actual use; The deep fusion network model at the key section is dynamically updated by using model fusion. At preset intervals, multiple vehicle parameters of each key section of the highway are compared separately; data analysis algorithms are used to determine whether vehicle parameters have changed significantly; when changes in vehicle parameters at the key section are detected, the update interval of the multimodal traffic data at the key section is adjusted, and the newly collected multimodal traffic data is gradually integrated into the training data set using online learning, the model is updated and a new model is obtained; by comparing the performance of the new model and the original model on the verification data set, the weights of the new model and the original model for fusing and outputting multiple vehicle parameters are determined; in the online learning process of the model, only part of the newly collected multimodal traffic data is integrated into the training set each time, and part of the model parameters are updated.

2. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: The preprocessing includes the spatiotemporal alignment of multimodal traffic data; the spatiotemporal alignment of multimodal traffic data is to synchronize the time and unify the spatial coordinates of vehicle kinematic data, vehicle three-dimensional point cloud data and vehicle video images.

3. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: When training the deep fusion network model, obtain multimodal traffic data for training in different time periods, different environmental data, and different road sections, and use data enhancement strategies to expand the data and build a training data set; In the training data set, vehicle video images are annotated based on vehicle kinematic data and vehicle three-dimensional point cloud data, and the annotation content is related to the output vehicle parameters.

4. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: The various vehicle parameters include vehicle flow, vehicle position, vehicle speed, vehicle type and vehicle travel direction.

5. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: Based on the architecture that combines ResNet and Transformer, a deep fusion network model is constructed, where the deep fusion network model includes a data fusion input layer, a feature extraction layer, a multi-scale feature interaction module, a spatiotemporal attention module and a prediction output layer.

6. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: Based on historical multiple vehicle parameters and historical traffic flow level information, as well as deep learning, a traffic flow detection model is constructed to characterize the mapping relationship between multiple vehicle parameters and traffic flow level information.

7. The high-speed vehicle flow detection method based on deep learning according to claim 6 is characterized in that: Historical traffic flow level information includes standard traffic flow level classification and dynamic adjustment information of traffic flow level recognition based on historical external conditions. The traffic flow level information is adjusted after the introduction of historical external conditions through deep learning technology to build a traffic flow detection model.

8. The high-speed vehicle flow detection method based on deep learning according to claim 1 is characterized in that: Vehicle parameter changes include traffic volume growth or decline exceeding threshold requirements and the emergence of new traffic volume peaks.

9. The high-speed vehicle flow detection method based on deep learning according to claim 5 is characterized in that: Construct a comprehensive impact coefficient I, the impact factor of traffic environment data is W, and the impact factor of historical congestion data is H, then: in, , are weight coefficients, which respectively represent the importance of traffic environment data and historical congestion data on traffic flow. is the correction constant; The initial estimated value of the vehicle detection parameter group obtained by performing vehicle detection through the deep fusion network model is F0. After comprehensively considering multiple factors, the vehicle detection parameter group F is: In the multi-scale feature interaction module, a multi-scale feature fusion method based on the attention mechanism is adopted; the feature maps of different scales are F1, F2, F3…F n , corresponding attention weights w1, w2, w3…w n , the attention weights are calculated as follows: in, It is achieved by transforming the feature map F i The score value obtained by performing linear transformation and activation function operation; ,in and They are respectively the learnable weight matrix and bias vector; The fused features are: This fusion feature is input into the subsequent model to obtain the estimated value of the vehicle detection parameter group , combined with the vehicle parameter calculation under the above comprehensive influencing factors, the final vehicle detection parameter group for: .

10. A high-speed vehicle flow detection system based on deep learning, characterized in that: The method for detecting high-speed vehicle flow based on deep learning according to any one of claims 1 to 9 is implemented, wherein the system comprises: A data acquisition module, used to acquire multimodal traffic data and environmental data from each key stage of the expressway, wherein the multimodal traffic data includes vehicle video images, vehicle kinematic data and vehicle three-dimensional point cloud data; A data processing module, used for preprocessing the acquired multimodal traffic data, wherein the preprocessing includes time-space alignment of the multimodal traffic data; Model building module, used to build deep fusion network model and traffic flow detection model; An execution module is used to input the pre-processed multi-modal traffic data and environmental data into a pre-trained deep fusion network model to perform vehicle detection, tracking and traffic flow statistics, and output a variety of vehicle parameters used to characterize the characteristics of high-speed traffic flow at corresponding key sections; it is also used to input a variety of vehicle parameters into a pre-trained traffic flow detection model to output traffic flow level information at corresponding key sections; The update module is used to dynamically update the deep fusion network model at each key section by model fusion according to the changes in vehicle parameters at the key section within a preset time; it is also used to update the high-speed traffic flow detection model according to the changes in traffic flow level information during actual use.

Images