A lane-level traffic detection and prediction method, electronic equipment and storage medium

Abstract

A lane-level traffic detection and prediction method, electronic equipment and storage medium belong to the field of intelligent traffic management. In order to solve the problem of improving the efficiency and accuracy of lane-level traffic detection and prediction. The present application comprises using a roadside monitoring camera to collect lane video; the collected lane video is input into the detection and tracking module, the detection and tracking module analyzes the lane video frame by frame based on the YOLOv5 target detection neural network and the DeepSort feature extraction model, obtains the detection and tracking result of the traffic vehicle, including tracking id, type, position and confidence data; a regional automatic generation method of vehicle trajectory is constructed, a vehicle polygon configuration region is generated, and the lane traffic related data of the vehicle trajectory in the vehicle polygon configuration region is calculated; the time and space dependence relationship is jointly modeled by combining the space-time attention mechanism, the relationship between time and traffic flow in a single lane is described, and the lane-level traffic is predicted.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic management technology, specifically relating to a lane-level traffic flow detection and prediction method, electronic equipment, and storage medium. Background Technology

[0002] With the accelerating pace of urbanization, traffic flow monitoring has become a key requirement in urban planning and intelligent transportation. Existing traffic flow monitoring methods are mostly based on sensors, cameras, or radar, but these methods have limitations in terms of cost, maintenance, and accuracy. Among them, methods based on ultrasonic and millimeter-wave radar suffer from engineering complexity, high costs, and limited information acquisition. In contrast, roadside monitoring cameras can obtain rich traffic information, including vehicle type, vehicle location, traffic flow, traffic density, and vehicle speed. However, existing video traffic flow detection methods are limited by factors such as equipment layout, environmental changes, and data noise. They require manual configuration of detection virtual lines or polygonal areas by professionals, and due to issues such as camera obstruction and movement, roadside monitoring cameras require regular maintenance. Existing methods also exhibit significant detection errors in complex environments, hindering large-scale deployment.

[0003] The invention patent with application number 201410826965.6 and invention title "Video Traffic Flow Detection Method and Device" discloses a video traffic flow detection method based on background subtraction. The method obtains road traffic video information of a preset detection area; extracts images frame by frame from the obtained road traffic video information to obtain grayscale images of each frame and grayscale background images under vehicle-free conditions; obtains a background subtraction image; determines whether vehicle outlines exist in the background subtraction image; if so, counts the vehicle outlines; otherwise, segments the background subtraction image, and for the partial information of the background subtraction image segmented into each lane, determines whether vehicle outlines conforming to preset rules exist; if so, counts the vehicle outlines conforming to the preset rules in different lanes respectively. However, this method may be affected by the preset rule settings when determining whether vehicle outlines conform to the preset rules exist. If the preset rules are not comprehensive or accurate enough, it may lead to missed detections or false detections. In complex road traffic and dense traffic flow conditions, the background subtraction method may have a high false detection rate because when traffic flow is high and vehicles are dense, the background subtraction image may contain a large number of non-vehicle variations. In addition, steps such as extracting images frame by frame, performing background subtraction, segmentation, and counting require a lot of computing resources, which may result in a slow running speed of the method and make it unsuitable for scenarios that require real-time processing.

[0004] Therefore, there is an urgent need for a fully automated, high-precision traffic flow detection method that can adapt to the multi-view and complex environments of surveillance cameras. Summary of the Invention

[0005] The problem to be solved by this invention is to improve the efficiency and accuracy of lane-level traffic flow detection and prediction, and to propose a lane-level traffic flow detection and prediction method, electronic device and storage medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A lane-level traffic flow detection and prediction method includes the following steps:

[0008] S1. Use roadside surveillance cameras to collect lane video;

[0009] S2. The collected lane video is input into the detection and tracking module. The detection and tracking module analyzes the lane video frame by frame based on the YOLOv5 target detection neural network and the DeepSort feature extraction model to obtain the detection and tracking results of traffic vehicles, including tracking ID, type, location and confidence data.

[0010] S3. Based on the detection and tracking results of traffic vehicles obtained in step S2, determine whether to automatically generate a vehicle polygon configuration area. If yes, proceed to step S5; otherwise, proceed to step S4.

[0011] S4. Construct an automatic region generation method for vehicle trajectories, generate vehicle polygon configuration regions from the traffic vehicle detection and tracking results obtained in step S2, and calculate lane flow related data of vehicle trajectories within the vehicle polygon configuration regions.

[0012] S5. Based on the lane flow-related data of vehicle trajectories within the vehicle polygon configuration area obtained in step S4, and combining the spatiotemporal attention mechanism to jointly model the temporal and spatial dependencies, the relationship between time and traffic flow in a single lane is described to predict lane-level traffic flow.

[0013] Furthermore, in step S2, the YOLOv5 object detection neural network and the DeepSort feature extraction model are adjusted using a dataset from multiple perspectives of roadside surveillance cameras.

[0014] Furthermore, the judgment criteria for step S3 are as follows:

[0015] S3.1. If it is the first time processing offline or online video of the location, extract a set of tracking trajectories with a duration of 5 minutes. If the number of trajectories is too small, continue to extract; then proceed to step S4 for automatic area generation; otherwise, proceed to step S5 for lane-level traffic flow prediction.

[0016] S3.2. If the roadside traffic camera is obstructed or moved, the area automatic generation in step S4 is repeated; otherwise, proceed to step S5 for lane-level traffic flow prediction.

[0017] Furthermore, the specific implementation method of step S4 includes the following steps:

[0018] S4.1. Based on the detection and tracking results of traffic vehicles obtained in step S2, read the vehicle trajectory data, perform outlier detection on the vehicle trajectory, use the local outlier factor algorithm to perform unsupervised abnormal trajectory point detection, filter outliers, and obtain the filtered vehicle trajectory data.

[0019] S4.2. Process and filter the filtered vehicle trajectory data. First, obtain the midpoint coordinates of each trajectory. Then, scale the midpoint coordinates according to the scaling factor to obtain the scaled bounding rectangle. Next, classify the vehicle trajectory data within the bounding rectangle according to the trajectory direction slope to obtain the processed vehicle trajectory data.

[0020] S4.3. Fit and cluster the processed vehicle trajectory data obtained in step S4.2. Use ridge regression to fit the trajectory points and calculate the slope. Then use the kernel density estimation algorithm to cluster the trajectory directions to obtain vehicle trajectory data with confirmed directions.

[0021] S4.4. Perform cluster analysis on the vehicle trajectory data of the confirmed direction obtained in step S4.3 to determine the lane to which the trajectory belongs; use DBSCAN clustering to determine the formed trajectory clusters, mark the lanes of the corresponding clusters, and count the number of trajectories for each label to estimate the lane flow.

[0022] Furthermore, the specific implementation method of step S5 includes the following steps:

[0023] S5.1. Based on the lane flow-related data of vehicle trajectories within the vehicle polygon configuration area obtained in step S4, construct a dynamic spatiotemporal graph edge weight calculation method, the expression of which is:

[0024]

[0025] in, For a moment Lane to the lane edge weights, For time window The number of vehicles changing lanes from inner lane i to lane j. For lane At any moment average vehicle speed These are the first hyperparameter and the second hyperparameter, respectively. As a random factor, For lane j at time The average vehicle speed;

[0026] S5.2. Construct a spatiotemporal attention mechanism to jointly model the temporal and spatial dependencies, and describe the relationship between time and traffic flow in a single lane;

[0027] S5.2.1. Define a spatial attention mechanism to capture the correlation between lanes, expressed as:

[0028]

[0029] in, For lane With lane At any moment The strength of the association, For lane At any moment The feature vectors include flow rate, speed, and traffic light status; Let the query matrix be the k-th attention head. Let K be the key matrix of the k-th attention head. For normalized exponential functions, For model dimensions;

[0030] Spatial attention mechanisms capture the traffic flow relationships between lanes through attention mechanisms;

[0031] S5.2.2. Based on the time attention mechanism, construct a time attention coefficient to capture the importance of historical moments. The expression is as follows:

[0032]

[0033] in, For lane At any moment With historical moments Time attention coefficient This is the model parameter matrix; the time attention coefficients represent the similarity of features between the current time step and historical time steps.

[0034] S5.2.3. Construct a lane spatiotemporal attention fusion mechanism to form a joint model of spatiotemporal features. The model is defined as follows:

[0035]

[0036] in, These are the parameters of the model's hidden layer. For lane The set of adjacent lanes; For the first A matrix of values ​​for each attention head; For the number of heads;

[0037] The spatiotemporal attention fusion mechanism integrates temporal and spatial features to form a spatiotemporal attention network;

[0038] S5.3. Combine the dynamic spatiotemporal graph edge weights obtained in step S5.1 with the lane spatiotemporal attention fusion mechanism to train and predict lane-level traffic flow.

[0039] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the lane-level traffic flow detection and prediction method.

[0040] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lane-level traffic flow detection and prediction method.

[0041] The beneficial effects of this invention are:

[0042] The lane-level traffic flow detection and prediction method described in this invention directly utilizes urban surveillance video data, eliminating the need for any on-site equipment installation and saving significant costs.

[0043] The present invention provides a lane-level traffic flow detection and prediction method, which proposes an automatic area generation method that can be directly applied to cameras with different perspectives and positions. Regular automatic generation and maintenance reduce operation and maintenance costs and facilitate large-scale deployment and application.

[0044] The lane-level traffic flow detection and prediction method described in this invention is based on a lane-level spatiotemporal attention mechanism. It can take into account the impact of vehicle lane changes and can detect traffic problems that may be caused by emergencies such as severe weather and traffic accidents in advance. Attached Figure Description

[0045] Figure 1 This is a flowchart of a lane-level traffic flow detection and prediction method according to the present invention;

[0046] Figure 2 The spatiotemporal attention mechanism constructed for this invention is used to jointly model the temporal and spatial dependencies. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0048] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0049] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 and attached Figure 2 Detailed explanation is as follows:

[0050] Example 1:

[0051] A lane-level traffic flow detection and prediction method includes the following steps:

[0052] S1. Use roadside surveillance cameras to collect lane video;

[0053] S2. The collected lane video is input into the detection and tracking module. The detection and tracking module analyzes the lane video frame by frame based on the YOLOv5 target detection neural network and the DeepSort feature extraction model to obtain the detection and tracking results of traffic vehicles, including tracking ID, type, location and confidence data.

[0054] Furthermore, in step S2, the YOLOv5 object detection neural network and the DeepSort feature extraction model are adjusted using a dataset from multiple perspectives of roadside surveillance cameras.

[0055] S3. Based on the detection and tracking results of traffic vehicles obtained in step S2, determine whether to automatically generate a vehicle polygon configuration area. If yes, proceed to step S5; otherwise, proceed to step S4.

[0056] Furthermore, the judgment criteria for step S3 are as follows:

[0057] S3.1. If it is the first time processing offline or online video of the location, extract a set of tracking trajectories with a duration of 5 minutes. If the number of trajectories is too small, continue to extract; then proceed to step S4 for automatic area generation; otherwise, proceed to step S5 for lane-level traffic flow prediction.

[0058] S3.2. If the roadside traffic camera is obstructed or moved, the area automatic generation in step S4 is repeated; otherwise, proceed to step S5 for lane-level traffic flow prediction.

[0059] S4. Construct an automatic region generation method for vehicle trajectories, generate vehicle polygon configuration regions from the traffic vehicle detection and tracking results obtained in step S2, and calculate lane flow related data of vehicle trajectories within the vehicle polygon configuration regions.

[0060] Furthermore, the specific implementation method of step S4 includes the following steps:

[0061] S4.1. Based on the detection and tracking results of traffic vehicles obtained in step S2, read the vehicle trajectory data, perform outlier detection on the vehicle trajectory, use the local outlier factor algorithm to perform unsupervised abnormal trajectory point detection, filter outliers, and obtain the filtered vehicle trajectory data.

[0062] Local outlier Point neighborhood Locally accessible density of other points within the point The average of the ratios of locally accessible densities is expressed as:

[0063]

[0064] in, It is a point The Locally reachable density, representing points The Neighborhood All points inside Average reachable distance, It is the neighborhood range. It is a point The Locally achievable density, It is a point The Neighborhood;

[0065] Defined as:

[0066]

[0067] in, It is a point Time The Reachable distance, represented by a point The Distance and point Time The larger of the distances:

[0068] S4.2. Process and filter the filtered vehicle trajectory data. First, obtain the midpoint coordinates of each trajectory. Then, scale the midpoint coordinates according to the scaling factor to obtain the scaled bounding rectangle. Next, classify the vehicle trajectory data within the bounding rectangle according to the trajectory direction slope to obtain the processed vehicle trajectory data.

[0069] Furthermore, the default scaling factor is 0.8; the processed vehicle trajectory data is obtained as forward and reverse vehicle trajectory data.

[0070] S4.3. Fit and cluster the processed vehicle trajectory data obtained in step S4.2. Use ridge regression to fit the trajectory points and calculate the slope. Then use the kernel density estimation algorithm to cluster the trajectory directions to obtain vehicle trajectory data with confirmed directions.

[0071] The kernel density estimation algorithm works by placing a kernel function near each vehicle trajectory point, then taking a weighted average of the kernel function values ​​to estimate the probability density, extracting the main direction of the vehicle trajectory, determining whether the vehicle trajectory is forward or reverse based on the angle between the clustered vehicle trajectory direction vector and the current vehicle trajectory vector, and further dividing the trajectory based on the position of the vehicle trajectory point in the main direction of the trajectory.

[0072] The method for estimating vehicle trajectories using ridge regression on vehicle trajectory points is as follows:

[0073]

[0074] in, It is a ridge regression loss function; It refers to the location of the trajectory point; It is a trajectory estimation of location; It is the regularization weight; `i` is the regularization variable; `i` is the lane number; `n` is the total length of the trajectory sequence.

[0075] The formula for calculating the probability density estimator in the kernel density estimation algorithm is:

[0076]

[0077] in, It is the probability density distribution of the trajectory variable; Let h be the bandwidth parameter, K be the kernel function, and x be the trajectory variable;

[0078] Furthermore, by adjusting the bandwidth parameter and the kernel function type, the smoothness and shape of the estimated probability density curve can be affected.

[0079] S4.4. Perform cluster analysis on the vehicle trajectory data of the confirmed direction obtained in step S4.3 to determine the lane to which the trajectory belongs; use DBSCAN clustering to determine the formed trajectory clusters, mark the lanes of the corresponding clusters, and count the number of trajectories for each label to estimate the lane flow.

[0080] Furthermore, after obtaining the adaptive analysis results of traffic vehicle trajectories, the traffic flow data is saved. For real-time traffic flow data, statistics are compiled every 30 seconds and written to the database; offline traffic flow data is written to the corresponding offline video or text file.

[0081] Furthermore, by utilizing vehicle lane-changing frequency and lane speed differences extracted from surveillance camera video images, combined with the lane-level traffic flow extracted in the previous section, and leveraging the multimodal scene understanding capabilities of AI models in numerical and image understanding, high-precision lane-level traffic flow prediction can be achieved.

[0082] S5. Based on the lane flow-related data of vehicle trajectories within the vehicle polygon configuration area obtained in step S4, and combining the spatiotemporal attention mechanism to jointly model the temporal and spatial dependencies, the relationship between time and traffic flow in a single lane is described to predict lane-level traffic flow.

[0083] Furthermore, the specific implementation method of step S5 includes the following steps:

[0084] S5.1. Based on the lane flow-related data of vehicle trajectories within the vehicle polygon configuration area obtained in step S4, construct a dynamic spatiotemporal graph edge weight calculation method, the expression of which is:

[0085]

[0086] in, For a moment Lane to the lane edge weights, For time window The number of vehicles changing lanes from inner lane i to lane j. For lane At any moment average vehicle speed These are the first hyperparameter and the second hyperparameter, respectively. As a random factor, For lane j at time The average vehicle speed;

[0087] The edge weights of the dynamic graph reflect the intensity of vehicle interaction between lanes, taking into account the frequency of lane changes and the difference in lane speed; the initial settings are... The first term in the formula represents the lane change frequency characteristic: capturing the dynamic correlation between vehicles in lanes, such as the enhanced interaction between left-turn lanes and straight lanes during green light; the second term represents the speed difference characteristic: if the speed difference between adjacent lanes is large (such as congested lanes and unobstructed lanes), the probability of vehicles changing lanes is higher, and the weight increases accordingly.

[0088] S5.2. Construct a spatiotemporal attention mechanism to jointly model the temporal and spatial dependencies, and describe the relationship between time and traffic flow in a single lane;

[0089] S5.2.1. Define a spatial attention mechanism to capture the correlation between lanes, expressed as:

[0090]

[0091] in, For lane With lane At any moment The strength of the association, For lane At any moment The feature vectors include flow rate, speed, and traffic light status; Let the query matrix be the k-th attention head. Let K be the key matrix of the k-th attention head. For normalized exponential functions, For model dimensions;

[0092] Spatial attention mechanisms capture the traffic flow relationships between lanes through attention mechanisms;

[0093] S5.2.2. Based on the time attention mechanism, construct a time attention coefficient to capture the importance of historical moments. The expression is as follows:

[0094]

[0095] in, For lane At any moment With historical moments Time attention coefficient This is the model parameter matrix; the time attention coefficients represent the similarity of features between the current time step and historical time steps.

[0096] S5.2.3. Construct a lane spatiotemporal attention fusion mechanism to form a joint model of spatiotemporal features. The model is defined as follows:

[0097]

[0098] in, These are the parameters of the model's hidden layer. For lane The set of adjacent lanes; For the first A matrix of values ​​for each attention head; For the number of heads;

[0099] The spatiotemporal attention fusion mechanism integrates temporal and spatial features to form a spatiotemporal attention network;

[0100] S5.3. Combine the dynamic spatiotemporal graph edge weights obtained in step S5.1 with the lane spatiotemporal attention fusion mechanism to train and predict lane-level traffic flow.

[0101] Furthermore, the model is trained by combining dynamic spatiotemporal graph edge weights with lane spatiotemporal attention mechanisms; the model structure diagram is shown below. Figure 2 After training, the weights are then distributed to each edge node for prediction, and any abnormal traffic situations that may occur in the future are reported in real time.

[0102] This implementation method directly utilizes urban surveillance video data without requiring any on-site equipment installation, saving significant costs. It proposes an automatic area generation method, directly applicable to cameras at different angles and locations, with regular automatic generation and maintenance, reducing operational costs and facilitating large-scale deployment and application. The prediction method is based on a lane-level spatiotemporal attention mechanism, which can consider the impact of vehicle lane changes and proactively detect potential traffic problems caused by unforeseen events such as severe weather and traffic accidents.

[0103] Example 2:

[0104] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the lane-level traffic flow detection and prediction method described in Embodiment 1.

[0105] The computer device of the present invention may include a processor and a memory, such as a microcontroller containing a central processing unit. Furthermore, the processor executes the computer program stored in the memory to implement the steps of the lane-level traffic flow detection and prediction method described above.

[0106] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0107] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0108] Example 3:

[0109] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lane-level traffic flow detection and prediction method described in Embodiment 1.

[0110] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-described modeling method for modifyable relation-driven modeling data based on CREO software can be implemented.

[0111] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0112] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A lane-level traffic flow detection and prediction method, characterized in that, Includes the following steps: S1. Use roadside surveillance cameras to collect lane video; S2. The collected lane video is input into the detection and tracking module. The detection and tracking module analyzes the lane video frame by frame based on the YOLOv5 target detection neural network and the DeepSort feature extraction model to obtain the detection and tracking results of traffic vehicles, including tracking ID, type, location and confidence data. S3. Based on the detection and tracking results of traffic vehicles obtained in step S2, determine whether to automatically generate a vehicle polygon configuration area. If yes, proceed to step S5; otherwise, proceed to step S4. S4. Construct an automatic region generation method for vehicle trajectories, generate vehicle polygon configuration regions from the traffic vehicle detection and tracking results obtained in step S2, and calculate lane flow related data of vehicle trajectories within the vehicle polygon configuration regions. S5. Based on the lane flow related data of vehicle trajectories in the vehicle polygon configuration area obtained in step S4, and combined with the spatiotemporal attention mechanism, jointly model the temporal and spatial dependencies, describe the relationship between time and traffic flow in a single lane, and predict lane-level traffic flow. The specific implementation method of step S5 includes the following steps: S5.

1. Based on the lane flow-related data of vehicle trajectories within the vehicle polygon configuration area obtained in step S4, construct a dynamic spatiotemporal graph edge weight calculation method, the expression of which is: Among them, w ij (t) represents the edge weight from lane i to lane j at time t, N switch (i→j,t-ΔT,t) represents the number of vehicles changing lanes from lane i to lane j within the time window ΔT, v i (t) represents the average vehicle speed of lane i at time t, and α and β are the first hyperparameter and the second hyperparameter, respectively. v is a random factor j (t) represents the average speed of lane j at time t; S5.

2. Construct a spatiotemporal attention mechanism to jointly model the temporal and spatial dependencies, and describe the relationship between time and traffic flow in a single lane; S5.2.

1. Define a spatial attention mechanism to capture the correlation between lanes, expressed as: in, h represents the correlation strength between lane i and lane j at time t. i (t) is the feature vector of lane i at time t, including traffic flow, speed, and traffic light status; Q (k) Let K be the query matrix for the k-th attention head. (k) Let d be the key matrix of the k-th attention head, Softmax be the normalized exponential function, and d be the key matrix of the k-th attention head. k For model dimensions; Spatial attention mechanisms capture the traffic flow relationships between lanes through attention mechanisms; S5.2.

2. Based on the time attention mechanism, construct a time attention coefficient to capture the importance of historical moments. The expression is as follows: in, Let i be the time attention coefficient of lane i at time t and the historical time t-τ. This is the model parameter matrix; the time attention coefficients represent the similarity of features between the current time step and historical time steps. S5.2.

3. Construct a lane spatiotemporal attention fusion mechanism to form a joint model of spatiotemporal features. The model is defined as follows: Where, h′ i (t) represents the hidden layer parameters of the model. V is the set of adjacent lanes of lane i; (k) Let K be the value matrix of the k-th attention head; K is the number of attention heads. The spatiotemporal attention fusion mechanism integrates temporal and spatial features to form a spatiotemporal attention network; S5.

3. Combine the dynamic spatiotemporal graph edge weights obtained in step S5.1 with the lane spatiotemporal attention fusion mechanism to train and predict lane-level traffic flow.

2. The lane-level traffic flow detection and prediction method according to claim 1, characterized in that, In step S2, the YOLOv5 object detection neural network and the DeepSort feature extraction model are adjusted using a dataset from multiple perspectives of roadside surveillance cameras.

3. The lane-level traffic flow detection and prediction method according to claim 2, characterized in that, The judgment criteria for step S3 are as follows: S3.

1. If it is the first time processing offline or online video of the location, extract a set of tracking trajectories with a duration of 5 minutes. If the number of trajectories is too small, continue to extract; then proceed to step S4 for automatic area generation; otherwise, proceed to step S5 for lane-level traffic flow prediction. S3.

2. If the roadside traffic camera is obstructed or moved, the area automatic generation in step S4 will be performed again; otherwise, proceed to step S5 for lane-level traffic flow prediction.

4. The lane-level traffic flow detection and prediction method according to claim 3, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. Based on the detection and tracking results of traffic vehicles obtained in step S2, read the vehicle trajectory data, perform outlier detection on the vehicle trajectory, use the local outlier factor algorithm to perform unsupervised abnormal trajectory point detection, filter outliers, and obtain filtered vehicle trajectory data. S4.

2. Process and filter the filtered vehicle trajectory data. First, obtain the midpoint coordinates of each trajectory. Then, scale the midpoint coordinates according to the scaling factor to obtain the scaled bounding rectangle range. Next, classify the vehicle trajectory data within the bounding rectangle range according to the trajectory direction slope to obtain the processed vehicle trajectory data. S4.

3. Fit and cluster the processed vehicle trajectory data obtained in step S4.

2. Use ridge regression to fit the trajectory points and calculate the slope. Then use the kernel density estimation algorithm to cluster the trajectory directions to obtain vehicle trajectory data with confirmed directions. S4.

4. Perform cluster analysis on the vehicle trajectory data of the confirmed direction obtained in step S4.3 to determine the lane to which the trajectory belongs; DBSCAN clustering is used to identify the formed trajectory clusters, the lanes of the corresponding clusters are marked, and the number of trajectories for each label is counted to estimate the lane flow.

5. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the lane-level traffic flow detection and prediction method according to any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the lane-level traffic flow detection and prediction method according to any one of claims 1-4.

Images