A traffic scene congestion cause identification method based on multi-modal perception

Abstract

This invention relates to the field of transportation, and more particularly to a method for identifying the causes of traffic congestion based on multimodal perception. First, a static directed road network topology is constructed based on acquired road vector data or maps. Traffic flow feature vectors are obtained based on collected road scene videos. Signal feature vectors are obtained based on the current operational status of traffic lights at each intersection. Meteorological feature vectors are obtained based on acquired meteorological data. The traffic flow feature vector, signal feature vector, and meteorological feature vector are concatenated into a multimodal feature vector. Then, based on the static directed road network topology, the multimodal feature vector is input into a bidirectional causal propagation decomposition network for processing, outputting the congestion severity of the current road segment and the contribution ratio of each congestion cause. This method solves the technical problems of lacking structured road network modeling capabilities, low efficiency of multi-source feature fusion, lack of a unified processing framework, and difficulty in quantifying and modeling the influence of upstream and downstream traffic conditions.

Description

Technical Field

[0001] This invention relates to the field of transportation, and more particularly to a method for identifying the causes of traffic congestion based on multimodal perception. Background Technology

[0002] Traffic congestion not only reduces residents' travel efficiency, wastes energy, and pollutes the environment, but also affects multiple fields such as emergency response and logistics, bringing a serious socio-economic burden. Therefore, accurately identifying the causes of traffic congestion and providing a scientific basis for traffic management and optimization has become an important direction for research in intelligent transportation systems. Traditional methods for identifying the causes of traffic congestion mainly rely on single-modal data, such as video surveillance images, geomagnetic induction data, GPS trajectory data, or traffic flow sensor data, and use rule-based models or statistical pattern recognition algorithms for analysis. For example, they use sudden changes in vehicle speed and flow to determine the time of congestion, and analyze bottleneck locations and flow compression through traffic flow theory. However, these methods have significant limitations. On the one hand, single-modal data cannot fully reflect complex traffic scenarios; on the other hand, traditional models lack robustness and generalization ability when dealing with dynamically changing and multi-factor-intertwined traffic environments. Multimodal perception-based methods for identifying the causes of traffic congestion not only break through the limitations of traditional single-sensor models, but also provide strong technical support for understanding the formation mechanism of traffic congestion and improving the accuracy of traffic management through deep fusion of multi-source data. Summary of the Invention

[0003] This invention provides a method for identifying the causes of traffic congestion based on multimodal perception, in order to solve the problems of lack of structured road network modeling capabilities, which leads to the inability to effectively correlate data; low efficiency of multi-source feature fusion and lack of a unified processing framework; and the difficulty in quantifying and modeling the impact of upstream and downstream traffic conditions.

[0004] The present invention provides a method for identifying the causes of traffic congestion in a traffic scenario based on multimodal perception, comprising the following steps:

[0005] S1. Based on the acquired road vector data or map, construct a static directed road network topology; based on the collected road scene video, perform category detection on each frame of the image, assign a unique ID to each detected motor vehicle and maintain its trajectory, perform statistical analysis on all trajectories belonging to the current road segment, and extract traffic flow feature vectors; based on the acquired current traffic light operation status at each intersection, calculate the three-dimensional indicator vector and the traffic light time ratio to obtain the signal feature vector; after standardizing the acquired meteorological data, integrate it into a meteorological feature vector; horizontally concatenate the traffic flow feature vector, signal feature vector, and meteorological feature vector into a multimodal feature vector;

[0006] S2. Based on the static directed road network topology, the multimodal feature vectors are input into the bidirectional causal propagation decomposition network for processing, and the congestion severity of the current road segment and the contribution ratio of each congestion cause are output.

[0007] Preferably, S2 specifically includes:

[0008] In the feature encoding stage of the bidirectional causal propagation decomposition network, the input multimodal feature vector is input into a two-layer fully connected neural network. The first layer of the fully connected neural network processes the multimodal feature vector to obtain the first hidden layer feature vector.

[0009] Preferably, S2 specifically includes:

[0010] The second fully connected neural network processes the feature vectors of the first hidden layer to obtain the feature vectors of the second hidden layer.

[0011] Preferably, S2 specifically includes:

[0012] In the causal semantic embedding decomposition stage of the bidirectional causal propagation decomposition network, a causal projection matrix is ​​introduced for mapping based on the feature vector of the second hidden layer to obtain the causal contribution vector.

[0013] Preferably, S2 specifically includes:

[0014] In the bidirectional single-step causal impact aggregation stage along the road network in the bidirectional causal propagation decomposition network, based on the traffic flow feature vector, the propagation weight of the upstream road segment and the queue spillover weight of the downstream road segment are calculated respectively. Based on the propagation weight of the upstream road segment and the queue spillover weight of the downstream road segment, the causal contribution vector of the upstream road segment and the causal contribution vector of the downstream road segment are weighted and summed respectively to obtain the weighted summation result of the upstream road segment and the downstream road segment.

[0015] Preferably, S2 specifically includes:

[0016] The causal contribution vector, the weighted sum of the upstream and downstream road segments are added together to obtain the comprehensive causal vector.

[0017] Preferably, S2 specifically includes:

[0018] In the congestion severity and contribution ratio calculation stage of the bidirectional causal propagation decomposition network, the comprehensive causal vector is linearly combined and activated to obtain the congestion severity.

[0019] Preferably, S2 specifically includes:

[0020] Normalize each component of the comprehensive causal vector to obtain a probability distribution; multiply the probability distribution by the congestion severity to obtain the contribution ratio of each congestion cause dimension.

[0021] The beneficial effects of the technical solution of the present invention are:

[0022] 1. By modeling existing urban road vector data or high-precision maps, a refined static directed road network topology with a directed graph structure is constructed. Intersections are abstracted as nodes, and each one-way lane or one-way road segment is abstracted as a directed edge, and geometric and functional attributes are assigned. This makes traffic network modeling more accurate, improves the precision and operability of traffic system modeling, and provides reliable topological support for congestion analysis.

[0023] 2. By extracting traffic flow feature vectors through video processing, extracting signal feature vectors by reading the operation status of traffic lights, and acquiring and standardizing meteorological data to form meteorological feature vectors, real-time perception of multi-dimensional traffic conditions can be achieved.

[0024] 3. By decomposing network features through bidirectional causal propagation to extract potential interaction relationships, and then projecting them onto a causal space with physical semantics, we can decompose different dimensions of congestion causes such as "signal control" and "weather influence". This not only identifies congestion, but also explains the specific reasons for congestion, thus enhancing the application value.

[0025] 4. Simulate the forward propagation and backward flow of congestion in real traffic, dynamically adjust the causes of congestion on the current road segment, more accurately reflect the spatiotemporal evolution of congestion causes in complex traffic networks, and improve the accuracy of analysis. Attached Figure Description

[0026] Figure 1 This is a flowchart of a traffic congestion cause identification method based on multimodal perception, as described in this invention. Detailed Implementation

[0027] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0029] The following description, in conjunction with the accompanying drawings, details a specific scheme for a traffic congestion cause identification method based on multimodal perception provided by the present invention.

[0030] See attached document Figure 1 The diagram illustrates a flowchart of a traffic congestion cause identification method based on multimodal perception, provided by an embodiment of the present invention. The method includes the following steps:

[0031] S1. Based on the acquired road vector data or map, construct a static directed road network topology; based on the collected road scene video, perform category detection on each frame of the image, assign a unique ID to each detected motor vehicle and maintain its trajectory, perform statistical analysis on all trajectories belonging to the current road segment, and extract traffic flow feature vectors; based on the acquired current traffic light operation status at each intersection, calculate the three-dimensional indicator vector and the traffic light time ratio to obtain the signal feature vector; after standardizing the acquired meteorological data, integrate it into a meteorological feature vector; horizontally concatenate the traffic flow feature vector, signal feature vector, and meteorological feature vector into a multimodal feature vector.

[0032] Based on existing urban road vector data or high-precision maps obtained from existing map databases, a static directed road network topology with a directed graph structure is constructed. Each intersection is abstracted as a node, and each one-way lane or one-way road segment is abstracted as a directed edge. Each directed edge is assigned key geometric and functional attributes, including the actual length of the road segment, the number of lanes, the design speed of the road segment, upstream connecting nodes, downstream connecting nodes, etc. The design speed of the road segment comes from road design documents, urban road engineering design specifications, or local technical standards.

[0033] Traffic-specific cameras deployed at fixed locations such as roadside poles, overpasses, or tunnel entrances capture real-time video of road scenes. This video is first fed into a pre-trained target detection model, such as the YOLO series of detection algorithms, to detect motor vehicles, non-motor vehicles, and pedestrians in each frame, outputting the bounding box, category confidence score, and category label for each target. Further, a multi-target tracking algorithm, such as ByteTrack, assigns a unique ID to each detected motor vehicle between consecutive video frames and maintains its trajectory. Statistical analysis is performed on all trajectories belonging to the current road segment to extract traffic flow feature vectors. Specifically, this includes counting the number of times a vehicle appears after deduplication to obtain the total number of vehicles, reflecting the vehicle population within the road segment; dividing the total number of vehicles by the actual length of the road segment yields the average number of vehicles per meter of road, i.e., vehicle density. A higher vehicle density indicates denser traffic and a higher likelihood of congestion; combining the instantaneous speeds of all vehicles, a harmonic average or an arithmetic average (when resources are limited) is used to calculate the spatial average speed, reflecting the traffic flow of the entire road segment. Multiplying the calculated vehicle density by the spatial average speed yields the traffic volume of the road segment. For each tracked vehicle target, the instantaneous speed of the vehicle is calculated by the pixel displacement of the vehicle between consecutive frames and the video frame rate. The calculation formula is as follows: , in, Indicates the instantaneous speed of the vehicle; This represents the vehicle's displacement in the real world between consecutive frames; Indicates the time interval between consecutive frames; This represents the pixel-to-actual distance conversion factor, estimated using the camera's perspective and physical reference objects. This indicates the pixel displacement of the vehicle in consecutive frames; This indicates the video frame rate.

[0034] The system reads the current operating status of traffic lights at each intersection from the signal controller, including the designed duration of each of the red, green, and yellow lights, as well as the specific light color at the current moment and how long the current light has been on. A three-dimensional indicator vector represents the current light color: red light corresponds to a 1 in the first bit and 0 in the rest; green light corresponds to a 1 in the second bit and 0 in the third bit; and yellow light corresponds to a 1 in the third bit and 0 in the rest. Simultaneously, based on the duration of the current light color, divided by the designed duration of the current light color, a signal time percentage between 0 and 1 is obtained. This signal time percentage reflects the progress of the current light phase. For example, a very low remaining green light percentage may indicate impending red, while a prolonged red light without turning green could cause severe congestion. The three-dimensional indicator vector and the signal time percentage together constitute the signal feature vector.

[0035] Meteorological data such as precipitation intensity, visibility, and wind speed at the current moment are obtained through urban meteorological monitoring stations or public meteorological grid data interfaces and stored in historical meteorological databases. The Z-score standardization algorithm based on historical meteorological data statistics in the historical meteorological database is used to process the standardized meteorological data and integrate it into meteorological feature vectors.

[0036] Traffic flow feature vectors, signal feature vectors, and meteorological feature vectors are horizontally concatenated into a multimodal feature vector and stored in a historical multimodal feature vector database for use in subsequent bidirectional causal propagation decomposition networks.

[0037] S2. Based on the static directed road network topology, the multimodal feature vectors are input into the bidirectional causal propagation decomposition network for processing, and the congestion severity of the current road segment and the contribution ratio of each congestion cause are output.

[0038] Based on the static directed road network topology, the multimodal feature vector of the current road segment is input into the bidirectional causal propagation decomposition network for processing, and the congestion severity of the current road segment and the contribution ratio of each congestion cause are output.

[0039] The bidirectional causal propagation decomposition network consists of four stages: feature encoding, causal semantic embedding decomposition, bidirectional single-step causal impact aggregation along the road network, and calculation of congestion severity and contribution ratio.

[0040] In the feature encoding stage, the input multimodal feature vector is fed into a two-layer fully connected neural network. The first fully connected neural network linearly weights the multimodal feature vector and adds a bias vector, then uses the ReLU activation function to obtain the first hidden layer feature vector. The second fully connected neural network linearly weights the first hidden layer feature vector again and adds a bias vector, and also uses ReLU activation to obtain the second hidden layer feature vector. Both fully connected neural networks use the ReLU activation function, the purpose of which is to perform nonlinear combination and refinement of features in the multimodal feature vector, and enhance the interactive expressive ability between features.

[0041] The formula for the feature encoding stage is expressed as follows: , in, Indicates time node To the node The first hidden layer feature vector corresponding to the road segment is used to perform the first nonlinear transformation on the multimodal feature vector, with a dimension of . , Typical values ​​are 64, 128, and 256; This represents the modified linear unit activation function, used to introduce nonlinearity and prevent gradient vanishing; This represents the first-layer weight matrix, used for linear transformation of features in the multimodal feature vector to the hidden space, with dimension 1. , The dimension of the multimodal feature vector is given by the first layer weight matrix, which is randomly initialized and learned through backpropagation during training. Indicates time node To the node The transpose of the multimodal feature vector of the road segment, with dimension . , Indicates the transpose symbol; This represents the first-level bias vector, used to increase the translational degrees of freedom of the linear transformation, with dimension . The first layer bias vector is randomly initialized and learned through backpropagation during training; , in, Indicates time node To the node The road segment corresponds to the second hidden layer feature vector, with dimension . , Typical values ​​are 128, 256, and 512; This represents the second-layer weight matrix, with dimension 1. The second-layer weight matrix is ​​randomly initialized and learned through backpropagation during training. This represents the second-layer bias vector, with dimension . The second-layer bias vector is randomly initialized and learned through backpropagation during training.

[0042] In the causal semantic embedding decomposition stage, a causal projection matrix is ​​used to directly project the second hidden layer feature vector obtained in the feature encoding stage into a low-dimensional space. Each dimension of the low-dimensional space is artificially assigned a clear physical meaning, namely the category of congestion causes. For example, the first dimension specifically corresponds to "improper signal control" (red light too long, green light insufficient, unreasonable timing, etc.), the second dimension corresponds to "the impact of severe weather", and the third dimension corresponds to "interference from non-motorized vehicles or pedestrians".

[0043] Causal semantic embedding decomposition formula: , in, Indicates time node To the node The causal contribution vector of the road segment, with dimension . , The number of congestion cause categories is determined by the historical multimodal feature vectors in the historical multimodal feature vector database and the actual congestion situation. Unsupervised learning methods such as K-means clustering, GMM, and PCA+ clustering are used to explore the potential number of congestion cause categories. This represents the causal projection matrix, used to linearly map the feature vectors of the second hidden layer to a causal dimension with fixed semantics, where the dimension is... The causal projection matrix is ​​randomly initialized and learned through backpropagation during training. Represents the causal projection bias vector, with dimension . The causal projection bias vector is randomly initialized and learned through backpropagation during training.

[0044] In the bidirectional, single-step causal effect aggregation phase along the road network, causal transmission in both directions is considered simultaneously:

[0045] On the one hand, there is the propagation effect from upstream to downstream. From all upstream road segments that directly flow into the starting node of the current road segment, the proportion of traffic flow of the upstream road segments is used as the propagation weight, which reflects which path carries more traffic demand when normal traffic flow propagates from upstream to downstream.

[0046] On the other hand, there is the effect of queue backflow from downstream to upstream. From all downstream road segments that flow directly out of the current road segment's end node, the degree to which the speed of the downstream road segment is significantly lower than that of the current road segment, i.e. whether the downstream is already severely congested, is used to calculate the queue backflow weight. The queue backflow weight reflects the intensity of vehicle queues flowing back from downstream to upstream.

[0047] The formula for the aggregation of bidirectional, single-step causal effects along the road network is: , in, Indicates time node To the node The comprehensive causal vector of the road segment integrating upstream and downstream influences; Indicates the starting node of the current road segment. The set of all directly upstream road segments; Represents a node To the node The upstream section; Indicates the start point from the current road segment's end node. The set of all directly downstream road segments that flow out; Represents a node To the node The downstream section; Indicates the upstream section For the current node To the node The propagation weight of the road segment, i.e., the upstream road segment The percentage of traffic flow is calculated using the following formula: , Indicates the upstream section At the delayed moment The traffic flow is derived from the traffic flow feature vector. This indicates the upstream propagation time delay, which depends on the upstream road segment. The ratio of the actual length of the road segment to the average speed in space; Indicates the starting node of the current road segment. The total flow of all directly upstream road segments; Represents a very small positive number, used to prevent the denominator from being 0, and takes the value of 0. ; Indicates time node To the node The causal contribution vector of the upstream road segment; Indicates downstream section For the current node To the node The queue overflow weight of a road segment is calculated using the following formula: , Indicates time Downstream section Spatial average velocity, This indicates that the time delay of downstream congestion backflow propagation depends on the downstream road segment. The ratio of the actual length of the road segment to the absolute value of the characteristic velocity of the congestion wave propagating upstream, the absolute value of the characteristic velocity of the congestion wave propagating upstream is obtained based on classical traffic flow theory and measured statistics; Indicates time node To the node The average spatial speed of the road segment; This indicates a value greater than or equal to 0; Indicates time node To the node The causal contribution vector of the downstream road segment.

[0048] By simultaneously capturing two real-world traffic phenomena—"congestion propagation forward" and "congestion backflow"—the accuracy and completeness of congestion cause decomposition in complex road network environments are significantly improved.

[0049] In the congestion severity and contribution ratio calculation stage, the comprehensive causal vector from the bidirectional single-step causal influence aggregation stage along the road network is linearly combined and then compressed to the 0-1 interval using the sigmoid function to obtain the congestion severity of the current road segment. The higher the congestion severity, the more significant the current congestion phenomenon. Then, each component of the comprehensive causal vector is subjected to softmax normalization to transform it into a probability distribution form. Multiplying each term of the probability distribution by the congestion severity yields the contribution ratio of each congestion cause dimension, and the sum of the contribution ratios of all congestion cause dimensions is exactly equal to the congestion severity.

[0050] The formula for calculating congestion severity is: , in, Indicates time node To the node The severity of congestion on the relevant road sections; This represents an activation function used to map any real number to... The interval is used to normalize the severity of congestion. The output weight vector represents the severity of congestion and is used to... A linear weighted average of the causal dimensions determines the contribution strength of each congestion cause to the severity of congestion. The dimensions are... The congestion severity output weight vector is randomly initialized and learned through backpropagation during training. This represents the transpose symbol.

[0051] The formula for calculating the contribution ratio is: , in, Indicates time No. The proportion of each cause of congestion contributing to current congestion. ; Indicates time No. The probability distribution of the causes of congestion.

[0052] It outputs the current road segment's congestion severity and the contribution ratio of each congestion cause in real time, improving the accuracy and interpretability of congestion management.

[0053] In summary, a method for identifying the causes of traffic congestion based on multimodal perception has been developed.

[0054] The order of the embodiments is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0055] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0056] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for identifying the causes of traffic congestion based on multimodal perception, characterized in that, Includes the following steps: S1. Based on the acquired road vector data or map, construct a static directed road network topology; based on the collected road scene video, perform category detection on each frame of the image, assign a unique ID to each detected motor vehicle and maintain its trajectory, perform statistical analysis on all trajectories belonging to the current road segment, and extract traffic flow feature vectors; based on the acquired current traffic light operation status at each intersection, calculate the three-dimensional indicator vector and the traffic light time ratio to obtain the signal feature vector; after standardizing the acquired meteorological data, integrate it into a meteorological feature vector; horizontally concatenate the traffic flow feature vector, signal feature vector, and meteorological feature vector into a multimodal feature vector; S2. Based on the static directed road network topology, the multimodal feature vectors are input into the bidirectional causal propagation decomposition network for processing, and the congestion severity of the current road segment and the contribution ratio of each congestion cause are output.

2. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 1, characterized in that, S2 specifically includes: In the feature encoding stage of the bidirectional causal propagation decomposition network, the input multimodal feature vector is input into a two-layer fully connected neural network. The first layer of the fully connected neural network processes the multimodal feature vector to obtain the first hidden layer feature vector.

3. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 2, characterized in that, S2 specifically includes: The second fully connected neural network processes the feature vectors of the first hidden layer to obtain the feature vectors of the second hidden layer.

4. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 3, characterized in that, S2 specifically includes: In the causal semantic embedding decomposition stage of the bidirectional causal propagation decomposition network, a causal projection matrix is ​​introduced for mapping based on the feature vector of the second hidden layer to obtain the causal contribution vector.

5. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 4, characterized in that, S2 specifically includes: In the bidirectional single-step causal influence aggregation stage along the road network in the bidirectional causal propagation decomposition network, based on the traffic flow feature vector, the propagation weight of the upstream road segment and the queue spillover weight of the downstream road segment are calculated respectively. Based on the propagation weight of the upstream road segment and the queue spillover weight of the downstream road segment, the causal contribution vector of the upstream road segment and the causal contribution vector of the downstream road segment are weighted and summed respectively to obtain the weighted summation result of the upstream road segment and the downstream road segment.

6. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 5, characterized in that, S2 specifically includes: The causal contribution vector, the weighted sum of the upstream and downstream road segments are added together to obtain the comprehensive causal vector.

7. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 6, characterized in that, S2 specifically includes: In the congestion severity and contribution ratio calculation stage of the bidirectional causal propagation decomposition network, the comprehensive causal vector is linearly combined and activated to obtain the congestion severity.

8. The method for identifying the causes of traffic congestion based on multimodal perception according to claim 7, characterized in that, S2 specifically includes: Normalize each component of the comprehensive causal vector to obtain a probability distribution; multiply the probability distribution by the congestion severity to obtain the contribution ratio of each congestion cause dimension.

Images