Traffic risk prediction method based on spatial-temporal feature fusion and dynamic attention

Abstract

The invention discloses a traffic risk prediction method based on spatio-temporal feature fusion and dynamic attention, and the method comprises the steps: firstly introducing a spatio-temporal feature fusion module based on a cross-attention mechanism, effectively capturing a cooperative relation between semantic information and motion information, and dynamically adjusting the weight of each piece of modal information according to a traffic scene; therefore, the alignment effect and the expression capability of the cross-modal features are improved. And a dynamic object attention module is further designed, and only key traffic participants with relatively high risk in the current frame are concerned, so that the response timeliness and the judgment precision of the model in an actual scene are enhanced. Experimental results on a DAD data set show that the method provided by the invention is remarkably superior to the existing mainstream method in two key indexes of average precision and average early warning time, AP exceeds 90%, mTTA reaches 4.45 seconds or above, and the application potential and engineering practical value of the method in a complex traffic environment are fully verified.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent transportation, and particularly relates to a traffic risk prediction method based on spatio-temporal feature fusion and dynamic attention. BACKGROUND

[0002] Traffic accidents are one of the main reasons that threaten human life safety and cause economic losses worldwide. Traffic accidents not only cause serious damage to vehicles and road facilities, but also can cause serious casualties, especially in high-speed driving or complex traffic environments, the consequences are more serious. Therefore, how to effectively reduce the occurrence of traffic accidents has become an important problem that needs to be solved in the field of intelligent transportation and autonomous driving.

[0003] In order to prevent accidents, in recent years, the academic and industrial circles have paid great attention to the key task of traffic accident risk prediction. Traffic accident risk prediction aims to analyze the behavior of traffic participants, environmental factors and potential conflict patterns, and identify the risk area or target of dangerous events that may occur in advance, so as to provide early warning information for drivers or autonomous driving systems, and take evasive measures in advance to maximize the safety of driving. The wide application of this technology is of great significance to improve the level of road traffic safety and reduce the rate of traffic accidents.

[0004] With the development of sensor technology and intelligent vehicles, the driving recorder has become a visual perception device widely deployed in modern vehicles. It has the advantages of convenient installation, wide range of collection, low data cost, etc., and can continuously record the visual information in front of the vehicle during driving, becoming an important data source for traffic scene understanding. Therefore, the research on traffic risk prediction based on driving recorder video has gradually emerged, becoming a key research direction in intelligent driving assistance systems. Compared with high-cost sensors such as laser radar and millimeter wave radar, the driving recorder provides a more feasible perception scheme, especially for low-cost intelligent driving systems.

[0005] At present, a large number of studies have been carried out around the video accident prediction method, which can be mainly divided into single-modal method and multi-modal fusion method. Among them, the single-modal method is mainly based on RGB image modeling, which can be further divided into the following categories: (1) the method based on recurrent neural network (RNN), which uses time series modeling to capture the dynamic change features between image frames; (2) the method based on graph convolution network (GCN), which builds a graph structure between traffic participants to model the interaction relationship to realize risk judgment; (3) the method based on the difference between predicted position and real position, which focuses on detecting behavior deviation to infer potential accident risk. These single-modal methods have relatively simple structure and high computational efficiency, but due to the limitations of RGB images in capturing object motion information, it is difficult to fully exploit potential dynamic behavior features, resulting in instability of prediction effect in complex scenes.

[0006] To make up for the shortcomings of single modal method, more and more researches try to fuse multiple modal information, so as to improve the perception ability and robustness of the model. The multi-modal method mainly includes: (1) the method of fusing RGB image and depth map, which improves the understanding ability of the model to the scene structure by introducing spatial distance information; (2) the method of fusing RGB image and optical flow map, which combines static appearance information and dynamic motion characteristics to more comprehensively describe the behavior mode of traffic participants. Among them, the dual modal method of RGB+optical flow has been widely concerned in recent years due to its complementarity in representing static and dynamic characteristics, and has achieved excellent performance in many traffic video analysis tasks.

[0007] Although the method of fusing RGB image and optical flow map has made certain progress in performance, there are still two key problems in practical application: (1) the fusion strategy is static and single. Most of the existing methods use simple feature addition or splicing and other static strategies, which cannot fully model the complex nonlinear interaction relationship between semantic information and motion information in the space-time dimension, and it is difficult to realize the deep fusion between information. At the same time, there is a lack of mechanism to dynamically adjust the modal weight according to different traffic scenes, which limits the generalization ability and prediction stability of the model when facing complex and variable environment; (2) historical feature introduces redundant interference. In order to enhance the target representation in the current frame, some methods use historical hidden features for target enhancement. However, in the highly dynamic traffic environment, some traffic participants may no longer appear in the current frame, and the use of their historical features not only cannot provide effective information, but also may introduce noise or outdated features, which interfere with the model's accurate judgment of the current risk and reduce the overall prediction performance. SUMMARY

[0008] In order to overcome the shortcomings of the prior art, the present application provides a traffic risk prediction method based on spatio-temporal feature fusion and dynamic attention. The method first introduces a spatio-temporal feature fusion module based on cross-attention mechanism, effectively captures the collaborative relationship between semantic and motion information, and dynamically adjusts the weight of each modal information according to the traffic scene, thereby improving the alignment effect and expression ability of cross-modal features. Further, a dynamic object attention module is designed to focus only on key traffic participants with high risk in the current frame, thereby enhancing the timeliness and discrimination accuracy of the model in actual scenes. The experimental results on the DAD dataset show that the proposed method is significantly better than the existing mainstream methods in terms of average precision and average warning time, with AP exceeding 90% and mTTA reaching more than 4.45 seconds, fully verifying the application potential and engineering practical value of the present application in complex traffic environment.

[0009] The technical scheme adopted by the present application to solve its technical problems is as follows:

[0010] Step 1: extract the optical flow information from the continuous RGB image sequence collected by the dashcam;

[0011] Step 2: extract the spatial and motion features contained in the visual information to realize the deep modeling of the behavior state of the traffic participants;

[0012] Step 3: design a spatio-temporal feature fusion module STFF;

[0013] Step 4: use a dynamic object attention module DOA to identify the most potentially threatening target in the current frame;

[0014] Step 5: use a gated recurrent unit GRU to model the time sequence of the fused feature sequence.

[0015] Preferably, the step 1 is specifically:

[0016] Step 1-1: let the original input RGB image sequence be:

[0017] I={I1,I2,…,I T}

[0018] Where I t represents the RGB image frame collected at time t, the total number of frames is T, t=1…T;

[0019] Step 1-2: use a pre-trained RAFT optical flow estimation model to calculate the optical flow map F t-1 for each pair of consecutive image frames (I t , I t ), and obtain the optical flow map sequence:

[0020] F={F2,F3,…,F T}

[0021] Where F t =RAFT(I t-1 ,I t ), represents the pixel-level displacement vector field from the t-1th frame to the tth frame, which is used to reflect the dynamic changes of objects in the scene;

[0022] Step 1-3: since the optical flow map is calculated from two consecutive image frames, the length of its sequence is one frame less than the original RGB image sequence, i.e. |{F}|=T-1; a frame of all-zero optical flow map F1 is supplemented at the beginning of the optical flow map sequence, i.e.

[0023] F1=0∈R H×W×2

[0024] Where H and W represent the height and width of the image respectively, and the channel number of 2 represents the horizontal and vertical optical flow components;

[0025] Step 1-4: The final aligned optical flow map sequence is:

[0026] F' = {F1, F2, …, F T}

[0027] The RGB image sequence I and the optical flow map sequence F' are aligned in the time dimension.

[0028] Preferably, step 2 is specifically:

[0029] Step 2-1: Perform target detection on each frame of the RGB image sequence I t using a pre-trained target detector to obtain the bounding box set of all traffic participants in the frame; retain the top N targets with the highest detection confidence;

[0030] Step 2-2: Use a pre-trained VGG-16 neural network to extract features from the RGB image and the optical flow map respectively, and the extraction process includes two levels:

[0031] (1) Frame-level feature extraction: globally encode the entire frame image and the optical flow map to capture the overall semantics and dynamic trends at the scene level;

[0032] (2) Target-level feature extraction: use the detected bounding box to crop and extract features for each target region, further focusing on the semantics or motion changes of local key targets;

[0033] Step 2-3: All extracted features are mapped to a unified dimension d through a fully connected layer to model feature alignment and interaction; finally, the features extracted from the RGB image are defined as spatial features S, and the features extracted from the optical flow map are defined as temporal features T.

[0034] Preferably, step 3 is specifically:

[0035] In the STFF module, the spatial feature S i is used as the query vector Query, and the temporal feature T i is used as the key Key and value Value respectively, and the correlation between traffic participants in the semantic space and time dynamics is modeled through cross-modal attention calculation;

[0036] is represented as:

[0037]

[0038] where φ represents the fully connected layer, σ represents the softmax operation, and F i is the fused feature representation.

[0039] Preferably, step 4 is specifically:

[0040] Step 4-1: for each target feature that has been fused in the current frame The DOA module models the target feature through two learnable parameter matrices and Combining a nonlinear activation function, the target feature is modeled and its corresponding importance weight is calculated; after normalization by the softmax operation, the importance weight is element-wise weighted with the original target feature, and finally converges to a single aggregated feature representation The calculation formula is as follows:

[0041]

[0042] where tanh is the hyperbolic tangent activation function, and represents element-wise multiplication, is the aggregated feature after fusion, represents the bias vector;

[0043] Step 4-2: concatenate the aggregated feature with the frame-level fusion feature corresponding to the current frame to obtain the final feature input X for time series modeling i :

[0044]

[0045] Preferably, the step 5 is specifically:

[0046] Step 5-1: at each time step i, the GRU receives the input feature X of the current frame i and the hidden state h′ aggregated at the last time step i-1 , after a series of gating calculations, the hidden state h of the current frame is output i The specific calculation process is as follows:

[0047] z i = δ(W z X i + U z h′ i-1 + B z )

[0048] r i = δ(W r X i + U r h′ i-1 + B r )

[0049]

[0050] where δ represents the sigmoid function, and represents element-wise multiplication, W zU z B z W r U r B r W h U h B h All are learnable parameters, z i Controlling the current state h i How much of it is derived from the hidden state h′ of the previous time step? i-1 Passed down, r i Indicates the current input X i and the hidden state h′ from the previous time step i-1 How to combine them This represents the candidate hidden state at the current moment;

[0051] Step 5-2: Obtain the hidden state h i Then, it is mapped to a risk prediction score through two fully connected layers, and the softmax function is used to output the probability α of the accident occurring in the current frame. i ;

[0052] α i =σ(φ(φ(h) i ))).

[0053] Preferably, the target detector is YOLOv5 or Faster R-CNN.

[0054] An electronic device includes: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to enable the electronic device to perform the traffic risk prediction method described above.

[0055] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described traffic risk prediction method.

[0056] A chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the traffic risk prediction method described above.

[0057] The beneficial effects of this invention are as follows:

[0058] This invention proposes a traffic accident risk prediction method that integrates spatiotemporal features and a dynamic attention mechanism, aiming to improve prediction accuracy and early warning capabilities in dynamic traffic scenarios. The method first introduces a spatiotemporal feature fusion module based on a cross-attention mechanism to effectively capture the synergistic relationship between semantic and motion information. It then dynamically adjusts the weights of each modality based on the traffic scenario, thereby improving the alignment and expressive power of cross-modal features. To avoid interference from irrelevant targets in historical frames, this invention further designs a dynamic object attention module that focuses only on key traffic participants with high risk in the current frame, thus enhancing the model's response timeliness and discrimination accuracy in real-world scenarios. Experimental results on the DAD dataset show that the proposed method significantly outperforms existing mainstream methods in both average accuracy and average warning time (mTTA), with AP exceeding 90% and mTTA reaching over 4.45 seconds, fully validating the application potential and engineering practical value of this invention in complex traffic environments. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the implementation of the traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention in this invention.

[0060] Figure 2 This is a dynamic graph showing the changes in average precision (AP) and mean time-to-accident (mTTA) during the training process of the proposed TSCFNet and the existing advanced model THAT-Net.

[0061] Figure 3 This is a graph showing the performance of the proposed TSCFNet and other comparative models in terms of their respective best AP models within the recall range [40%, 90%] for Time-to-Accident (TTA). Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0063] The purpose of this invention is to provide a traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention, addressing the problems of existing methods such as limited modality fusion strategies, insufficient interaction between semantic and motion information, and decreased prediction performance caused by the introduction of irrelevant historical features into target representation. This method integrates RGB images and optical flow graph information, designs a cross-modal spatiotemporal feature fusion mechanism, and introduces a dynamic attention strategy. It can intelligently filter high-risk targets based on the current traffic scenario and accurately model the dynamic relationships between multimodal information, thereby significantly improving the accuracy and generalization ability of accident risk prediction.

[0064] according toFigure 1 As shown, this invention designs a novel Two-Stream Cross-attention Fusion Network (TSCFNet) based on spatiotemporal feature fusion and dynamic attention, comprising the following steps:

[0065] Step 1: This invention first extracts optical flow information from a continuous RGB image sequence captured by a dashcam. Specifically, let the original input RGB image sequence be:

[0066] I = {I1, I2, ..., I} T}

[0067] To obtain motion information of traffic participants in image sequences, this invention employs a pre-trained RAFT (Recurrent All-Pairs Field Transforms) optical flow estimation model, which estimates the motion of each pair of consecutive image frames (I... t-1 ,I t Calculate the optical flow map F t The optical flow map sequence was obtained:

[0068] F = {F2, F3, ..., F} T}

[0069] Among them, F t =RAFT(I t-1 ,I t ), representing the pixel-level displacement vector field from frame t-1 to frame t, used to reflect the dynamic changes of objects in the scene.

[0070] Since the optical flow map is calculated from two consecutive image frames, its sequence length is one frame shorter than the original RGB image sequence, i.e., |{F}| = T-1. To ensure temporal alignment between the RGB images and the optical flow map in the subsequent two-stream network, this invention supplements the optical flow map sequence with a single frame of all-zero optical flow map F1 at the beginning, i.e.:

[0071] F1=0∈R H×W×2

[0072] Where H and W represent the height and width of the image, respectively, and the number of channels (2) indicates the optical flow components in the horizontal and vertical directions. The final aligned optical flow map sequence is as follows:

[0073] F′={F1,F2,…,F T}

[0074] At this point, the RGB image sequence I and the optical flow map sequence F′ are aligned in the time dimension, providing a consistent input basis for subsequent dual-stream feature extraction and fusion.

[0075] Step 2: After completing the construction of the image and optical flow map sequence, the present invention further extracts the spatial and motion features contained in the visual information to achieve deep modeling of the behavioral state of traffic participants.

[0076] First, this invention employs a pre-trained object detector (e.g., YOLOv5 or Faster R-CNN) to process each frame of the RGB image sequence. t Object detection is performed to obtain the bounding boxes of all traffic participants (such as pedestrians, vehicles, bicycles, etc.) in the frame. To reduce redundant information and focus on key objects, only the top N objects with the highest detection confidence are retained for subsequent analysis.

[0077] Next, based on the obtained target bounding box, this invention uses a pre-trained VGG-16 neural network to extract features from the RGB image and optical flow map, respectively. The extraction process includes two levels:

[0078] (1) Frame-level feature extraction: Global encoding of the entire frame image and optical flow map to capture the overall semantics and dynamic trends at the scene level;

[0079] (2) Target-level feature extraction: Using the detected bounding boxes, each target region is cropped and features are extracted to further focus on the semantic or motion changes of local key targets.

[0080] To unify feature dimensions, reduce computational complexity, and improve the efficiency of subsequent multimodal fusion, all extracted features are mapped to a unified dimension d through a fully connected layer for feature alignment and interactive modeling. Finally, the features extracted from the RGB image are defined as spatial features S, while the features extracted from the optical flow map are represented as temporal features T.

[0081] Step 3: To effectively fuse the spatial semantic information from the RGB image with the motion temporal information from the optical flow map, this invention designs a Spatio-Temporal Feature Fusion Module (STFF), the structure of which is as follows: Figure 1 As shown, this module aims to uncover the deep correlation between spatial and temporal features, and adaptively adjust the fusion strategy between modalities by combining semantic differences in different traffic scenarios, thereby enhancing the discriminative power of the final feature representation and improving the model's ability to identify and perceive potential accident risks.

[0082] Considering that spatial and temporal features originate from different data modalities (RGB images and optical flow maps), their semantic distributions differ. Directly concatenating or adding them may lead to inconsistencies or semantic shifts, affecting the fusion effect. Therefore, to achieve more effective feature alignment and fusion, this invention introduces a cross-attention mechanism in the STFF module. By guiding information between modalities, it enhances the targeting and accuracy of the fusion.

[0083] Specifically, in the STFF module, spatial feature S i As a query vector, with time feature T i Using these as keys and values ​​respectively, cross-modal attention computation is employed to model the correlation between traffic participants in semantic space and temporal dynamics. This structure enables the model to selectively focus on the most relevant dynamic behavioral changes based on current spatial semantic information, thereby improving the accuracy of risk factor identification.

[0084] The calculation method for this fusion process can be expressed as follows:

[0085]

[0086] Where φ represents a fully connected layer, σ represents the softmax operation, and F i Feature representation after fusion.

[0087] Step 4: In real-world traffic scenarios, certain traffic participants (such as vehicles suddenly accelerating or pedestrians crossing the road) often pose a higher risk potential due to their high speed or abrupt changes in behavior. Therefore, accurately identifying the most potentially threatening targets in the current frame is crucial for improving the accuracy and robustness of accident warnings.

[0088] Traditional methods typically rely on target features from historical frames to model object importance, attempting to enhance the current feature representation through temporal continuity. However, in highly dynamic and complex traffic environments, some objects may have disappeared in the current frame. Continuing to use their historical features may introduce outdated or irrelevant information, causing noise interference and affecting the model's judgment performance.

[0089] To address the aforementioned problems, this invention proposes a Dynamic Object Attention Module (DOA), such as... Figure 1 As shown, this module dynamically evaluates the importance of each object based solely on the target features detected in the current frame through an attention mechanism, thereby effectively identifying the key targets that have the greatest impact on risk assessment at the current moment. It does not rely on historical frame information, thus avoiding the introduction of redundant or invalid information.

[0090] Specifically, for each target feature that has been fused in the current frame The DOA module uses two learnable parameter matrices. and By incorporating a non-linear activation function, the target features are modeled, and their corresponding importance weights are calculated. After being normalized using a softmax operation, these weights are element-wise weighted with the original target features, ultimately converging into a single aggregated feature representation. The calculation formula is as follows:

[0091]

[0092] Where tanh is the hyperbolic tangent activation function, and ⊙ denotes element-wise multiplication. This refers to the characteristics of the fused polymer.

[0093] Subsequently, the aggregated target features Frame-level fusion features corresponding to the current frame The features are concatenated to obtain the final feature input X used for temporal modeling. i :

[0094]

[0095] This feature represents the fusion of global scene information and key object information, possessing stronger semantic expression and risk perception capabilities. It can be used as input to subsequent GRU networks to model the temporal evolution process in video sequences.

[0096] Step 5: To capture the dynamic evolution features between consecutive frames and more accurately identify potential traffic accident risks, this invention employs a Gated Recurrent Unit (GRU) to perform temporal modeling on the fused feature sequence. GRU is a lightweight and efficient recurrent neural network structure capable of mitigating gradient vanishing and modeling long-term dependencies. Internally, it controls the degree of fusion between historical information and the current input through update and reset gates, thereby effectively extracting the temporal dynamic features of traffic participant behavior changes. At each time step i, the GRU receives the input feature X of the current frame. i and the hidden state h′ obtained from the aggregation of the previous time step i-1 After a series of gating calculations, the hidden state h of the current frame is output. i The specific calculation process is as follows:

[0097] z i =δ(W z X i +U zh′ i-1 +B z )

[0098] r i =δ(W r X i +U r h′ i-1 +B r )

[0099]

[0100] Where δ represents the sigmoid function, ⊙ represents element-wise multiplication, and all weight matrices and bias terms (such as W) are also represented. z U h B r (etc.) are all learnable parameters.

[0101] Get the hidden state h i Subsequently, this paper maps the risk prediction score to a risk prediction score using two fully connected layers, and uses the softmax function to output the probability α of the accident occurring in the current frame. i .

[0102] α i =σ(φ(φ(h) i )))

[0103] Through the aforementioned time-series modeling mechanism, the model can effectively capture potential abnormal changes in the behavioral evolution process, thereby improving the accuracy and robustness of accident risk prediction.

[0104] Example:

[0105] refer to Figure 2 As shown, Figure 2 This figure illustrates the dynamic relationship between AP (Average Precision) and mTTA (Mean Time To Advance) of our proposed method TSCFNet and the existing state-of-the-art model THAT-Net during training. Based on the DAD dataset, each point represents the model's AP and mTTA performance at a specific training epoch, thus visually demonstrating the model's performance characteristics at different training stages.

[0106] Overall, both models primarily improve AP in the early stages of training to ensure accurate identification of high-risk events. As training progresses, the models gradually optimize mTTA, thereby achieving earlier incident warnings. Compared to THAT-Net, TSCFNet demonstrates superior performance in several key stages, including the initial training stage, the AP peak stage, the AP-mTTA balancing stage, and the maximum mTTA stage, showcasing excellent learning ability and stability.

[0107] Specifically, TSCFNet achieved an AP of 67.24% and a median TTA of 1.79 seconds in the first training epoch, demonstrating strong initial convergence ability. In the later stages of training, TSCFNet's performance continued to improve. The rightmost point in the figure corresponds to the model's highest AP (95.57%) and corresponding mTTA (3.96 seconds) achieved during training, representing improvements of approximately 8% and 0.8 seconds respectively compared to THAT-Net, significantly outperforming the comparison methods. Furthermore, the topmost point in the figure represents the model's longest mTTA (4.97 seconds) and corresponding AP (71.84%), significantly extending the warning time while maintaining high accuracy, further validating the superiority of this method in balancing prediction accuracy and lead time.

[0108] Overall, TSCFNet maintained a high level of performance stability throughout the training process, with AP consistently above 65% and mTTA remaining stable above 1.5 seconds, demonstrating a strong ability to detect accident risks.

[0109] As shown in Table 1, the experimental comparison results of each model on the DAD dataset are presented. The table shows that TSCFNet outperforms all other methods in both the mean accuracy (AP) and mean time to warning (mTTA) metrics, demonstrating the best overall performance. In particular, compared to THAT-Net, the current mainstream method that fuses RGB images and optical flow maps, TSCFNet improves AP by 13% and mTTA by 0.3 seconds, reaching 90.10% and 4.47 seconds respectively.

[0110] This result demonstrates that the proposed spatiotemporal feature dynamic fusion mechanism has significant advantages in terms of accuracy and early warning lead time, and can more effectively support the early prediction of traffic accidents.

[0111] Table 1 compares the average accuracy and warning time of different models on the DAD dataset.

[0112]

[0113]

[0114] As shown in Table 2, the performance of each method on the best AP model on the DAD dataset is compared. Experimental results show that TSCFNet outperforms existing mainstream methods in both AP and mTTA metrics, demonstrating a leading performance advantage. Specifically, the best TSCFNet model achieves 95.57% AP and 3.96 seconds mTTA, representing improvements of 8.3% and 0.82 seconds respectively compared to the fusion method THAT-Net.

[0115] This improvement is not only reflected in the accuracy of predictions, but more importantly, in the significant extension of warning time. In real-world driving scenarios, an extra second of reaction time is crucial for drivers, effectively improving their response speed to emergencies and thus enhancing overall road safety and accident prevention capabilities.

[0116] Table 2 Comparison of best accuracy and average warning time for different models on the DAD dataset.

[0117]

[0118]

[0119] To further evaluate the model's early warning capability under different recall rates Figure 3 This demonstrates the performance of the best AP model in TTA (Time to Action) when recall is between 40% and 90%. Figure 3 As can be seen, the proposed model achieves superior TTA performance compared to existing methods at most recall levels, demonstrating good stability and generalization ability. Particularly when the recall reaches 90%, TSCFNet's TTA improves to 3.82 seconds, significantly outperforming other methods and showcasing its leading performance in high-recall scenarios. This experiment further validates the practical value of this method in providing sufficient warning time while maintaining high recall.

[0120] In summary, this invention proposes a traffic accident risk prediction method that integrates spatiotemporal features and a dynamic attention mechanism, aiming to improve prediction accuracy and early warning capabilities in dynamic traffic scenarios. This method first introduces a spatiotemporal feature fusion module based on a cross-attention mechanism to effectively capture the synergistic relationship between semantic and motion information. It then dynamically adjusts the weights of each modality based on the traffic scenario, thereby improving the alignment and expressive power of cross-modal features. To avoid interference from irrelevant targets in historical frames, this invention further designs a dynamic object attention module that focuses only on key traffic participants with high risk in the current frame, thus enhancing the model's response timeliness and discrimination accuracy in real-world scenarios. Experimental results on the DAD dataset show that the proposed method significantly outperforms existing mainstream methods in both average accuracy and average warning time (mTTA), with AP exceeding 90% and mTTA reaching over 4.45 seconds, fully validating the application potential and engineering practical value of this invention in complex traffic environments.

Claims

1. A traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention, characterized in that, Includes the following steps: Step 1: Extract optical flow information from the continuous RGB image sequence captured by the dashcam; Step 2: Extract the spatial and motion features contained in the visual information to achieve in-depth modeling of the behavioral state of traffic participants; Step 3: Design the spatiotemporal feature fusion module STFF; Step 4: Use the Dynamic Object Attention (DOA) module to identify the most potentially threatening target in the current frame; Step 5: Use a gated recurrent unit (GRU) to perform time-series modeling on the fused feature sequence.

2. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 1, characterized in that, Step 1 specifically involves: Step 1-1: Let the original input RGB image sequence be: I={I1,I2,…,I T } Among them, I t This represents the RGB image frame acquired at time t, with a total of T frames, t = 1…T; Steps 1-2: Using a pre-trained RAFT optical flow estimation model, for each pair of consecutive image frames (I... t-1 ,I t Calculate the optical flow map F t The optical flow map sequence was obtained: F={F2,F3,…,F T } Among them, F t =RAFT(I t-1 ,I t ), representing the pixel-level displacement vector field from frame t-1 to frame t, used to reflect the dynamic changes of objects in the scene; Steps 1-3: Since the optical flow map is calculated from two consecutive images, its sequence length is one frame shorter than the original RGB image sequence, i.e., |{F}| = T-1; a single frame of all-zero optical flow map F1 is added at the beginning of the optical flow map sequence, i.e.: F1=0∈R H×W×2 Where H and W represent the height and width of the image, respectively, and the number of channels is 2 to represent the optical flow components in the horizontal and vertical directions; Steps 1-4: The final aligned optical flow map sequence is as follows: F′={F1,F2,…,F T } The RGB image sequence I and the optical flow map sequence F′ are aligned in the time dimension.

3. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 2, characterized in that, Step 2 specifically involves: Step 2-1: Use a pre-trained object detector to process each frame of the RGB image sequence. t Perform object detection to obtain the bounding box set of all traffic participants in the frame; retain the top N objects with the highest detection confidence. Step 2-2: Use a pre-trained VGG-16 neural network to extract features from the RGB image and optical flow map respectively. The extraction process includes two levels: (1) Frame-level feature extraction: Global encoding of the entire frame image and optical flow map to capture the overall semantics and dynamic trends at the scene level; (2) Target-level feature extraction: Using the detected bounding boxes, each target region is cropped and its features are extracted. Further focus on the semantic or motion changes of key local targets; Steps 2-3: All extracted features are mapped to a unified dimension d through a fully connected layer for feature alignment and interactive modeling; finally, the features extracted from the RGB image are defined as spatial features S, and the features extracted from the optical flow map are defined as temporal features T.

4. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 3, characterized in that, Step 3 specifically involves: In the STFF module, spatial feature S i As the query vector Query, with time feature T i Using these as keys and values ​​respectively, cross-modal attention computation is used to model the relationships between traffic participants in semantic space and temporal dynamics. Represented as: Where φ represents a fully connected layer, σ represents the softmax operation, and F i This represents the feature representation after fusion.

5. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 4, characterized in that, Step 4 specifically involves: Step 4-1: For each target feature that has been fused in the current frame The DOA module uses two learnable parameter matrices. and By combining nonlinear activation functions, the target features are modeled and their corresponding importance weights are calculated. After the importance weights are normalized by the softmax operation, they are element-wise weighted with the original target features, and finally converged into a single aggregated feature representation. The calculation formula is as follows: Where tanh is the hyperbolic tangent activation function, and ⊙ denotes element-wise multiplication. The aggregated features after fusion Represents the bias vector; Step 4-2: Aggregate features Frame-level fusion features corresponding to the current frame The features are concatenated to obtain the final feature input X used for temporal modeling. i :

6. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 5, characterized in that, Step 5 specifically involves: Step 5-1: At each time step i, the GRU receives the input features X of the current frame. i and the hidden state h′ obtained from the aggregation of the previous time step i-1 After a series of gating calculations, the hidden state h of the current frame is output. i The specific calculation process is as follows: z i =δ(W z X i +U z h′ i-1 +B z ) r i =δ(W r X i +U r h′ i-1 +B r ) Where δ represents the sigmoid function, ⊙ represents element-wise multiplication, and W z U z B z W r U r B r W h U h B h All are learnable parameters, z i Controlling the current state h i How much of it is derived from the hidden state h′ of the previous time step? i-1 Passed down, r i Indicates the current input X i and the hidden state h′ from the previous time step i-1 How to combine them This represents the candidate hidden state at the current moment; Step 5-2: Obtain the hidden state h i Then, it is mapped to a risk prediction score through two fully connected layers, and the softmax function is used to output the probability α of the accident occurring in the current frame. i ; a i =σ(φ(φ(h i )))。 7. The traffic risk prediction method based on spatiotemporal feature fusion and dynamic attention according to claim 6, characterized in that, The target detector is either YOLOv5 or Faster R-CNN.

8. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A chip, characterized in that, include: A processor for retrieving and running a computer program from memory, causing a device on which the chip is mounted to perform the method as described in any one of claims 1 to 7.

Images