Traffic risk estimation method and system based on splicing frame and causal attention
By combining frame interpolation and causal attention in a traffic risk estimation method, and integrating grayscale weight maps and depth maps, the ResNet-50 network and sparse causal attention module are improved. This solves the problem of inaccurate target feature extraction in complex traffic scenarios and achieves high-precision risk estimation.
Patent Information
- Application Number
- CN202511712353.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-20
- Publication Date
- 2026-02-10
AI Technical Summary
In complex traffic scenarios, existing technologies are prone to interference with foreground target feature extraction due to changes in lighting, occlusion, and similar textures. This makes it difficult to effectively balance background interference with the dynamic information of the subject, resulting in insufficient accuracy and efficiency in risk estimation.
A frame-complementing strategy is adopted to fuse the grayscale weight map and the depth map. The channel and spatial attention are enhanced by improving the ResNet-50 network, and the spatiotemporal correlation is enhanced by combining the sparse causal attention module, which suppresses background interference and improves the accuracy of target feature extraction.
Significantly improves the accuracy and robustness of risk estimation in complex traffic environments, reducing the risk of misjudgment caused by cluttered backgrounds.
Smart Images
Figure CN121505556A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of traffic scenario risk assessment technology, and in particular to a traffic risk estimation method and system based on frame interpolation and causal attention. Background Technology
[0002] Traffic scenario risk estimation is a core prerequisite for intelligent driving systems to avoid safety hazards. Its core lies in accurately assessing scenario risks by integrating the spatial location information of traffic participants with temporal information about their actions and causal relationships. This technology is not only a key support for understanding intelligent driving scenarios but also provides an important reference for the temporal and spatial information fusion of general video scenarios.
[0003] With the development of computer technology, driving systems have evolved from traditional collision warning and active avoidance systems to intelligent perception systems based on deep learning. Currently, the industry exhibits two major trends: first, reducing reliance on expensive, difficult-to-install, and easily damaged precision equipment such as multi-camera systems and LiDAR; and second, leveraging end-to-end networks and large models to enhance spatiotemporal information fusion to improve estimation accuracy.
[0004] Traditional methods often employ a two-stream network architecture to capture spatial and temporal information separately, requiring separate training of the two branches during training, or relying on textual supplementation to assist in temporal feature extraction, resulting in significant resource consumption. Furthermore, in complex traffic scenarios, changes in lighting, occlusion, and similar textures can easily interfere with foreground target feature extraction. Existing solutions often ignore the differences between different targets at the same level and struggle to effectively balance the extraction of background interference and subject dynamic information, resulting in insufficient spatiotemporal consistency and ultimately limiting the accuracy and efficiency of risk estimation. Summary of the Invention
[0005] To address the aforementioned issues, this invention proposes a traffic risk estimation method and system based on frame patching and causal attention. This method incorporates a portion of the background as near-black patches in sparse causal attention, effectively suppressing background interference in traffic scenes, improving the accuracy of target feature extraction, and significantly enhancing the accuracy and robustness of risk estimation in complex traffic environments.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a traffic risk estimation method based on frame interpolation and causal attention, comprising: RGB spatial video data of traffic scenes is acquired, and the original spatial map is obtained through frame-level segmentation. The original spatial map is then preprocessed to obtain a grayscale weight map and a single-channel depth map. Based on the frame stitching strategy, the keyframes extracted from the original spatial map are fused with the grayscale weight map and the depth map is supplemented to obtain five-dimensional input data. Five-dimensional input data is fed into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention. An SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, enhancing spatiotemporal correlation. Dimensional reshaping is performed before the feature map is input to the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information. Finally, the risk assessment result is obtained through a global average pooling layer and a fully connected layer.
[0007] Secondly, the present invention provides a traffic risk estimation system based on frame interpolation and causal attention, comprising: The raw frame processing unit is configured to acquire RGB spatial video data of a traffic scene, obtain a raw spatial map through frame-level segmentation, and preprocess the raw spatial map to obtain a grayscale weight map and a single-channel depth map. The frame stitching processing unit is configured to, based on the frame stitching strategy, fuse the keyframes extracted from the original spatial map with the grayscale weight map and supplement the depth map to obtain five-dimensional input data. The risk estimation unit is configured to input five-dimensional input data into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention; an SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, enhancing spatiotemporal correlation; dimensionality reshaping is performed before the feature map is input to the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information; finally, the risk estimation result is obtained through a global average pooling layer and a fully connected layer.
[0008] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the traffic risk estimation method based on frame patching and causal attention described in the first aspect.
[0009] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the traffic risk estimation method based on frame patching and causal attention described in the first aspect.
[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention utilizes frame-level segmentation combined with YOLOv3 to generate grayscale weight maps and Depth Anything v2 to obtain depth maps, supplementing planar spatial and 3D depth information. A frame-mating strategy is then employed to select key frames and fuse the two maps, suppressing background interference, reducing redundant computation, and enhancing the target features of traffic participants, thus addressing the problem of insufficient information from traditional RGB images alone. Simultaneously, the improved ResNet-50 incorporates a CBAM module in the residual blocks to enhance channel and spatial attention, improving target extraction accuracy. An SCA module is introduced after the residual layer, along with dimensionality reshaping, adapting to static networks processing five-dimensional temporal data while accurately capturing inter-frame spatiotemporal correlations and avoiding frame order disorder. Finally, the output is processed through pooling and fully connected layers, significantly improving the accuracy and robustness of risk classification in complex traffic scenarios and reducing the risk of misjudgment caused by cluttered backgrounds.
[0011] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.
[0013] Figure 1 The main flowchart of a traffic risk estimation method based on frame interpolation and causal attention provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a traffic risk estimation method based on frame patching and causal attention, provided in an embodiment of the present invention. Detailed Implementation
[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0015] Example 1 like Figure 1 As shown, this embodiment discloses a traffic risk estimation method based on frame patching and causal attention, including the following steps: S1: Acquire RGB spatial video data of traffic scene, obtain the original spatial map through frame-level segmentation, and preprocess the original spatial map to obtain grayscale weight map and single-channel depth map; S2: Based on the frame stitching strategy, the keyframes extracted from the original spatial map are fused with the grayscale weight map and the depth map is supplemented to obtain five-dimensional input data; S3: Input the five-dimensional input data into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention. An SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, thereby enhancing spatiotemporal correlation. Dimensional reshaping is performed before the feature map is input into the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information. Finally, the risk estimation result is obtained through a global average pooling layer and a fully connected layer.
[0016] Next, combined Figure 2 This embodiment provides a detailed description of a traffic risk estimation method based on frame patching and causal attention.
[0017] (a) Data preprocessing The RGB spatial video data of the traffic scene is acquired and then preprocessed in three ways: frame-level segmentation, weighting, and depth estimation. The data is then input into the model for evaluation.
[0018] First, frame-level segmentation is performed. The continuous RGB spatial video is divided into continuous video frame images, with each video segmented into 20 consecutive video frame images, to obtain the original spatial map.
[0019] Next, spatial information of traffic participants is extracted. Continuous video frame images are input into the YOLOv3 network. The YOLOv3 network uses multi-scale object detection to select targets in the video frames. A weight map is then created based on the object detection results to extract spatial information of traffic participants in the traffic scene.
[0020] Initialize the weighted image as a single-channel grayscale image with a grayscale value of 128, and set 255... The target confidence score is used as the pixel value within the target detection bounding box. Specifically, it is obtained by multiplying the target detection confidence score (ranging from 0 to 1) by 255 (the maximum pixel value in the grayscale image). A higher confidence score results in a pixel value closer to 255 (pure white, highlight effect), thus reflecting the target's credibility through the grayscale value.
[0021] After this processing, a grayscale weight map is obtained: the target area is highlighted, and the remaining pixels are all medium gray with a grayscale value of 128.
[0022] Next, depth information of traffic participants is extracted. In this embodiment, the Depth Anything v2 monocular image depth estimation model is used to estimate depth and obtain a single-channel depth map.
[0023] The above preprocessing operations address the issue of insufficient depth perception when relying solely on RGB images. The overall preprocessing not only filters out key visual and spatial data but also reduces background interference, providing high-quality, highly targeted input data for subsequent frame stitching, fusion, and risk estimation.
[0024] (ii) Background separation based on frame interpolation strategy to fuse strong spatial information Furthermore, by employing a frame-splitting strategy, spatial and depth information are combined into strong spatial information that contains both planar spatial location and depth information, thereby separating the background.
[0025] First, consecutive RGB video frames are sampled using a strategy of evenly selecting 5 frames from 20 frames. The sampled 5 RGB video frames are then combined with the grayscale weight maps of their corresponding 5 frames from the preprocessing stage. Simultaneously, the single-channel depth maps of their corresponding 5 frames from the preprocessing stage are concatenated three times with the image itself along the channel dimension to obtain a three-channel grayscale image.
[0026] This method involves stitching a portion of the background with unchanging black blocks, reducing background interference, and also allows the highlighted subject to participate in more stitching processes, showcasing richer dynamic information about the subject.
[0027] Specifically, the processing flow for consecutive video frame inputs in the frame interpolation strategy is as follows: Five frames are selected evenly from 20 frames. Let the set of selected frames be... but: (1) Selected frames The original spatial image is element-wise multiplied with the grayscale weighted image, and then the image size is adjusted by resizing. (2) in, For the first Frame space map, For the first The grayscale weight map corresponding to the original spatial map of the frame. For the weighted number of Frame space graph.
[0028] In the grayscale weighted image, the background pixels have a grayscale value of 128, while the target area pixels have a grayscale value of 255. Confidence level (close to 255, high brightness). When the original spatial image is element-wise multiplied with the weight map, the original pixels in the background region are multiplied by 128 (medium gray weight), while the original pixels in the target region are multiplied by a high brightness weight close to 255. During subsequent stitching, the brightness of the background region is significantly suppressed after calculation, appearing as a black block with almost no change in detail; while the target region retains clear information due to its high weight, making the target stand out more and ultimately reducing the interference of the background on the task.
[0029] Assume each frame The three-channel depth map is but: (3) in, Indicates the first Single-channel depth map of a frame, This indicates splicing along the channel dimension.
[0030] The final input list is obtained by stacking all weighted spatial maps and stitched depth maps along the frame dimension (0th dimension). : (4) The data consists of five dimensions: time dimension T (number of video frames), batch dimension B, channel dimension C, height dimension H, and width dimension W.
[0031] This integrates spatial and depth information at the input level and separates background interference information to highlight traffic target participants.
[0032] This embodiment selects 5 key frames evenly from 20 video frames, integrates grayscale weight map and depth map information, strengthens the target of traffic participants and suppresses background interference by multiplying element by element. The three-channel stitched depth map is adapted for subsequent processing. The frame dimension stacking integrates the dynamic and spatial depth information of multiple frames. This not only avoids the computational burden caused by redundant frames, but also highlights the credible area of the target and supplements the depth dimension, allowing the model to focus on the core traffic participants. This effectively improves the accuracy and robustness of target recognition in complex traffic scenarios and reduces the risk of misjudgment caused by cluttered backgrounds.
[0033] (III) Risk estimation of spatiotemporal correlation enhancement based on sparse causal attention 1. Backbone Network: This embodiment uses ResNet-50 as the basic estimation model. The ResNet-50 includes an input layer, four residual layers (Conv2_x, Conv3_x, Conv4_x, Conv5_x), a global average pooling layer, and a fully connected layer.
[0034] The residual layer contains a total of 16 residual blocks, of which Conv2_x contains 3, Conv3_x contains 4, Conv4_x contains 6, and Conv5_x contains 3.
[0035] All residual blocks use the Bottleneck structure, and after the third BatchNormalization (BN) of each Bottleneck residual structure, a CBAM (Convolutional Block Attention Module) is added.
[0036] Convolutional attention combines channel attention and spatial attention, enabling precise focus on key features of traffic participants in traffic scenarios, suppression of irrelevant background interference, and improved model accuracy in extracting target features. Both training and validation processes utilize cross-entropy loss as the loss function.
[0037] 2. Enhanced spatiotemporal correlation: To enhance the model's ability to focus on the spatial features of a single frame while also considering the temporal evolution of these features across consecutive frames, this embodiment introduces an SCA (Sparse Causal Attention) module after each residual layer of the improved ResNet-50.
[0038] First, since ResNet-50 is a static image classification network, it cannot directly process five-dimensional data. The five-dimensional data must first be reduced to a four-dimensional format that the model can recognize. Therefore, the five-dimensional data must first be reduced to a four-dimensional format. In the list of five-dimensional data, the product of the time dimension T and the batch dimension B is merged into a batch dimension b, which is then input into a ResNet-50 residual layer. After residual block computation and CBAM attention enhancement, the output is a feature map containing four dimensions: batch b, channel c, height h, and width w. .
[0039] While such dimensionality reduction adapts to the model input, it cannot guarantee that the frame sequence information is not corrupted.
[0040] In order for the SCA module to accurately identify the sequence of consecutive frames in each batch and avoid frame disorder affecting spatiotemporal correlation capture, it is necessary to separate the actual batch dimension B and time dimension T.
[0041] Therefore, in the feature map Before inputting into the SCA module, use the inverse operation to find the actual batch dimension value. : (5) Where T is the time dimension, and its value is the length of the input video frame.
[0042] Then the actual batch dimension The first dimensional reshaping is completed by multiplying and merging the height h and width w using the channel c and time dimension T. (6) in, This represents a dimensional reshaping tool, which will Reshaping into dimensions A four-dimensional tensor. Then, the dimensional order is adjusted to perform a dimensional transpose: (7) in, Indicates will The dimensions with index 2 and index 3 are swapped. and .
[0043] The next step will be After merging with T and performing another dimension reshaping, input the result into the SCA module: (8) This explicit dimensional reorganization ensures that the SCA input can be accurately identified as the correct frame value, avoiding the input of out-of-order consecutive frames.
[0044] At this point, the input to the SCA module is three-dimensional ( ) Feature map, and video frame length T.
[0045] This embodiment addresses the frame order disorder issue in adapting temporal data to static networks through dimensional reshaping before inputting to SCA. Specifically, it first splits the real batch dimension B and the time dimension T to correct the frame sequence information confusion caused by dimensionality reduction of the five-dimensional data; then, it merges the height and width dimensions, adjusts the dimension order, and reshapes the feature map a second time, converting it into a format recognizable by SCA. This process clarifies the hierarchical relationship of the frame sequence, ensuring that SCA accurately captures the temporal logic of consecutive frames in each batch, avoiding frame disorder from disrupting spatiotemporal correlation capture, laying a data foundation for SCA to strengthen the dynamic correlation between frames through sparse causal attention, and ensuring the accuracy of subsequent target motion feature analysis in traffic scenes.
[0046] After processing by the SCA module, the output is... For 3D with added attention weights ( Feature map.
[0047] To avoid affecting subsequent residual layer processing, the output of the SCA module also needs to be... The first dimension is broken down to restore the true batch and time dimensions, clarifying the hierarchical relationship of the frame sequence: (9) Then combine dimension c with dimension h After swapping positions, By merging with T, the dimensions of the feature map are adjusted to the [b, c, h, w] format acceptable to the next residual layer, ensuring data flow compatibility. (10) This allows it to be input into the next residual layer.
[0048] In this embodiment, to ensure that the features of each layer can fully capture spatiotemporal correlations, an SCA module is introduced after each residual layer of ResNet-50. After the SCA module enhances spatiotemporal attention, the feature map dimensions need to be restored to a standard format that the residual layer can process. Therefore, the aforementioned dimensionality adjustment operation is performed to ensure successful input into the next residual layer, thereby achieving multi-scale, end-to-end enhancement of target motion features in traffic scenes.
[0049] For dimensional reshaping between residual layers, this embodiment first splits the first dimension of the SCA output to restore the true batch and time dimensions, clarifying the hierarchical relationship of the frame sequence and avoiding frame order disorder from affecting subsequent feature processing. Then, through dimension exchange and merging, the feature map format is adjusted to the [b, c, h, w] standard format acceptable to the residual layer. This process not only preserves the spatiotemporal correlation features enhanced by the SCA module, but also solves the dimensional incompatibility problem between the SCA output and the residual layer input, ensuring smooth data flow in the network and providing a guarantee for the subsequent continuous extraction and optimization of traffic scene target features by the residual layer.
[0050] The SCA module first processes the input image according to n The pixels are grouped into blocks of n. Within each block, SCA can simultaneously query the K and V values of the current and historical frames. Then, the K and V values of the current frame (let's say frame i) are concatenated with the K and V values of frame ij, frame i-j+1, frame i-j+2, ..., frame i-1, and so on, for the query value Q of the current frame to use for querying.
[0051] The final result is a feature map that integrates the local spatial aggregation features of the current and historical frames and has spatiotemporal correlation attention weights, providing feature support for subsequent traffic scenario risk correlation analysis.
[0052] In this embodiment, according to n n-pixel blocks are used to generate key-value pairs (K and V), which are then linked to the current and historical frames for querying by Q. This accurately captures the spatiotemporal relationships between frames, clearly depicting the movement trajectories and dynamic interactions of traffic participants. This process overcomes the shortcomings of traditional static networks in handling time-series data, allowing the model to more accurately perceive dynamic changes in traffic scenarios and providing more comprehensive spatiotemporal features for risk estimation.
[0053] 3. Risk Category Assessment: The output of the last SCA module is then processed through a global average pooling layer and a fully connected layer to obtain the estimation result. The specific processing flow is as follows: The output of the last SCA module is reshaped into five-dimensional features: batch, time, channel, height, and width. (11) in, This is a characteristic output by the SCA module.
[0054] The replacement dimension brings the channel dimension forward and the time dimension backward, then reshapes it again, merging the batch and channel dimensions to prepare for time dimension compression: (12) (13) in, Indicates will The positions of each dimension are changed according to the corresponding index position, thus completing the time dimension reshaping and dimension adjustment of the output features of the SCA module.
[0055] The time dimension is compressed to 1 using a single-channel convolution, and then the dimension of size 1 is removed to complete feature normalization. This removes the time dimension T, which represents the duration of the input video and is identical for every sample. This is because, in data processing, since each sample has the same video duration, the time dimension T is repetitive and lacks distinguishing information for all samples. Therefore, removing it simplifies the data structure, avoids redundant dimensions interfering with subsequent model calculations, and improves processing efficiency. (14) Separate batch and channel features based on batch and channel reshaping characteristics: (15) After passing through a size of 1 The adaptive global average pooling of 1 yields the global feature vector: (16) in, For sample index, For channel indexing, , This is the spatial location index for the sample.
[0056] Finally, the global feature vector is mapped to the evaluation class space through a linear transformation via a fully connected layer, outputting the unnormalized classification result: (17) in, For category indexing For the number of categories, The weight matrix is the first... Line number Column elements, Paranoia vector The first in Each element.
[0057] To verify the effectiveness of this embodiment, model training and testing were conducted on a high-performance server. The server system was Ubuntu 22.04 with an Intel(R) Xeon(R) Platinum 8836C CPU with a clock speed of 2.30GHz. Two NVIDIA GeForce RTX 4090 graphics cards with 24GB of VRAM and a maximum power consumption of 450W were used. The system employed CUDA 11.0, the PyTorch framework, the conda environment, and Python 3.9. The dataset used was the TRA dataset, which includes videos from mainstream video websites, DashcamVideo, and the DADA dataset, as well as some self-shot videos using a DJI Osmo camera and a dashcam installed in a car. Weighted confidence normalization was used to calculate the static complexity, and Hausdorff distance was used to calculate the dynamic complexity. The risk level of the dataset was assessed by weighted summation of the static and dynamic complexities.
[0058] Table 1. Accuracy of model risk estimation under different conditions;
[0059] Table 1 shows the accuracy of model risk estimation under different conditions (including whether a single spatial flow structure or a dual spatial flow and optical flow structure is used, whether depth information is added, whether a sparse causal attention module (SCA) is used, and whether the sampling frame selection strategy uses a frame interpolation strategy, no frame interpolation strategy, or full frame selection strategy).
[0060] The experimental results show that in the case of a single spatial flow, without SCA and without combining depth and spatial information to form strong spatial information, the accuracy is only 62.2%. If only SCA is used, without depth information or frame patching strategies, neither the full frame selection nor the 20-frame-select-5-frame sampling strategy achieves an accuracy of 80%. In the case of a dual-flow spatial and optical flow structure, the 20-frame-select-5-frame sampling strategy achieves an accuracy of 83.2%. However, using frame patching and SCA, the single spatial flow structure surpasses the dual-flow structure in accuracy by 3.4%. Adding depth and spatial information to form strong spatial information further improves the accuracy by 1.2%. Combining depth and spatial information into strong spatial information, along with frame patching and the SCA module, significantly improves the accuracy of model evaluation.
[0061] This specific embodiment combines frame-level segmentation with YOLOv3 to generate grayscale weight maps and Depth Anything v2 to obtain depth maps, supplementing planar space and 3D depth information and addressing the problem of insufficient spatial information when using only RGB images. Then, a frame-padding strategy is used to select key frames and fuse the weight map and depth map, suppressing background interference and avoiding redundant frames that increase computational burden. Secondly, an improved ResNet-50 is adopted, adding a CBAM module to the residual block to enhance channel and spatial attention, solving the problem of inaccurate target feature extraction. An SCA module is added after the residual layer, coupled with multi-step dimensional reshaping, which is suitable for processing temporal data with static networks and accurately captures inter-frame spatiotemporal correlations, overcoming the shortcomings of traditional networks in handling video temporal information, ultimately improving the accuracy and robustness of risk estimation in complex traffic scenarios.
[0062] Example 2 This embodiment provides a traffic risk estimation system based on frame patching and causal attention, including: The raw frame processing unit is configured to acquire RGB spatial video data of a traffic scene, obtain a raw spatial map through frame-level segmentation, and preprocess the raw spatial map to obtain a grayscale weight map and a single-channel depth map. The frame stitching processing unit is configured to, based on the frame stitching strategy, fuse the keyframes extracted from the original spatial map with the grayscale weight map and supplement the depth map to obtain five-dimensional input data. The risk estimation unit is configured to input five-dimensional input data into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention; an SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, enhancing spatiotemporal correlation; dimensionality reshaping is performed before the feature map is input to the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information; finally, the risk classification result is obtained through a global average pooling layer and a fully connected layer.
[0063] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the traffic risk estimation method based on frame patching and causal attention as described in Embodiment 1 above.
[0064] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the traffic risk estimation method based on frame patching and causal attention as described in Embodiment 1 above.
[0065] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.
[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A traffic risk estimation method based on frame interpolation and causal attention, characterized in that, include: RGB spatial video data of traffic scenes is acquired, and the original spatial map is obtained through frame-level segmentation. The original spatial map is then preprocessed to obtain a grayscale weight map and a single-channel depth map. Based on the frame stitching strategy, the keyframes extracted from the original spatial map are fused with the grayscale weight map and the depth map is supplemented to obtain five-dimensional input data. Five-dimensional input data is fed into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention. An SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, enhancing spatiotemporal correlation. Dimensional reshaping is performed before the feature map is input to the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information. Finally, the risk classification result is obtained through a global average pooling layer and a fully connected layer.
2. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The frame-level segmentation divides continuous RGB space video data into a preset number of continuous video frame images.
3. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The preprocessing specifically involves: generating a grayscale weight map for each frame's original spatial map through target detection, wherein the target region is assigned a bright pixel value according to the confidence level, the background is a fixed grayscale value, and a single-channel depth map is obtained through depth estimation.
4. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The frame-merging strategy extracts keyframes from the original spatial map and fuses them with a grayscale weight map to supplement a depth map, resulting in five-dimensional input data, specifically including: Based on the frame stitching strategy, key frames are uniformly extracted from the original spatial map of multiple frames. Each key frame is multiplied element by element with its corresponding grayscale weight map to obtain a weighted spatial map. At the same time, the single-channel depth map is stitched together along the channel dimension to form a three-channel depth map. All weighted spatial maps and three-channel depth maps are stacked along the frame dimension to form five-dimensional input data.
5. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The improved ResNet-50 network includes an input layer, four residual layers, a global average pooling layer, and a fully connected layer; each residual layer contains multiple residual blocks.
6. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The specific dimensional reshaping of the feature map before inputting it into the SCA module is as follows: the four-dimensional feature map output by the residual layer is split into the true batch dimension using the inverse operation, and the height dimension and width dimension are multiplied together, the dimension order is adjusted and the true batch dimension and time dimension are merged to form a three-dimensional feature map that is input into the SCA module. The dimensional reshaping performed when the feature map is passed between residual layers is as follows: the three-dimensional feature map output by the SCA module is split into batch dimension and time dimension, the dimensional order is adjusted and the channel and spatial dimensions are merged to restore it as a four-dimensional feature map for input into the next residual layer.
7. The traffic risk estimation method based on frame patching and causal attention as described in claim 1, characterized in that, The process of obtaining risk classification results through a global average pooling layer and a fully connected layer specifically includes: reshaping the features output by the last SCA module into five dimensions and adjusting the order of the dimensions; using a 1×1 convolution to compress the time dimension to 1 and removing that dimension; obtaining a global feature vector through adaptive global average pooling; and finally mapping the feature vector to the risk category space through a fully connected layer to output the classification result.
8. A traffic risk estimation system based on frame interpolation and causal attention, characterized in that, include: The raw frame processing unit is configured to acquire RGB spatial video data of a traffic scene, obtain a raw spatial map through frame-level segmentation, and preprocess the raw spatial map to obtain a grayscale weight map and a single-channel depth map. The frame stitching processing unit is configured to, based on the frame stitching strategy, fuse the keyframes extracted from the original spatial map with the grayscale weight map and supplement the depth map to obtain five-dimensional input data. The risk estimation unit is configured to input five-dimensional input data into an improved ResNet-50 network for risk estimation. The improved ResNet-50 adds a CBAM module to each residual block to enhance channel and spatial attention; an SCA module is added after each residual layer to simultaneously query information from historical and current frames through sparse causal attention, enhancing spatiotemporal correlation; dimensionality reshaping is performed before the feature map is input to the SCA module and during transmission between residual layers to ensure correct processing of frame sequence information; finally, the risk classification result is obtained through a global average pooling layer and a fully connected layer.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the traffic risk estimation method based on frame patching and causal attention as described in any one of claims 1-7.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the traffic risk estimation method based on frame patching and causal attention as described in any one of claims 1-7.