Traffic risk prediction method based on traffic object space-time distribution diagram

Abstract

The invention relates to a driving risk prediction method based on a traffic object space-time distribution diagram, and the method comprises the following steps: obtaining a vehicle-mounted forward video frame, and determining a region of interest; based on the region of interest, target detection is carried out, and a traffic object set is identified and screened out; on the basis of the region of interest, carrying out space-time information labeling on the traffic object, and constructing a traffic object space-time distribution diagram; and after filtering preprocessing is carried out on the traffic object space-time distribution map, the traffic object space-time distribution map is input into a pre-constructed lightweight risk prediction model, and the travel risk is predicted in real time. Compared with the prior art, the method has the advantages of improving the accuracy and reliability of risk prediction and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving, and in particular to a method for predicting driving risks based on a spatiotemporal distribution map of traffic objects. Background Technology

[0002] Forward collisions are a major type of traffic accident. To reduce their occurrence, researchers are conducting studies in areas such as road improvement, traffic management, and advanced driver assistance systems. Among these efforts, many automakers are requiring vehicles to be equipped with Forward Collision Warning (FCW) systems, making FCW one of the primary solutions for collision prevention.

[0003] Early Forward Collision Warning (FCW) algorithms primarily relied on radar data. However, existing radar is expensive and has limited coverage, while vision-based Advanced Driver Assistance Systems (ADAS) have made significant progress thanks to recent advancements in computer vision and deep learning technologies. Companies like Mobileye and Tesla have achieved 82% accuracy in FCW based on video data. Furthermore, policies in countries like China requiring all commercial vehicles to be equipped with onboard dashcams have made widespread deployment of forward collision warning systems possible.

[0004] Current research largely relies on object detection algorithms to build vehicle collision risk prediction models. These methods extract bounding boxes from video data, capturing information such as relative distance and speed changes of traffic objects to calculate indicators like TTC (Time to Collision) for early warning. However, object detection algorithms often suffer from false positives and false negatives, leading to errors in TTC calculation. Furthermore, these methods only provide alerts based on immediate indicators, making it difficult to predict the occurrence of risky events. Some research has proposed building end-to-end classification models based on video to achieve forward collision warning (FCW). This approach considers identifying potential risk scenarios in advance, thus more effectively reducing vehicle collision risk. However, because end-to-end models do not filter interacting objects, they may contain a large amount of redundant information unrelated to forward collisions, such as distant buildings and roadside facilities. This can interfere with model performance and lead to decreased prediction accuracy.

[0005] To improve the performance of vehicle collision risk prediction models, it is necessary to accurately screen key interaction objects and remove interfering information during model building, enabling the model to focus on the main collision target. Given the limited accuracy of target detection, it is also necessary to minimize interference from missed and false detections and construct a more reasonable data representation method. Summary of the Invention

[0006] The purpose of this invention is to provide a method for predicting driving risks based on the spatiotemporal distribution map of traffic objects, which improves the accuracy and reliability of risk prediction.

[0007] The objective of this invention can be achieved through the following technical solutions: A method for predicting driving risks based on the spatiotemporal distribution map of traffic objects includes the following steps: Acquire forward-facing video frames from the vehicle and determine the region of interest; Based on the region of interest, target detection is performed to identify and filter out a set of traffic objects; Based on the region of interest, the traffic objects are labeled with spatiotemporal information to construct a spatiotemporal distribution map of the traffic objects; The spatiotemporal distribution map of the traffic objects is input into a pre-built lightweight risk prediction model to predict the risk of travel in real time.

[0008] Furthermore, the step of determining the region of interest includes: Based on the vehicle-mounted forward video frames Lane line detection algorithm is used to obtain the left and right lane lines. To form a set of lane lines , is represented as: , In the formula, The lane slope, The intercept; According to the left and right lane lines With vehicle-mounted forward video frames The intersection of the lower boundaries defines the region of interest, represented as: , In the formula, For region of interest, The point where the two lane lines intersect is the vanishing point. , These are the left and right lane lines and the vehicle's forward video frame, respectively. The intersection of the lower boundaries.

[0009] Furthermore, when the lane detection algorithm cannot identify lane lines, a default lane line is used to determine the region of interest as a substitute, as shown below: , in, These are the default lane lines and the vehicle's forward-facing video frames, respectively. The intersection of the lower boundaries is defined as: , In the formula, To replace the region of interest, To determine the vanishing point, take the vanishing point of the previous frame or an empirically set value. , This refers to the width and height of the forward-facing video frame in the vehicle.

[0010] Furthermore, the step of identifying and filtering the set of traffic objects includes: For each frame of the vehicle-mounted forward video frame For the corresponding region of interest, a convolutional neural network is used for feature extraction to obtain a multi-scale feature map, represented as follows: , In the formula, For the extracted multi-scale feature maps. It is a convolutional neural network; Based on multi-scale feature maps Object detection is achieved through classification prediction and bounding box regression: , In the formula, For the first The candidate boxes belong to the target category. Confidence level, Indicates the coordinates of the center point of the target box. These are the width and height of the target bounding box, respectively; Set a collection of traffic object categories Only retain the category belonging to the target detection results. The selected objects are filtered out, and the remaining objects are filtered according to the following rules: , In the formula, For traffic object detection boxes, For the first i One category; Based on the filtering rules, a set of traffic objects is obtained. , represented as: , In the formula, The number of traffic objects detected.

[0011] Furthermore, the steps for constructing the spatiotemporal distribution map of the traffic objects include: The region of interest is classified and labeled with internal and external targets, and the targets and classification results are added to the traffic object set. Based on the set of traffic objects with added targets and classification results, spatiotemporal information is labeled on the traffic objects to obtain a spatiotemporal distribution map of the traffic objects.

[0012] Furthermore, the step of obtaining the set of traffic objects including the target and classification results includes: Set up a traffic object detection frame The lower center point is the projection center of the traffic object and the road surface, represented as: , In the formula, The center position of the projection. Indicates the coordinates of the center point of the target box. The height of the target bounding box; Given the ordinate At that time, combining the equations of the left and right lane lines, we solve for the abscissas of the left and right lane lines at their respective heights, where the equations of the left and right lane lines are expressed as: , , In the formula, , These are the left and right lane lines, respectively. The horizontal and vertical coordinates of the corresponding left and right lane lines are: , These are the intercepts of the left and right lane lines, respectively. The horizontal coordinates of the left and right lane lines are represented as follows: , In the formula, , Let x be the x-coordinate of the left and right lane lines. , The slope of the left and right lane lines; Based on the x-coordinates of the left and right lane lines , By comparing the results, we define the rules for determining the spatial category of the region of interest, and obtain the spatial category of the region of interest. The rules for determining the spatial category are expressed as follows: , In the formula, This represents the spatial category of the region of interest. A value of 0 indicates that the interactive object is within the region of interest, a value of -1 indicates that the interactive object is to the left of the region of interest, and a value of... This indicates that the interactive object is located to the right of the region of interest; Adding the spatial category of the region of interest to the traffic object set yields: , In the formula, To add the target and classification results to the traffic object set, For traffic object detection boxes, Indicates the coordinates of the center point of the target box. These are the width and height of the target bounding box, respectively. For the first The confidence level of each candidate box belonging to the target category. The number of traffic objects detected.

[0013] Furthermore, the step of obtaining the spatiotemporal distribution map of traffic objects includes: Based on the set of traffic objects with added targets and classification results For each traffic object detection box On the baseline The corresponding lateral coverage area is determined above, and is used as the width of the traffic object, represented as follows: , In the formula, The x-coordinate of the center point of the target box. The width of the target bounding box; According to the spatial category of the region of interest The value of is used to map the horizontal coverage area to the corresponding RGB layer: , In the formula, For mapping layers, , , These are the three corresponding layers for RGB; Within the horizontal range of the corresponding RGB layer, the pixel value is set to the corresponding identifier value p: , In the formula, This is the video frame index, corresponding to the vertical position in the spatiotemporal distribution map of traffic objects; After performing the above processing on each vehicle-mounted forward video frame, the lines of all vehicle-mounted forward video frames are arranged sequentially along the vertical axis, and the R, G, and B layers are overlaid to obtain the final spatiotemporal distribution map of traffic objects, represented as follows: , In the formula, This is a spatiotemporal distribution map of traffic objects. This is a pixel-by-pixel overlay operation.

[0014] Furthermore, the lightweight risk prediction model is constructed using a lightweight convolutional neural network, including convolutional layers, pooling layers, and fully connected layers. An attention mechanism is introduced after each convolutional layer for feature enhancement. The execution process of the lightweight risk prediction model includes: Convolutional layers are used to extract the spatial and temporal trajectory features of the spatiotemporal distribution map of the traffic objects: , In the formula, For the first l The spatial and temporal trajectory feature maps output by the layer, This is for convolutional layer processing operations. , For the first Layer convolution kernel weights and biases, A spatiotemporal distribution map of traffic objects; After each convolutional layer, an attention mechanism is used for feature enhancement: , In the formula, For the first l Enhanced spatial and temporal trajectory feature maps of the layers. For attention mechanisms; Enhanced features after processing the last convolutional layer and the corresponding attention mechanism. Dimensionality reduction is achieved through pooling layers, and then mapped to a classification output through a fully connected layer: , In the formula, To predict probabilities, , These are the parameters for the fully connected layer. For fully connected layer operations, This is the index of the last convolutional layer.

[0015] Furthermore, the loss function used during training of the lightweight risk prediction model is: , In the formula, Cross-entropy classification loss, For the number of categories, One-hot encoding of the real label. To predict probabilities.

[0016] Furthermore, the method also includes filtering preprocessing the spatiotemporal distribution map of the traffic objects before inputting it into the lightweight risk prediction model. Specific steps include: Let the first The sequence of bottom center points of each traffic object in consecutive frames is as follows: , In the formula, For the t-th frame The lower center point of each traffic object, Let be the coordinates of the center point of the target box in frame t. Let be the height of the target bounding box in frame t; The following filtering algorithm is used to filter the lower center point sequence: , In the formula, This is the order of the lower center points after filtering. This is a filtering operation; Based on the filtering results, the spatiotemporal distribution map of the traffic objects after filtering is obtained. .

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention extracts the spatiotemporal change information of traffic objects from unstructured video data, constructs a spatiotemporal distribution map of traffic objects, realizes a reasonable representation of the conflict information of interactive objects, effectively suppresses the interference of irrelevant traffic environment information, and performs risk prediction through a lightweight risk prediction model, which can effectively improve the accuracy and reliability of risk prediction.

[0018] (2) This invention constructs a spatiotemporal distribution map of traffic objects with significant discriminative features, thus possessing strong distinguishing capabilities at the input level. Since the input features can fully express the spatiotemporal relationships of various objects in the traffic scene, high-precision risk identification can be achieved without using a complex deep neural network structure. This design results in a smaller overall computational load and faster inference speed, meeting the requirements of vehicle-mounted systems for real-time performance and computing resources.

[0019] (3) The lightweight risk prediction model in this invention significantly reduces the dependence on the amount of training data, and can still achieve stable performance even with limited training samples. In real driving environments, high-risk events are often scarce and random, and collecting and labeling a large number of such samples is not only difficult but also costly. Experimental results show that even with a limited number of samples, the method of this invention still maintains high prediction accuracy and stability, demonstrating good small-sample adaptability and generalization performance. Therefore, this invention has important engineering application value in real-world scenarios where high-risk events are scarce, and can provide an efficient and reliable risk prediction solution for vehicle active safety systems.

[0020] (4) The present invention introduces filtering preprocessing operation on the spatiotemporal distribution map of traffic objects, which can effectively alleviate the problems of missed detection and false detection in the target detection process, thereby improving the accuracy and reliability of risk prediction.

[0021] (5) The verification results based on natural driving data of commercial vehicles show that the method of this invention has high accuracy and stability in driving risk prediction. Even with a limited number of samples, it still maintains excellent prediction performance, demonstrating good generalization ability and engineering applicability. Compared with existing end-to-end methods, this invention shows advancement and superiority in feature representation, risk prediction accuracy, and reliability. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is the ROI obtained by lane line detection according to the present invention; Figure 3 This is a schematic diagram illustrating the process of generating the spatiotemporal distribution map of traffic objects according to the present invention; Figure 4 The filtering process of the spatiotemporal distribution map of traffic objects in this invention is shown in Figure (a), which is an example of missed detection processing and Figure (b) is an example of false detection processing. Figure 5 This is the T-SNE visualization result of the spatiotemporal distribution map of traffic objects in this invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0024] This embodiment provides a method for predicting driving risks based on the spatiotemporal distribution map of traffic objects, such as... Figure 1 As shown, the method includes the following steps: Step 1: Traffic Object Identification and Screening This invention uses natural driving video data from a company's commercial vehicles for empirical analysis. The video resolution is 760×368 pixels, and the frame rate is 4 FPS (Frames Per Second). Due to the limited number of accidents during the data collection period, high-risk events were used as surrogate events. High-risk events are those in which the vehicle brakes and decelerates at a rate greater than 0.4g. Traffic safety analysis experts categorized the video data into high-risk scenarios (traffic scenarios where at least one party must take action to avoid a collision) and non-conflict scenarios (traffic scenarios requiring driver reaction but without conflicting parties, thus avoiding collisions). Two experts jointly determined the scenario to which the video belonged; if opinions differed, a third expert made the judgment. To maintain sample balance, this study collected a total of 1397 scenarios, including 596 high-risk conflict scenarios and 801 randomly selected non-conflict scenarios.

[0025] Typically, 2 seconds is an acceptable reaction time for drivers. In order to successfully predict driving risks 2 seconds in advance, a 6-second continuous video of the vehicle's forward-facing camera is taken from 8 seconds to 2 seconds before the driver's emergency braking.

[0026] Step 1.1 Lane line detection and ROI region extraction For each frame of the vehicle's forward-facing video Apply lane line detection algorithms to obtain left and right lane lines. gather: , in, The lane slope, It is the intercept. It is determined by the left and right lane lines. The region of interest (ROI) is determined by the intersection of the ROI with the lower boundary of the image. , in: The point where the two lane lines intersect is the vanishing point. , These are the intersection points of the left and right lane lines and the lower boundary of the image, respectively.

[0027] like Figure 2 As shown in the figure, the triangular area is the ROI obtained by detecting lane lines.

[0028] Step 1.2 ROI Alternative Construction When Lane Line Detection Fails When the detection algorithm cannot reliably identify lane lines, such as on rural roads, at intersections, or in poorly lit nighttime scenes, a default lane line is used as a substitute. Assuming the image width is... The height is The default lower boundary point of the lane line is defined as: , Vanishing point If left unchanged, it can be taken as the vanishing point of the previous frame or an empirically set value. Then the alternative ROI is: , Step 1.3 Target Detection and Traffic Object Screening An advanced object detection algorithm, such as the YOLO series, is selected as the vehicle interaction object detection algorithm. This model is based on a convolutional neural network and processes the input video frames... Perform feature extraction: , in, This represents the multi-scale feature map extracted by the convolutional network. Taking the YOLO model as an example, the model achieves object detection through classification prediction and bounding box regression: , in, For the first The candidate boxes belong to the target category. (e.g., confidence level of "vehicles"); Indicates the coordinates of the center point of the target box; These are the width and height of the target bounding box, respectively.

[0029] To achieve specific identification of traffic participants, a set of traffic participant categories is set up. Retain the category belonging to the target detection results The selected object is filtered to remove other irrelevant objects, and the filtering rules are as follows: , After this filtering operation, the target set belonging to the traffic object category is obtained: , in, The number of traffic objects detected.

[0030] In this embodiment , Figure 2 This document presents a sample example of the traffic object bounding box information extracted through the above operations.

[0031] Step 2: Constructing the Spatiotemporal Distribution Map of Traffic Objects Step 2.1 Classification and Labeling of ROI Targets Inside and Outside Set up a traffic object detection frame The lower center point is shown below, which represents the center position of the projection of the target onto the road surface: , Based on the aforementioned lane line equation: , , Given the ordinate When the left and right lane lines are at that height, find the x-coordinates of the corresponding lane lines: , Based on the above and By comparing the ROIs, we define the rules for determining the ROI spatial category. Indicates the ROI spatial category, with the following possible values: , Where 0 indicates within the ROI (main interaction object), -1 indicates to the left of the ROI (left-side interaction object), and +1 indicates to the right of the ROI (right-side interaction object). Processing each traffic object as described above yields: , Step 2.2 Spatiotemporal information annotation of traffic objects For each video frame The set of traffic objects identified in Perform the following processing.

[0032] , (1) For each target On the baseline The upper part determines the corresponding horizontal coverage area, representing the width of the traffic object: , (2) Based on ROI spatial category Take the value and map the range to the corresponding RGB layer: , (3) Within the horizontal range of the corresponding layer, set the pixel value to the corresponding identifier value p (which can be 1 or a certain pixel value): , in, For video frame indexes, corresponding to the vertical position of the traffic object in the spatiotemporal distribution map (time axis from top to bottom).

[0033] (4) After processing each frame of the video according to the above rules, arrange the lines of all frames sequentially along the vertical axis, and overlay the R, G, and B layers to obtain the final Traffic Object Spatio-Temporal Distribution Map (TSDMap): , in This indicates a pixel-by-pixel overlay operation.

[0034] In this embodiment, p=255, and the process of generating the spatiotemporal distribution map of traffic objects is as follows: Figure 3 As shown.

[0035] Step 3: Building a Lightweight Risk Prediction Model Step 3.1 Model Structure and Loss Function Design (1) Model input: The spatiotemporal distribution map (TSDMap) of traffic objects generated from each vehicle-mounted forward video is used as the model input: , in These represent the height and width of the spatiotemporal distribution map of traffic objects, respectively, with 3 indicating the RGB three-channel configuration.

[0036] (2) Lightweight Convolutional Neural Network Structure: To avoid computational inefficiency and overfitting, a lightweight convolutional neural network (CNN) structure is adopted, including convolutional layers, pooling layers, and fully connected layers. The convolutional layers are used to extract spatial and temporal trajectory features from the spatiotemporal distribution map of traffic objects. , in: This is the output feature map of the previous layer. The first Layer convolution kernel weights and biases, , which is the spatiotemporal distribution map of the input traffic objects.

[0037] (3) Enhanced attention mechanism: An attention mechanism is introduced after each convolutional layer: , This aims to enhance the model's responsiveness to major conflict traffic objects and improve the targeting and robustness of feature extraction.

[0038] (4) Classification output and loss function: The convolutional features are mapped to classification output through a fully connected layer: , in: This is the index of the last convolutional layer. These are the parameters of the fully connected layer; the loss function uses cross-entropy classification loss: , in For the number of categories, One-hot encoding of the real label. To predict probabilities.

[0039] In this case, category C=2, and the spatiotemporal distribution map of traffic objects is uniformly scaled to 64*64 pixels. The structural design of the model is shown in Table 1.

[0040] Table 1. Model Structure Design Step 3.2 Sample Filtering and Quality Improvement The quality of the spatiotemporal distribution map of traffic objects is highly dependent on the accuracy of target detection. Since traffic objects exhibit continuous motion characteristics in the spatiotemporal dimension, ideally, when detected correctly, the spatiotemporal distribution map of traffic objects, composed of their trajectories, should appear as a smooth, continuous curve. However, in actual target detection, the following problems often occur: (1) Missed detection: A target is not detected in a certain frame, causing the originally continuous trajectory to be interrupted; (2) False detection: Non-target areas are incorrectly identified as targets, causing abnormal changes in the trajectory, such as a sudden increase or decrease in the target width. These anomalies can disrupt the continuity and smoothness of the trajectory, potentially leading to misjudgments in conflict perception and risk prediction. Therefore, filtering preprocessing is introduced to smooth and correct the trajectory. Let the first... The sequence of bottom center points of each traffic object in consecutive frames is as follows: , The trajectory sequence is processed using filtering algorithms (such as median filtering, moving average filtering, Kalman filtering, etc.). , in The filtered trajectory points ensure a smooth and continuous trajectory. The filtered spatiotemporal distribution map of traffic objects is used as input to a lightweight convolutional neural network for training. , This ensures that the model can learn the spatiotemporal trajectory patterns of real traffic objects, reduces noise interference caused by missed detections and false detections, and improves the accuracy of conflict feature extraction.

[0041] In this embodiment, median filtering was selected, such as Figure 4 On the left side of Figure (a), due to missing YOLO detection results, the semantic contour map shows discontinuities in the red trajectories; similarly, in Figure 4 In Figure (b), the green area on the left side shows obvious false positives, and there are also discrete line segments on both sides due to false positives. This information can easily interfere with the model's judgment. Median filtering can be used to address this. Figure 4 The missing trajectory in graph (a) can be completed, and Figure 4 The obvious false positive detection boxes in Figure (b) were also screened out.

[0042] Step 4: Methodological Advancement Analysis Step 4.1 Baseline Model Selection The method of this invention belongs to the end-to-end driving risk prediction method. To verify the effectiveness and superiority of the proposed method, two typical end-to-end models are selected as baseline comparisons.

[0043] (1) Baseline Model 1: CNN-LSTM Model Existing research on risk prediction based on in-vehicle forward-facing video typically employs a two-stage framework of "spatial feature extraction + temporal modeling." Specifically, firstly, a convolutional neural network (CNN) is used to extract features from each video frame to obtain the spatial attributes of traffic objects (such as location and distance). Then, the feature sequences of consecutive frames are input into a temporal model (such as LSTM or Transformer) to capture the temporal evolution of the traffic scene, thereby achieving driving risk prediction. This type of model is collectively referred to as the CNN-LSTM model.

[0044] (2) Baseline Model 2: YOLO-LSTM Model To further improve the detection accuracy of the baseline model, the CNN module of the CNN-LSTM model was replaced with a YOLO object detection network. The YOLO model can more accurately identify and locate traffic objects in video frames within an end-to-end framework. Its detection results (including bounding box coordinates and class confidence scores) are input into the LSTM model to extract temporal features and perform risk assessment. This improved model is called the YOLO-LSTM model.

[0045] Step 4.2 Performance Indicator Selection For binary classification problems that distinguish between risk and non-risk, AUC is used as a performance metric to assess the risk of vehicle collisions. AUC is the area under the Receiver Operating Characteristic (ROC) curve. The horizontal and vertical axes of the ROC curve are the false positive rate (FPR) and the true positive rate (TPR), respectively. The ROC curve is composed of (FPR, TPR) coordinate points at different classification thresholds. The maximum value of AUC is 1. The larger the AUC, the better the model performance.

[0046] Step 4.3 Performance Comparison with Baseline Methods In the model performance evaluation, the driving risk prediction model based on the spatiotemporal distribution map of traffic objects proposed in this invention achieved an AUC of 0.91 when predicting risks 2 seconds in advance, demonstrating excellent risk discrimination ability. In comparison, the AUC value of baseline model 1 (CNN-LSTM) is 0.83, and the AUC value of baseline model 2 (YOLO-LSTM) is 0.86, both significantly lower than the performance level of the method proposed in this invention.

[0047] This demonstrates that the method of the present invention can more effectively capture the spatiotemporal distribution characteristics of traffic objects in traffic risk prediction tasks, exhibiting stronger feature representation capabilities and risk identification accuracy. Compared with commonly used end-to-end methods, the method of the present invention demonstrates significant advancements and superiority in both model performance and application reliability.

[0048] Step 4.4 Performance Indicators under Different Forecast Lead Times To further verify the performance stability of the method of this invention under different lead times for traffic risk prediction, the AUC index of the model was experimentally tested in prediction scenarios with lead times of 1 s, 2 s, 3 s, and 5 s. Experimental results show that: When the forecast lead time is 1 s, the model's AUC is 0.916. When the forecast lead time is 2 seconds, the AUC is 0.913. When the forecast lead time is 3 seconds, the AUC is 0.910. When the forecast lead time is 5 seconds, the AUC is 0.898.

[0049] As the lead time for prediction increases, the model performance shows a slight downward trend. However, under the prediction conditions of 1 s, 2 s, and 3 s in advance, the model maintains an excellent performance with AUC > 0.91, indicating that the method of this invention still has high accuracy and stability in short-term risk prediction tasks.

[0050] Step 4.5 Impact of Training Sample Size on Model Performance To further analyze the impact of the number of training samples on model performance, comparative experiments were conducted on the method of this invention under different training sample sizes, with a prediction lead of 2 seconds. Specifically, the number of training samples was gradually reduced from 1000 to 500, 300, 100, and 50, with corresponding AUC values ​​of 0.913, 0.912, 0.910, 0.895, and 0.887, respectively.

[0051] Experimental results show that as the number of training samples decreases, the model performance exhibits a certain degree of decline, but remains at a high level overall. Even with an extremely small sample size (only 50 samples), the method of this invention still achieves an excellent result of AUC=0.887, demonstrating good adaptability and robustness to small samples.

[0052] To explore why the algorithm is suitable for small sample modeling, the T-SNE algorithm was used to visualize the spatiotemporal distribution map of the generated traffic objects. Figure 5 As can be seen, the samples of different categories have good distinguishability, so a relatively accurate model can be completed without a complex model structure, and it can also perform well even with a small amount of data.

[0053] This characteristic has significant practical implications: in real-world driving scenarios, high-risk events are characterized by scarcity and randomness, making it difficult and costly to obtain a large number of labeled samples. However, the method of this invention maintains high predictive performance even with a limited sample size, demonstrating its strong generalization ability and engineering application value.

[0054] The method in this embodiment has the following advantages: 1) This method achieves efficient compression of in-vehicle forward-facing video. For a segment containing... N A video of frames, assuming each frame has a size of 1. W × H Then the video can be compressed into a single image of size [size missing]. W × T The image is compressed. To improve compression efficiency, only lateral coordinate information reflecting the relative lateral relationships of traffic participants (i.e., whether the target is in front of, to the left of, or to the right of the vehicle) is retained. While the longitudinal coordinate (reflecting distance) is not explicitly retained, it is implicitly expressed through the width of the traffic object's trajectory in the spatiotemporal distribution map: the closer the distance, the wider the trajectory; the farther the distance, the narrower the trajectory. The compressed image retains only target location and distance information closely related to the vehicle's interaction, filtering out irrelevant background features such as sky, buildings, trees, and road surface color, providing an efficient and focused feature representation foundation for subsequent traffic risk prediction.

[0055] 2) The method of this invention constructs a spatiotemporal distribution map of traffic objects with significant discriminative features, thus possessing strong distinguishing capabilities at the input level. Since the input features can fully express the spatiotemporal relationships of various objects in the traffic scene, high-precision risk identification can be achieved without employing complex deep neural network structures. This design results in a smaller overall computational load and faster inference speed, meeting the real-time and computational resource requirements of in-vehicle systems. Simultaneously, the lightweight network structure significantly reduces the dependence on the amount of training data, achieving stable performance even with limited training samples. In real-world driving environments, high-risk events are often scarce and random; collecting and labeling large quantities of such samples is not only difficult but also costly. Experimental results show that even with limited sample sizes, the method of this invention maintains high prediction accuracy and stability, demonstrating good small-sample adaptability and generalization performance. Therefore, the method of this invention has significant engineering application value in real-world scenarios where high-risk events are scarce, providing an efficient and reliable risk prediction solution for in-vehicle active safety systems.

[0056] In summary, this method extracts spatiotemporal variation information of traffic objects from unstructured video data to construct a spatiotemporal distribution map of traffic objects, thereby achieving a reasonable representation of conflict information of interactive objects and effectively suppressing interference from irrelevant traffic environment information. Furthermore, the method introduces median filtering on the spatiotemporal distribution map of traffic objects, which can effectively alleviate the problems of missed detections and false detections in the target detection process, thus improving the accuracy and reliability of risk prediction.

[0057] Validation results based on natural driving data from commercial vehicles demonstrate that the method of this invention exhibits high accuracy and stability in driving risk prediction. Even with a limited sample size, it maintains excellent predictive performance, demonstrating good generalization ability and engineering applicability. Compared with existing end-to-end methods, this invention demonstrates advancements and superiority in feature representation, risk prediction accuracy, and reliability.

[0058] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0060] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0064] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for predicting driving risks based on the spatiotemporal distribution map of traffic objects, characterized in that, Includes the following steps: Acquire forward-facing video frames from the vehicle and determine the region of interest; Based on the region of interest, target detection is performed to identify and filter out a set of traffic objects; Based on the region of interest, the traffic objects are labeled with spatiotemporal information to construct a spatiotemporal distribution map of the traffic objects; The spatiotemporal distribution map of the traffic objects is input into a pre-built lightweight risk prediction model to predict the risk of travel in real time.

2. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, The step of determining the region of interest includes: Based on the vehicle-mounted forward video frames Lane line detection algorithm is used to obtain the left and right lane lines. To form a set of lane lines , is represented as: ， In the formula, The lane slope, The intercept; According to the left and right lane lines With vehicle-mounted forward video frames The intersection of the lower boundaries defines the region of interest, represented as: ， In the formula, For region of interest, The point where the two lane lines intersect is the vanishing point. , These are the left and right lane lines and the vehicle's forward video frame, respectively. The intersection of the lower boundaries.

3. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 2, characterized in that, It also includes using a default lane line to determine the region of interest when the lane line detection algorithm cannot identify the lane line, as shown below: ， in, These are the default lane lines and the vehicle's forward-facing video frames, respectively. The intersection of the lower boundaries is defined as: ， In the formula, To replace the region of interest, To determine the vanishing point, take the vanishing point of the previous frame or an empirically set value. , This refers to the width and height of the forward-facing video frame in the vehicle.

4. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, The steps for identifying and filtering the set of traffic objects include: For each frame of the vehicle-mounted forward video frame For the corresponding region of interest, a convolutional neural network is used for feature extraction to obtain a multi-scale feature map, represented as follows: ， In the formula, This is for the extracted multi-scale feature map. It is a convolutional neural network; Based on multi-scale feature maps Object detection is achieved through classification prediction and bounding box regression: ， In the formula, For the first The candidate boxes belong to the target category. Confidence level, Indicates the coordinates of the center point of the target box. These are the width and height of the target bounding box, respectively; Set a collection of traffic object categories Only retain the category belonging to the target detection results. The selected objects are filtered out, and the remaining objects are filtered according to the following rules: ， In the formula, For traffic object detection boxes, For the first i One category; Based on the filtering rules, a set of traffic objects is obtained. , is represented as: ， In the formula, The number of traffic objects detected.

5. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, The steps for constructing the spatiotemporal distribution map of the traffic objects include: The region of interest is classified and labeled with internal and external targets, and the targets and classification results are added to the traffic object set. Based on the set of traffic objects with added targets and classification results, spatiotemporal information is labeled on the traffic objects to obtain a spatiotemporal distribution map of the traffic objects.

6. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 5, characterized in that, The steps for obtaining the set of traffic objects that include the target and classification results are as follows: Set up a traffic object detection frame The lower center point is the projection center of the traffic object and the road surface, represented as: ， In the formula, The center position of the projection. Indicates the coordinates of the center point of the target box. The height of the target bounding box; Given the ordinate At that time, combining the equations of the left and right lane lines, we solve for the abscissas of the left and right lane lines at their respective heights, where the equations of the left and right lane lines are expressed as: ， ， In the formula, , These are the left and right lane lines, respectively. The horizontal and vertical coordinates of the corresponding left and right lane lines are: , These are the intercepts of the left and right lane lines, respectively. The horizontal coordinates of the left and right lane lines are represented as follows: ， In the formula, , Let x be the x-coordinate of the left and right lane lines. , The slope of the left and right lane lines; Based on the x-coordinates of the left and right lane lines , By comparing the results, we define the rules for determining the spatial category of the region of interest, and obtain the spatial category of the region of interest. The rules for determining the spatial category are expressed as follows: ， In the formula, This represents the spatial category of the region of interest. A value of 0 indicates that the interactive object is within the region of interest, a value of -1 indicates that the interactive object is to the left of the region of interest, and a value of... This indicates that the interactive object is to the right of the region of interest; Adding the spatial category of the region of interest to the traffic object set yields: ， In the formula, To add the target and classification results to the traffic object set, For traffic object detection boxes, Indicates the coordinates of the center point of the target box. These are the width and height of the target bounding box, respectively. For the first The confidence level of each candidate box belonging to the target category. The number of traffic objects detected.

7. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 6, characterized in that, The steps for obtaining the spatiotemporal distribution map of traffic objects include: Based on the set of traffic objects with added targets and classification results For each traffic object detection box On the baseline The corresponding lateral coverage area is determined above, and is used as the width of the traffic object, represented as follows: ， In the formula, The x-coordinate of the center point of the target box. The width of the target bounding box; According to the spatial category of the region of interest The value of is used to map the horizontal coverage area to the corresponding RGB layer: ， In the formula, For mapping layers, , , These are the three corresponding layers for RGB; Within the horizontal range of the corresponding RGB layer, the pixel value is set to the corresponding identifier value p: ， In the formula, This is the video frame index, corresponding to the vertical position in the spatiotemporal distribution map of traffic objects; After performing the above processing on each vehicle-mounted forward video frame, the lines of all vehicle-mounted forward video frames are arranged sequentially along the vertical axis, and the R, G, and B layers are overlaid to obtain the final spatiotemporal distribution map of traffic objects, represented as follows: ， In the formula, This is a spatiotemporal distribution map of traffic objects. This is a pixel-by-pixel overlay operation.

8. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, The lightweight risk prediction model is constructed using a lightweight convolutional neural network, including convolutional layers, pooling layers, and fully connected layers. An attention mechanism is introduced after each convolutional layer for feature enhancement. The execution process of the lightweight risk prediction model includes: Convolutional layers are used to extract the spatial and temporal trajectory features of the spatiotemporal distribution map of the traffic objects: ， In the formula, For the first l The spatial and temporal trajectory feature maps output by the layer, This is for convolutional layer processing operations. , For the first Layer convolution kernel weights and biases, A spatiotemporal distribution map of traffic objects; After each convolutional layer, an attention mechanism is used for feature enhancement: ， In the formula, For the first l Enhanced spatial and temporal trajectory feature maps after layering. For attention mechanisms; Enhanced features after processing the last convolutional layer and the corresponding attention mechanism. Dimensionality reduction is achieved through pooling layers, and then mapped to a classification output through a fully connected layer: ， In the formula, To predict probabilities, , These are the parameters for the fully connected layer. For fully connected layer operations, This is the index of the last convolutional layer.

9. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, The loss function used during training of the lightweight risk prediction model is: ， In the formula, Cross-entropy classification loss, For the number of categories, One-hot encoding of the real label. To predict probabilities.

10. The method for predicting driving risks based on the spatiotemporal distribution map of traffic objects according to claim 1, characterized in that, It also includes filtering preprocessing before inputting the spatiotemporal distribution map of the traffic objects into the lightweight risk prediction model. Specific steps include: Let the first The sequence of bottom center points of each traffic object in consecutive frames is as follows: ， In the formula, For the t-th frame The lower center point of each traffic object, Let be the coordinates of the center point of the target box in frame t. Let be the height of the target bounding box in frame t; The following filtering algorithm is used to filter the lower center point sequence: ， In the formula, This is the filtered order of the lower center points. This is a filtering operation; Based on the filtering results, the spatiotemporal distribution map of the traffic objects after filtering is obtained. .

Images