Road traffic accident reconstruction system and method applicable to intelligent connected vehicles

Abstract

This invention discloses a road traffic accident reconstruction system and method applicable to intelligent connected vehicles, relating to the field of data processing technology. The system includes: a positioning module for acquiring the current accident status of a road traffic accident and locating the associated device set; a correction module for performing data collection of associated vehicles and roadside equipment to establish the original sensor data stream; a segmentation module for performing accident semantic node slicing using semantic time slice channels; a causal analysis module for outputting a time-resolved causal attribution score; and an authentication module for performing accident scene reconstruction under a generator-discriminator adversarial approach and performing confidence authentication of the candidate reconstruction set. It solves the technical problems of existing technologies, such as the difficulty in synchronously fusing multi-source heterogeneous data, the difficulty in uncovering implicit causal relationships in accidents, and the low credibility of accident scene reconstruction, achieving the technical effect of improving the temporal accuracy, causal interpretability, and result credibility of road traffic accident reconstruction.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a road traffic accident reconstruction system and method applicable to intelligent connected vehicles. Background Technology

[0002] With the rapid development of intelligent connected vehicles, information interaction between vehicles, road infrastructure, other vehicles, and cloud systems is becoming increasingly frequent, forming a complex vehicle-road cooperative environment that poses challenges to the analysis and reconstruction of road traffic accidents. Traditional road traffic accident investigations mainly rely on physical traces, witness testimonies, and limited data from onboard recording devices, often suffering from incomplete information, time asynchrony, and weak causal relationships, making it difficult to cope with intelligent connected scenarios involving multiple devices, high dynamics, and strong interactions. Accidents in intelligent connected environments involve multiple participants, including vehicles, roadside units (RSUs), cameras, and radar. The data is characterized by multiple sources, asynchronousness, and heterogeneity. For example, data collected by different devices may have time deviations, inconsistent coordinates, and semantic inconsistencies, directly affecting the accuracy and reliability of accident reconstruction. Furthermore, the causal relationships in the accident process are often implicit in the temporal dynamics and interactive events, which existing methods struggle to extract effectively, resulting in insufficient interpretability and credibility of the reconstruction results.

[0003] Therefore, current technologies face technical challenges such as difficulty in synchronously integrating multi-source heterogeneous data, difficulty in uncovering hidden causal relationships in accidents, and low reliability in accident scene reconstruction. Summary of the Invention

[0004] This application provides a road traffic accident reconstruction system and method applicable to intelligent connected vehicles, which solves the technical problems existing in the prior art, such as the difficulty in synchronously integrating multi-source heterogeneous data, the difficulty in mining the implicit causal relationship of accidents, and the low credibility of accident scene reconstruction. It achieves the technical effect of improving the temporal accuracy, causal interpretability, and result credibility of road traffic accident reconstruction.

[0005] This application provides a road traffic accident reconstruction system applicable to intelligent connected vehicles. The system includes: a positioning module for acquiring the current accident status of a road traffic accident and locating a set of associated devices based on the current accident status, the set of associated devices including associated vehicles and associated roadside equipment; a correction module for collecting data from associated vehicles and associated roadside equipment after setting a global time axis, establishing an original sensor data stream, the original sensor data stream undergoing time correction through the global time axis; a segmentation module for performing feature interaction analysis on the original sensor data stream, then using a semantic time slice channel to slice the accident semantic nodes of the original sensor data stream, establishing an accident semantic graph; a causal analysis module for inputting the accident semantic graph and the original sensor data stream into a window causal recognition model, outputting a time-resolved causal attribution score; and an authentication module for extracting causal conclusions based on the causal attribution score, using the causal conclusions as prior constraints, performing accident scene reconstruction under generative-discriminative adversarial methods, establishing a candidate reconstruction set, performing confidence authentication of the candidate reconstruction set under multiple optimization indicators, and establishing a reconstruction result.

[0006] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: a feature segmentation submodule, used to extract features from the original sensor data stream and establish video frame features, point cloud features, IMU features, and CAN bus features. The video frame features include target detection and behavior recognition; the point cloud features include obstacle distance and relative speed; the IMU features include yaw rate and longitudinal acceleration; and the CAN bus features include braking signals, steering signals, and throttle signals. A feature interaction analysis submodule is used to perform feature interaction analysis within the video frame features, point cloud features, IMU features, and CAN bus features using bidirectional attention, establish feature interaction analysis results, and use the feature interaction analysis results to complete the construction of an accident semantic graph.

[0007] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: a node segmentation submodule, used to establish a first segmentation constraint after reading the feature interaction analysis results in the semantic time slice channel, perform semantic event perception on the original sensor data stream, establish a second segmentation constraint, use the first segmentation constraint and the second segmentation constraint to perform accident semantic node identification under interactive authentication, perform accident semantic node slicing, and establish a semantic time slice; a graph construction submodule, configured with semantic label encoding for the semantic time slice, performs interactive feature node aggregation in each semantic time slice, wherein each node contains structured variables and unstructured summaries, connects adjacent time slice nodes with directed edges, configures time delay labels, and establishes an accident semantic graph using semantic label encoding, nodes, and directed edges.

[0008] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: a parsing submodule, used to parse the original sensor data stream, obtain multi-source observation streams and a set of structured variables, vectorize the accident semantic graph, establish a semantic embedding matrix, encode the set of structured variables into a hybrid feature vector, and perform time-series synchronization of the multi-source observation streams, semantic embedding matrix, and hybrid feature vectors within a unified sliding time window; a window causal identification submodule, used to call the window causal identification model to perform causal chain analysis within each sliding window, wherein the causal chain analysis includes correlation causal inference based on key accident nodes in the accident semantic graph; and an output submodule, used to establish and output a causal attribution score based on the causal chain analysis.

[0009] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: a discriminator construction submodule, used to construct a residual distribution authentication discriminator based on the prior constraints and the original sensor data stream; and an accident scene reconstruction submodule, used to obtain vehicle models of associated vehicles, take the current accident state as the final fitting result, reconstruct the accident scene based on the generator, perform reconstruction authentication through the discriminator, perform multiple rounds of adversarial iteration, and establish a candidate reconstruction set.

[0010] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: a multi-index construction submodule, used to construct a multi-index evaluation function, wherein the evaluation indicators of the multi-index evaluation function include physical consistency index, data matching degree index, and causal rationality index; and a confidence authentication submodule, used to perform confidence authentication of the candidate reconstruction set according to the multi-index evaluation function, filter the candidate reconstruction set according to the confidence authentication trust level, and obtain the reconstruction result.

[0011] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles also performs the following processing: a latent factor extraction submodule, which, after acquiring weather and environmental data, extracts latent factors based on the weather and environmental data and the original sensor data stream to establish latent influence features; and an enhanced identification submodule, which, after enhancing the attention of the window causal identification model based on the latent influence features, performs causal analysis.

[0012] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles also performs the following processing: a visualization display module, used to output the reconstruction result as a three-dimensional visualization reconstruction scene, and to perform scene data identification of the three-dimensional visualization reconstruction scene at key nodes, and to perform visualization display based on the three-dimensional visualization reconstruction scene with scene data identification.

[0013] In a possible implementation, the road traffic accident reconstruction system for intelligent connected vehicles further performs the following processing: performing data criticality authentication on the data collection results of associated vehicles and associated roadside equipment, and generating data criticality identifiers; performing source credibility authentication on the data collection results, and establishing data credibility identifiers; using the criticality identifiers and the credibility identifiers to perform data interaction authentication on the data collection results, and using the data interaction authentication results to establish the original sensor data stream.

[0014] This application also provides a method for reconstructing road traffic accidents applicable to intelligent connected vehicles. The method includes: acquiring the current accident state of a road traffic accident; locating a set of associated devices based on the current accident state, the set of associated devices including associated vehicles and associated roadside equipment; after setting a global time axis, performing data acquisition on associated vehicles and associated roadside equipment to establish an original sensor data stream, the original sensor data stream undergoing time correction through the global time axis; performing feature interaction analysis on the original sensor data stream, and then using a semantic time slice channel to slice accident semantic nodes of the original sensor data stream to establish an accident semantic graph; inputting the accident semantic graph and the original sensor data stream into a window causal recognition model, and outputting a causal attribution score with time resolution; extracting causal conclusions based on the causal attribution score, using the causal conclusions as prior constraints, performing accident scene reconstruction under generative-discriminative adversarial methods to establish a candidate reconstruction set, performing confidence authentication of the candidate reconstruction set under multiple optimization indicators, and establishing a reconstruction result.

[0015] This application proposes a road traffic accident reconstruction system and method for intelligent connected vehicles, comprising: a positioning module for acquiring the current accident status and locating associated device sets; a correction module for collecting data from associated vehicles and roadside equipment to establish an original sensor data stream; a segmentation module for slicing accident semantic nodes from the original sensor data stream using semantic time-slice channels; a causal analysis module for outputting time-resolved causal attribution scores; and an authentication module for reconstructing the accident scene under a generative-discriminative adversarial approach and performing confidence authentication on the candidate reconstruction set to establish the reconstruction result. This system addresses the technical problems of existing technologies, such as the difficulty in synchronously fusing multi-source heterogeneous data, the difficulty in uncovering implicit causal relationships in accidents, and the low credibility of accident scene reconstruction. It achieves the technical effect of improving the temporal accuracy, causal interpretability, and result credibility of road traffic accident reconstruction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 A schematic diagram of the road traffic accident reconstruction system for intelligent connected vehicles provided in this application embodiment.

[0018] Figure 2 A schematic flowchart of a road traffic accident reconstruction method for intelligent connected vehicles provided in this application embodiment.

[0019] Explanation of reference numerals in the attached diagram: Positioning module 10, Correction module 20, Segmentation module 30, Causal analysis module 40, Authentication module 50. Detailed Implementation

[0020] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0023] This application provides a road traffic accident reconstruction system suitable for intelligent connected vehicles, such as... Figure 1 As shown, the system includes:

[0024] The positioning module 10 is used to obtain the current accident status of a road traffic accident and locate the associated device set based on the current accident status. The associated device set includes associated vehicles and associated roadside equipment.

[0025] Preferably, the current accident status of a road traffic accident is obtained based on vehicle-triggered, roadside-triggered, or manually triggered by the traffic management platform. This includes intelligent connected vehicle driving information and driver operation information. The intelligent connected vehicle driving information refers to the external environment status and its own motion status detected and perceived by the vehicle through various sensors. This includes environmental perception information, such as images / video streams of surrounding vehicles, pedestrians, traffic signs, traffic lights, and lane lines captured by cameras, and point cloud data generated by LiDAR and millimeter-wave radar. The vehicle's own status information includes precise latitude and longitude, altitude, and heading angle located by GPS, BeiDou, etc., as well as vehicle three-axis acceleration and three-axis angular velocity provided by the IMU. Driver operation information refers to the control commands and human-machine interaction information that the driver applies to the vehicle. The control command information includes braking operation data such as brake pedal opening, brake pedal switch status, and whether ABS / ESP is triggered; driving operation data such as accelerator pedal opening and drive mode selection; and steering operation data such as steering wheel angle and steering wheel angular velocity. The human-machine interaction information includes the autonomous driving status, warning information issued by the vehicle to the driver such as collision warning and lane departure warning, as well as the driver's requests or actions to take over the vehicle.

[0026] Preferably, the vehicle detects a collision event through the airbag controller and collision sensors, and automatically sends an accident signal through the vehicle network, which usually includes the time, rough GPS location, license plate, etc.; the roadside smart camera or radar detects abnormal events such as sudden traffic flow stagnation, object splashing, abnormal parking, etc., and generates an accident alarm; after receiving the accident alarm, the traffic management center platform allows the operator to input the accident location and time. The spatial range of the accident is determined based on the current accident status. This involves defining a spatiotemporal range based on the time and approximate location of the accident, such as the area within 200 meters of the accident site and the two minutes before and after the accident. Within this spatiotemporal range, all data acquisition devices in the intelligent connected vehicle environment are selected to form a set of associated devices, including associated vehicles and associated roadside equipment. Associated vehicles include the vehicle directly involved in the accident and surrounding vehicles, including other intelligent connected vehicles that were within the spatiotemporal range before the accident, such as dashcams of following vehicles capturing the sudden lane change of the vehicle in front. Associated roadside equipment includes roadside units (RSUs) that record surrounding traffic flow data, intelligent monitoring cameras at intersections and road sections, and LiDAR / millimeter-wave radar devices for smart intersection perception, which provide high-precision point cloud data to accurately reconstruct the vehicle's trajectory and speed.

[0027] The calibration module 20 is used to perform data acquisition of associated vehicles and associated roadside equipment after setting the global time axis, and establish an original sensor data stream. The original sensor data stream is time-calibrated through the global time axis.

[0028] Preferably, setting a global timeline means synchronizing local clocks from different devices to a global reference using a high-precision time synchronization protocol, thereby achieving time synchronization between devices. Then, data acquisition commands are sent to associated vehicles and associated roadside equipment, requiring the upload of all sensor data recorded during the accident period, including data related to associated vehicles, such as camera video streams, LiDAR point cloud sequences, millimeter-wave radar data, inertial measurement unit data, and CAN bus data such as vehicle speed, steering angle, braking, and throttle status; and data related to roadside equipment, such as video from roadside cameras, point clouds from roadside LiDAR, and current status logs of traffic lights. Finally, the data related to associated vehicles and roadside equipment are combined to form the raw sensor data stream.

[0029] Preferably, the raw sensor data stream undergoes time correction via a global time axis, including absolute time alignment and interpolation resampling. Specifically, each data packet in the raw sensor data stream, such as an image frame, a point cloud frame, or a CAN signal, and its corresponding local device timestamp are acquired and converted to a unified global timestamp based on the offset from the global time axis. Different sensors have different data frequencies, such as 30Hz for cameras, 10Hz for LiDAR, and 100Hz for the CAN bus. The correction module uses the global time axis as a reference and generates a data stream on a unified time series for all data through an interpolation algorithm, thereby ensuring the accuracy of accident events in the raw sensor data stream.

[0030] Furthermore, the specific configuration of the calibration module 20 also includes: performing data criticality authentication on the data collection results of associated vehicles and associated roadside equipment, generating data criticality identifiers; performing source credibility authentication on the data collection results, establishing data credibility identifiers; using the criticality identifiers and the credibility identifiers to perform data interaction authentication on the data collection results, and using the data interaction authentication results to establish the original sensor data stream.

[0031] Preferably, the data collection results of associated vehicles and roadside equipment undergo data criticality authentication. This involves determining the usefulness of the data collection results within the current time and space range based on the current accident status data, and then generating data criticality labels. Each collected data point is labeled with a criticality level, such as high, medium, or low. For example, in a rear-end collision, the brake light video footage of the preceding vehicle and the brake signal data in its own CAN bus have extremely high criticality, while the data from a streetlight sensor 200 meters away from the accident scene has low criticality. Next, the source credibility of the data collection results is authenticated, including device identity authentication, device health status verification, and data integrity verification, to assess the data itself. The reliability and authenticity of the data are verified, such as determining whether the collected data comes from compliant equipment and checking the status of the sensor equipment during data collection, such as whether the camera lens is blocked by dirt or whether the radar is calibrated incorrectly. Then, the credibility indicator is output by combining the results of several inspections. Next, the criticality indicator and the credibility indicator are used to perform data interaction authentication of the data collection results. That is, data from different sources at the same time are compared to verify whether they can corroborate each other and whether there are any contradictions. The data interaction authentication result is output. Finally, the data interaction authentication result is used to establish the original sensor data stream, that is, a continuous data sequence that is time-aligned and includes video frames, LiDAR point clouds, vehicle signals, inertial measurement unit data, etc.

[0032] The segmentation module 30 is used to perform feature interaction analysis on the original sensor data stream, and then use the semantic time slice channel to perform accident semantic node slicing of the original sensor data stream to establish an accident semantic map.

[0033] Preferably, deep learning models such as convolutional neural networks are used to perform feature interaction analysis on the raw sensor data stream based on big data. Corresponding features are extracted from different modal data sources, such as video frame features, target features, behavioral features, obstacle features from point clouds, acceleration from IMU / CAN, and braking / accelerator status features. Then, a bidirectional attention mechanism is used to allow features from different sources to interact. When determining whether braking has begun, the focus is on the braking signal in the CAN bus and whether the brake lights are illuminated in the video, while also referencing changes in longitudinal acceleration in the IMU. Next, a user vehicle profile of the vehicle involved is obtained, which is a digital summary of its long-term driving behavior characteristics, such as the driver's habit of sudden braking, rapid acceleration, and frequent lane departures. During feature interaction analysis, this user vehicle profile is referenced. For example, for a vehicle with an aggressive driving style, moderate braking force may not be identified as an emergency braking point, while for a vehicle with conservative driving, the same braking force may be identified as a critical event. The vehicle profile provides personalized and dynamic threshold backgrounds for semantic node recognition, making the analysis results more accurate and avoiding misjudgments.

[0034] Preferably, the semantic time-slice channel is used to slice the original sensor data stream into accident semantic nodes. The semantic time-slice channel is an event detector that slides on the time axis to analyze the fused data stream after feature interaction analysis in real time. Accident semantic nodes refer to key actions or states that describe the evolution of an accident, which may include vehicle start-up, detection of an obstacle ahead, driver braking, steering to avoid an obstacle, collision, vehicle skidding, etc. Slicing the accident semantic nodes means locating the start and end times of key events through the event detector. Each segment is a semantic time-slice, which includes a complete key event. Finally, the sliced ​​semantic time-slices are organized and their temporal and logical relationships are described. That is, each semantic time-slice is used as a node, and the nodes are connected by arrows as directed edges to indicate the order of events. The time interval or logical relationship is marked on the arrow as a label, and finally, an accident semantic graph is obtained.

[0035] Furthermore, the specific configuration of the segmentation module 30 also includes a feature segmentation submodule, used to extract features from the original sensor data stream and establish video frame features, point cloud features, IMU features, and CAN bus features. The video frame features include target detection and behavior recognition, the point cloud features include obstacle distance and relative velocity, the IMU features include yaw rate and longitudinal acceleration, and the CAN bus features include braking signal, steering signal, and throttle signal. A feature interaction analysis submodule is also included, used to perform feature interaction analysis within the video frame features, point cloud features, IMU features, and CAN bus features using bidirectional attention, establish feature interaction analysis results, and use the feature interaction analysis results to complete the construction of an accident semantic graph.

[0036] Preferably, the feature segmentation submodule is used to extract features from the raw sensor data stream, including using convolutional neural networks to extract features from video frames, using 3D deep learning models such as PointNet and VoxelNet to extract features from 3D point cloud data, filtering and performing physical calculations on inertial measurement unit data for feature extraction, and extracting features from the raw data stream of the vehicle's internal network CAN bus. Specifically, video frame feature extraction includes target detection and behavior recognition. Target detection refers to identifying targets, such as vehicles, pedestrians, traffic signs, traffic lights, and their corresponding locations. Behavior recognition analyzes target actions, such as a vehicle turning, a pedestrian crossing the road, or a traffic light turning red. Point cloud feature extraction includes accurately calculating the absolute distance and relative speed between the vehicle and surrounding obstacles. IMU feature extraction includes outputting the yaw rate during turning and the longitudinal acceleration / deceleration during acceleration or braking. CAN bus feature extraction includes braking signals, steering signals, and throttle signals.

[0037] Preferably, the feature interaction analysis submodule is used for multimodal feature fusion. Specifically, a bidirectional attention mechanism is used to perform feature interaction analysis on video frame features, point cloud features, IMU features, and CAN bus features extracted from the original data. That is, the video frame features, point cloud features, IMU features, and CAN bus features of each modality are used as both queries and keys, and the attention weight between each pair of feature modalities is calculated. For example, video features are used as queries and point cloud features are used as keys to calculate vision-point cloud attention; CAN features are used as queries and IMU features are used as keys to calculate control-inertial attention. The attention between all modalities is iteratively calculated through a multi-layer Transformer structure, thereby establishing the feature interaction analysis results, which include the original feature information and the correlation weights between different modalities. Finally, using the feature interaction analysis results, the occurrence intervals of key events are identified through event detection. Each event is then used as a graph node, and the directed edges between nodes represent the temporal and logical relationships between events, ultimately constructing an accident semantic graph to intuitively display key events and their corresponding relationships.

[0038] Furthermore, the specific configuration of the segmentation module 30 also includes a node segmentation submodule, which is used to establish a first segmentation constraint after reading the feature interaction analysis results in the semantic time slice channel, perform semantic event perception on the original sensor data stream, establish a second segmentation constraint, use the first segmentation constraint and the second segmentation constraint to perform accident semantic node identification under interactive authentication, perform accident semantic node slicing, and establish a semantic time slice; and a graph construction submodule, which configures the semantic label encoding of the semantic time slice, performs interactive feature node aggregation in each semantic time slice, wherein each node contains structured variables and unstructured summaries, connects adjacent time slice nodes with directed edges, configures time delay labels, and uses semantic label encoding, nodes, and directed edges to establish an accident semantic graph.

[0039] Preferably, the node segmentation submodule is used to accurately identify the start and end boundaries of key events from a continuous, fused feature stream and slice them. Specifically, it reads the feature interaction analysis results in the semantic time slice channel and establishes the first segmentation constraint, that is, using gradient detection and energy change detection algorithms to analyze the abrupt change points of the feature interaction analysis results and determine the potential time points where the scene changes significantly; it performs semantic event perception on the original sensor data stream and establishes the second segmentation constraint, that is, using a pre-trained temporal convolutional network classifier to scan the original sensor data stream and directly detect and perceive specific semantic events, such as braking, steering, and collision; then, it uses the first and second segmentation constraints to perform accident semantic node identification under interactive authentication, that is, it compares and authenticates the first and second segmentation constraints, satisfying that there is a significant change in the fused features near the time point, and that the time interval is classified as a meaningful semantic event, thereby determining the effective and credible accident semantic node boundary; then, it performs accident semantic node slicing, including cutting the original fused feature stream and data stream according to the start and end time points determined after authentication, to obtain multiple semantic time slices, each of which contains a complete semantic event.

[0040] Preferably, one-hot encoding or embedding vectors are used to configure explicit semantic label encoding for each semantic time slice. Then, interactive feature node aggregation is performed within each semantic time slice. That is, within each semantic time slice, average pooling or attention pooling is used to aggregate all fused features within that time period to generate a feature vector representing the overall information of the event. This transforms each semantic time slice into a node in the graph. Each node contains structured variables and unstructured summaries. Structured variables refer to quantitative information extracted from aggregated features, such as average deceleration, maximum steering angle, minimum distance to the target object, etc. Unstructured summaries refer to event text summaries automatically generated by the model, such as the vehicle triggering AEB 15 meters away from the vehicle in front. Next, adjacent nodes are connected by directed edges in chronological order, with the direction indicating the order in which the events occur. Each edge is assigned a time delay label to record the time interval between the end of the previous event and the start of the next event, accurately describing the temporal relationship between events. Finally, an accident semantic graph is established, where each node is a semantic event, containing semantic label encoding, structured variables, and unstructured summaries. Each directed edge connects two consecutive events and includes a time delay label.

[0041] The causal analysis module 40 is used to input the accident semantic map and the original sensor data stream into the window causal recognition model and output a causal attribution score with time resolution.

[0042] Furthermore, the specific configuration of the causal analysis module 40 also includes: a parsing submodule, used to parse the original sensor data stream, obtain the multi-source observation stream and the set of structured variables, vectorize the accident semantic graph, establish a semantic embedding matrix, encode the set of structured variables into a hybrid feature vector, and perform time-series synchronization of the multi-source observation stream, the semantic embedding matrix, and the hybrid feature vector within a unified sliding time window; a window causal identification submodule, used to call the window causal identification model to perform causal chain analysis within each sliding window, wherein the causal chain analysis includes correlation causal inference based on key accident nodes in the accident semantic graph; and an output submodule, used to establish and output a causal attribution score based on the causal chain analysis.

[0043] Preferably, the original sensor data stream is decomposed into multi-source observation streams from different sources using a parsing submodule, such as separating independent video feature streams and radar feature streams, and extracting a set of quantifiable structured variables, such as velocity values, acceleration values, distance values, and steering angles. Then, each semantic node in the accident semantic graph is vectorized using an embedding model, and the vectors of all nodes are arranged in chronological order to form a semantic embedding matrix. The set of structured variables is then standardized and encoded, and fused into a hybrid feature vector representing the low-level physical state. Next, the multi-source observation streams, semantic embedding matrix, and hybrid feature vectors are synchronized in time within a unified sliding time window. That is, a sliding time window is defined, and the multi-source observation streams, semantic embedding matrix, and hybrid feature vectors are aligned on the time axis within each window to ensure that the state is described at the same moment. Finally, the synchronized multi-level data blocks aligned within each time window are output.

[0044] Preferably, the window causal identification model is invoked to perform causal chain analysis within each sliding window. This model is a prediction model based on a causal discovery algorithm, such as time series prediction. If one time series better predicts another, a causal relationship exists. The window causal identification model then analyzes the conditional dependencies between elements in the synchronized data block within the current time window. Based on key accident nodes in the accident semantic graph, causal inference is performed, prioritizing key accident nodes as endpoints to infer the most likely direct and indirect causes. For example, it analyzes whether the collision was caused by the sudden braking of the vehicle in front and the following vehicle following too closely, quantifying the contribution of each cause. This outputs a local causal chain or causal graph within the current time window, representing the causal relationship within that time period. Each identified causal chain is then quantitatively evaluated, and a corresponding causal attribution score is calculated to measure the strength of the cause's influence on the result and the confidence level of the causal relationship. Finally, a time-resolved causal attribution score is output, clearly indicating the causal relationship, precisely pinpointing the time window in which the causal relationship occurred, and attaching a quantified attribution score.

[0045] Furthermore, the specific configuration of the causal analysis module 40 also includes a latent factor extraction submodule, which is used to extract latent factors based on the weather and environmental data and the original sensor data stream after acquiring the weather and environmental data, and establish latent influence features; and an enhanced identification submodule, which is used to perform causal analysis after enhancing the attention of the window causal identification model based on the latent influence features.

[0046] Preferably, weather and environmental data are acquired, including precipitation, visibility, road surface dryness / wetness, ambient light intensity, and wind speed. Then, latent factors are extracted based on the weather and environmental data and the raw sensor data stream. Specifically, a multilayer perceptron (MLP) is used to map the input weather and environmental data to a low-dimensional feature vector, encoding the potential impact patterns of the weather and environmental data on driving behavior, and outputting latent impact features to quantitatively represent the overall impact of the external environment on the accident scenario. Next, attention enhancement is performed on the window causal recognition model based on the latent impact features, including concatenating and fusing the latent impact features with other features in the causal model and adjusting the attention weights within the window causal recognition model. Finally, the window causal recognition model enhanced with latent impact features is used to re-execute causal analysis, outputting causal chains and causal attribution scores, thereby improving the accuracy of causal attribution.

[0047] The authentication module 50 is used to extract causal conclusions based on causal attribution scores, use the causal conclusions as prior constraints, perform accident scene reconstruction under generator-discriminator adversarial approach, establish a candidate restoration set, perform confidence authentication of the candidate restoration set under multiple optimization indicators, and establish restoration results.

[0048] Preferably, causal conclusions are extracted and encoded as prior constraints based on causal attribution scores, which are inviolable physical logic rules. These include causal relationships that any generated scene must satisfy. For example, in the generated accident animation, the vehicle's braking reaction must occur after sensing the sudden braking of the vehicle in front, and the distance between the two vehicles must satisfy collision physics to ensure the causal rationality of the generated scene. Then, accident scene reconstruction under a generator-discriminator adversarial approach is performed. The generator generates a complete and continuous accident scene based on the original, noisy, or incomplete multi-source sensor data, such as vehicle trajectory, speed changes, and attitude sequences. The discriminator simultaneously receives real sensor data fragments and scene data generated by the generator, and determines whether the scene generated by the generator is real or fake. Specifically, the generator attempts to generate a scene that is as realistic as possible and conforms to causal constraints, while the discriminator attempts to distinguish between real data and generated scenes. Based on the feedback from the discriminator, the generator continuously adjusts its parameters to generate a scene that can better deceive the discriminator while still satisfying causal constraints. After multiple iterations, the generator can produce highly realistic and causally reasonable scenes, thereby generating multiple candidate scenes to form a candidate reconstruction set.

[0049] Furthermore, the specific configuration of the authentication module 50 also includes a multi-indicator construction submodule, used to construct a multi-indicator evaluation function, wherein the evaluation indicators of the multi-indicator evaluation function include physical consistency indicators, data matching degree indicators, and causal rationality indicators; and a confidence authentication submodule, used to perform confidence authentication of the candidate restoration set according to the multi-indicator evaluation function, filter the candidate restoration set according to the confidence authentication trust level, and obtain the restoration result.

[0050] Preferably, a multi-optimization metric-based confidence assessment of the candidate reconstruction set is performed. This involves using a multi-metric evaluation function to comprehensively evaluate the confidence level of each scenario in the candidate reconstruction set. This includes physical consistency metrics, data matching metrics, and causal rationality metrics. Specifically, the physical consistency metric assesses whether the generated scenario conforms to physical laws such as Newtonian mechanics. This is evaluated by calculating whether simulated physical quantities are conserved before and after key events, or whether the vehicle trajectory conforms to the constraints of the dynamic model, such as momentum conservation and energy conservation. For example, whether the vehicle's trajectory after a collision conforms to a physical projectile model. The data matching metric assesses the degree of agreement between the generated scenario and the actual recorded original sensor data. This is achieved by calculating the relationship between the physical quantities in the generated scenario and the actual sensor measurements. The root mean square error (RMSE) is used to measure the difference between the generated vehicle location and the actual location detected by the radar. The causal rationality index is used to evaluate whether the generated scenario strictly follows the prior causal constraints and to check whether the conditions in the causal chain are met in the generated scenario, such as the cause must precede the effect and the causal strength must match the attribution score. Finally, the weighted comprehensive confidence score of each candidate reconstruction scenario under multiple indicators is calculated, and the scenario with the highest comprehensive confidence score is selected as the final and most reliable reconstruction result. This includes the final selected high-confidence accident scenario sequence and the scores of each indicator. It is the optimal calculation estimate of the entire accident occurrence process and maximizes the satisfaction of the original data matching, physical law constraints, and causal logic constraints, thereby ensuring the accuracy and reliability of road traffic accident reconstruction.

[0051] Furthermore, the specific configuration of the authentication module 50 also includes a discriminator construction submodule, used to construct a residual distribution authentication discriminator based on the prior constraints and the original sensor data stream; and an accident scene reconstruction submodule, used to obtain the vehicle model of the associated vehicles, take the current accident state as the final fitting result, perform accident scene reconstruction based on the generator, and perform reconstruction authentication through the discriminator, execute multiple rounds of adversarial iteration, and establish a candidate reconstruction set.

[0052] Preferably, a residual distribution authentication discriminator is constructed based on prior constraints, the original sensor data stream, and a neural network model. This discriminator learns the residual distribution pattern of real data under prior constraints. Specifically, for real sensor data, the discriminator calculates whether it conforms to the prior constraints. For example, if the constraint is that a collision will inevitably lead to a sudden decrease in velocity, the discriminator verifies whether the velocity change before and after the collision in the real data meets this constraint. The calculated difference is the residual. Then, a large amount of real data and its residuals are input into the discriminator, enabling it to learn the residual distribution that should be present in data that conforms to real physics and causal laws. Finally, the residual distribution authentication discriminator is trained and obtained. The authenticity of the input data is indirectly authenticated by analyzing whether the residual between the input data and the prior constraints conforms to the learned real residual distribution. Accurate dynamic models are established for the vehicles involved in the accident, including parameters such as vehicle mass, size, tire friction coefficient, suspension characteristics, and engine / brake response curves. The current accident state is then used as the final fitting result. The accident scene is reconstructed based on the generator, and a complete sequence of accident scene parameters is output, such as the control input parameters of each vehicle at each moment, including steering wheel angle, throttle, and brake opening. These control input parameters are then input into the vehicle model of the associated vehicles for simulation, thereby reconstructing the specific details of the entire accident process, such as high-precision vehicle trajectory, speed, and acceleration sequences.

[0053] Preferably, the scene simulation results reconstructed by the generator are fed into a residual distribution authentication discriminator. The discriminator performs reconstruction authentication, including calculating the residual between the generated scene and the prior constraints, and determining whether the residual falls within the range of the true residual distribution, thereby giving an authentication score to characterize the scene's realism. Then, multiple rounds of adversarial iteration are executed. The generator aims to generate a scene that not only fits the current accident state, such as the final collision position and vehicle posture, but also obtains a high authentication score from the discriminator. The generator continuously adjusts its parameters based on the discriminator's feedback, attempting to generate scenes that can better deceive the discriminator. Finally, multiple candidate reconstruction scenes that can better fit the final accident state are obtained, forming a candidate reconstruction set.

[0054] Furthermore, the road traffic accident reconstruction system applicable to intelligent connected vehicles also includes a visualization display module, which is used to output the reconstruction result as a three-dimensional visualization reconstruction scene, and to perform scene data identification of the three-dimensional visualization reconstruction scene at key nodes, and to perform visualization display based on the three-dimensional visualization reconstruction scene with scene data identification.

[0055] Preferably, a 3D graphics engine is invoked to output the reconstruction results as a 3D visualized reconstructed scene, including environmental reconstruction and vehicle modeling. Specifically, based on the accident location information, a corresponding high-precision 3D map model is loaded, including lane lines, road signs, traffic lights, buildings, etc. Then, vehicle trajectory and attitude data are mapped onto the vehicle model in the 3D scene, driving the model to move and generating a continuous 3D animation that accurately reproduces the entire accident process. Then, scene data labels are added to the 3D visualized reconstructed scene at key nodes, which may include the moment before the collision, the instant braking begins, etc. Data labels are overlaid on this 3D scene, such as displaying key physical quantities next to or above the vehicle model, drawing arrows to indicate the direction of speed or acceleration, drawing fan-shaped areas to indicate the sensor detection range or driver's field of vision, directly annotating the event with text, or displaying causal conclusions. Finally, the 3D visualized reconstructed scene with scene data labels is visualized, including outputting an interactive 3D animation demonstration interface. Users can play / pause / jump, view from multiple perspectives, view the displayed data labels in detail, understand the vehicle status and the cause of the event, and choose to show or hide certain data labels.

[0056] In the above text, refer to Figure 1 A road traffic accident reconstruction system suitable for intelligent connected vehicles according to an embodiment of the present invention is described in detail. Next, reference will be made to... Figure 2 This invention describes a method for reconstructing road traffic accidents applicable to intelligent connected vehicles, according to embodiments of the present invention. The method for reconstructing road traffic accidents applicable to intelligent connected vehicles includes, for example... Figure 2 As shown, the method includes: obtaining the current accident status of a road traffic accident; locating a set of associated devices based on the current accident status, the set of associated devices including associated vehicles and associated roadside equipment; after setting a global time axis, performing data collection on associated vehicles and associated roadside equipment to establish an original sensor data stream, the original sensor data stream undergoing time correction through the global time axis; after performing feature interaction analysis on the original sensor data stream, performing accident semantic node slicing of the original sensor data stream using a semantic time slice channel to establish an accident semantic graph; inputting the accident semantic graph and the original sensor data stream into a window causal recognition model, outputting a causal attribution score with time resolution; after extracting causal conclusions based on the causal attribution score, using the causal conclusions as prior constraints, performing accident scene reconstruction under generative-discriminative adversarial methods to establish a candidate restoration set, performing confidence authentication of the candidate restoration set under multiple optimization indicators, and establishing a restoration result.

[0057] In one possible implementation, the road traffic accident reconstruction method for intelligent connected vehicles further includes: extracting features from the original sensor data stream to establish video frame features, point cloud features, IMU features, and CAN bus features. The video frame features include target detection and behavior recognition; the point cloud features include obstacle distance and relative speed; the IMU features include yaw rate and longitudinal acceleration; and the CAN bus features include braking signal, steering signal, and throttle signal. The method also involves using bidirectional attention to perform feature interaction analysis within the video frame features, point cloud features, IMU features, and CAN bus features, establishing feature interaction analysis results, and using these results to construct an accident semantic graph.

[0058] In one possible implementation, the road traffic accident reconstruction method applicable to intelligent connected vehicles further includes: after reading the feature interaction analysis results in the semantic time slice channel, establishing a first segmentation constraint, performing semantic event perception on the original sensor data stream, establishing a second segmentation constraint, using the first segmentation constraint and the second segmentation constraint to perform accident semantic node identification under interactive authentication, performing accident semantic node slicing, and establishing a semantic time slice; after configuring the semantic label encoding of the semantic time slice, performing interactive feature node aggregation in each semantic time slice, wherein each node contains structured variables and unstructured summaries, connecting adjacent time slice nodes with directed edges, configuring time delay labels, and establishing an accident semantic graph using semantic label encoding, nodes, and directed edges.

[0059] In one possible implementation, the road traffic accident reconstruction method for intelligent connected vehicles further includes: parsing the original sensor data stream, obtaining a multi-source observation stream and a set of structured variables, vectorizing the accident semantic graph, establishing a semantic embedding matrix, encoding the set of structured variables into a hybrid feature vector, and synchronizing the multi-source observation stream, semantic embedding matrix, and hybrid feature vector in a unified sliding time window; calling a window causal identification model to perform causal chain analysis within each sliding window, wherein the causal chain analysis includes performing correlation causal inference based on key accident nodes in the accident semantic graph; and establishing and outputting a causal attribution score based on the causal chain analysis.

[0060] In one possible implementation, the road traffic accident reconstruction method applicable to intelligent connected vehicles further includes: constructing a residual distribution authentication discriminator based on the prior constraints and the original sensor data stream; after obtaining the vehicle model of the associated vehicle, taking the current accident state as the final fitting result, reconstructing the accident scene based on the generator, and performing reconstruction authentication through the discriminator, executing multiple rounds of adversarial iteration, and establishing a candidate reconstruction set.

[0061] In one possible implementation, the road traffic accident reconstruction method applicable to intelligent connected vehicles further includes: constructing a multi-index evaluation function, wherein the evaluation indicators of the multi-index evaluation function include physical consistency index, data matching degree index, and causal rationality index; and a confidence authentication submodule, used to perform confidence authentication of the candidate reconstruction set according to the multi-index evaluation function, filter the candidate reconstruction set according to the confidence authentication trust level, and obtain the reconstruction result.

[0062] In one possible implementation, the road traffic accident reconstruction method for intelligent connected vehicles further includes: after acquiring weather environment data, extracting latent factors based on the weather environment data and the original sensor data stream to establish latent influence features; and performing causal analysis after enhancing the attention of the window causal identification model based on the latent influence features.

[0063] In one possible implementation, the road traffic accident reconstruction method for intelligent connected vehicles further includes: outputting the reconstruction result as a three-dimensional visualization reconstruction scene, identifying the scene data of the three-dimensional visualization reconstruction scene at key nodes, and visually displaying the three-dimensional visualization reconstruction scene with scene data identification.

[0064] In one possible implementation, the road traffic accident reconstruction method applicable to intelligent connected vehicles further includes: performing data criticality authentication on the data collection results of associated vehicles and associated roadside equipment to generate data criticality identifiers; performing source credibility authentication on the data collection results to establish data credibility identifiers; using the criticality identifiers and the credibility identifiers to perform data interaction authentication on the data collection results, and using the data interaction authentication results to establish the original sensor data stream.

[0065] The road traffic accident reconstruction system for intelligent connected vehicles provided in the embodiments of the present invention can execute the road traffic accident reconstruction method for intelligent connected vehicles provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0066] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0067] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A road traffic accident reconstruction system suitable for intelligent connected vehicles, characterized in that, The system includes: The positioning module is used to obtain the current accident status of a road traffic accident and locate a set of associated devices based on the current accident status. The set of associated devices includes associated vehicles and associated roadside equipment. The calibration module is used to perform data acquisition of associated vehicles and associated roadside equipment after setting the global time axis, and establish an original sensor data stream. The original sensor data stream is time-calibrated through the global time axis. The segmentation module is used to perform feature interaction analysis on the original sensor data stream, and then use the semantic time slice channel to slice the accident semantic nodes of the original sensor data stream to establish an accident semantic graph. The causal analysis module is used to input the accident semantic graph and the raw sensor data stream into the window causal recognition model and output a causal attribution score with time resolution. The authentication module is used to extract causal conclusions based on causal attribution scores, use the causal conclusions as prior constraints, perform accident scene reconstruction under generator-discriminator adversarial approach, establish a candidate restoration set, perform confidence authentication of the candidate restoration set under multiple optimization indicators, and establish restoration results. The segmentation module includes: The feature segmentation submodule is used to extract features from the original sensor data stream and establish video frame features, point cloud features, IMU features, and CAN bus features. The video frame features include target detection and behavior recognition. The point cloud features include obstacle distance and relative velocity. The IMU features include yaw rate and longitudinal acceleration. The CAN bus features include braking signal, steering signal, and throttle signal. The feature interaction analysis submodule is used to perform feature interaction analysis within video frame features, point cloud features, IMU features, and CAN bus features using bidirectional attention, establish feature interaction analysis results, and use the feature interaction analysis results to complete the construction of an accident semantic graph. The segmentation module further includes: The node segmentation submodule is used to establish a first segmentation constraint after reading the feature interaction analysis results in the semantic time slice channel, perform semantic event perception on the original sensor data stream, establish a second segmentation constraint, use the first segmentation constraint and the second segmentation constraint to perform accident semantic node identification under interactive authentication, perform accident semantic node slicing, and establish a semantic time slice. The graph construction submodule configures the semantic label encoding of semantic time slices, aggregates interactive feature nodes within each semantic time slice, where each node contains structured variables and unstructured summaries, connects adjacent time slice nodes with directed edges, configures time delay labels, and uses semantic label encoding, nodes, and directed edges to build an accident semantic graph. The authentication module includes: The discriminator construction submodule is used to construct a residual distribution authentication discriminator based on the prior constraints and the original sensor data stream; The accident scene reconstruction submodule is used to obtain the vehicle model of the associated vehicles, take the current accident state as the final fitting result, reconstruct the accident scene based on the generator, and perform reconstruction authentication through the discriminator. It then performs multiple rounds of adversarial iteration to establish a candidate reconstruction set. Also includes: The multi-indicator construction submodule is used to construct a multi-indicator evaluation function. The evaluation indicators of the multi-indicator evaluation function include physical consistency indicators, data matching degree indicators, and causal rationality indicators. The confidence authentication submodule is used to perform confidence authentication of the candidate restoration set according to the multi-index evaluation function, filter the candidate restoration set according to the confidence authentication trust level, and obtain the restoration result.

2. The road traffic accident reconstruction system for intelligent connected vehicles as described in claim 1, characterized in that, The causal analysis module includes: The parsing submodule is used to parse the original sensor data stream, obtain the multi-source observation stream and the set of structured variables, vectorize the accident semantic map, establish a semantic embedding matrix, encode the set of structured variables into a hybrid feature vector, and perform time synchronization of the multi-source observation stream, the semantic embedding matrix and the hybrid feature vector within a unified sliding time window. The window causal identification submodule is used to call the window causal identification model to perform causal chain analysis within each sliding window, wherein the causal chain analysis includes correlation causal reasoning based on key accident nodes in the accident semantic graph; The output submodule is used to build and output causal attribution scores based on causal chain analysis.

3. The road traffic accident reconstruction system for intelligent connected vehicles as described in claim 1, characterized in that, The causal analysis module also includes: The latent factor extraction submodule is used to extract latent factors based on the weather and environmental data and the original sensor data stream after acquiring the weather and environmental data, and to establish latent influence features. The enhanced identification submodule is used to perform causal analysis after enhancing the attention of the window causal identification model based on the latent influence characteristics.

4. The road traffic accident reconstruction system for intelligent connected vehicles as described in claim 1, characterized in that, The system also includes: The visualization module is used to output the restoration result as a 3D visualization reproduction scene, and to perform scene data identification of the 3D visualization reproduction scene at key nodes, and to perform visualization display based on the 3D visualization reproduction scene with scene data identification.

5. The road traffic accident reconstruction system for intelligent connected vehicles as described in claim 1, characterized in that, In the correction module, establishing the original sensing data stream includes: Data criticality authentication is performed on the data collection results of associated vehicles and associated roadside equipment, and data criticality labels are generated; Perform source credibility verification of data collection results and establish data credibility identifiers; The keyness identifier and the trustworthiness identifier are used to perform data interaction authentication of the data collection results, and the original sensor data stream is established using the data interaction authentication results.

6. A method for reconstructing road traffic accidents applicable to intelligent connected vehicles, characterized in that, The method is applied to the road traffic accident reconstruction system for intelligent connected vehicles as described in any one of claims 1-5, and the method includes: Obtain the current accident status of a road traffic accident, and locate the associated device set based on the current accident status. The associated device set includes associated vehicles and associated roadside equipment. After setting the global timeline, data collection is performed on associated vehicles and associated roadside equipment to establish an original sensor data stream. The original sensor data stream undergoes time correction through the global timeline. After performing feature interaction analysis on the original sensor data stream, the semantic time slice channel is used to slice the accident semantic nodes of the original sensor data stream to establish an accident semantic graph. The accident semantic graph and the raw sensor data stream are input into the window causal recognition model, and the causal attribution score with time resolution is output. After extracting causal conclusions based on causal attribution scores, these causal conclusions are used as prior constraints to perform accident scene reconstruction under generative-discriminative adversarial conditions, establish a candidate restoration set, perform confidence authentication of the candidate restoration set under multiple optimization indicators, and establish restoration results.

Images