A method and system for analyzing and visualizing traffic accident data of an autonomous driving vehicle

Abstract

The present invention relates to a method and system for analyzing and visualizing traffic accident data for autonomous vehicles. The method comprises: collecting raw data related to vehicle traffic accidents and standardizing and integrating the raw data; cleaning and verifying the standardized and integrated data, time-calibrating, and synchronizing the data to obtain processed data; obtaining the collision time point based on the speed and path of the accident vehicle; analyzing the driving mode at the time of collision, the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, the vehicle fault before the collision, the traffic signals and road markings before the collision, and the vehicle status after the collision based on the processed data and the collision time point to obtain key time points; and visually displaying the analysis results based on the processed data and the key time points. The present invention can achieve key time point marking and automatically generate and save scientific and standardized accident analysis reports.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic accident data processing for autonomous driving vehicles, and in particular to a method and system for analyzing and visualizing traffic accident data for autonomous driving vehicles. Background Art

[0002] With the gradual development of autonomous driving technology in China, the number of autonomous vehicles undergoing road testing continues to increase. Actual road testing of autonomous vehicles reveals a frequent occurrence of traffic violations and accidents. Unlike traditional vehicles, autonomous vehicles possess both vehicle and driver characteristics. The primary vehicle controller shifts from a single human driver to a collaborative driver of the autonomous driving system. Traditional accident investigation and evidence collection methods are no longer applicable to autonomous vehicle accident investigations, creating new demands and challenges for road traffic management. Currently, road testing regulations issued by autonomous driving test zones and regulatory agencies across China require autonomous driving test vehicles to have vehicle status recording, storage, and online monitoring capabilities. These capabilities include real-time feedback on vehicle control mode, location, speed, acceleration, and other motion states. The vehicles also automatically record and store environmental perception and response status, the real-time status of vehicle lights and signals, 360-degree video surveillance of the vehicle's exterior, in-vehicle video and voice surveillance of the tester's interaction with the driver, remote control commands received by the vehicle, and vehicle fault information, at least 90 seconds prior to the occurrence of an accident or failure.

[0003] Conducting road traffic accident investigations involving autonomous vehicles and determining liability requires fusion analysis of multiple sources, including perception, decision-making, control, and human-machine interaction, based on the actual vehicle operating data during the accident. This allows for a determination of safety hazards in the autonomous driving system of the accident vehicle, thereby determining the cause of the accident and liability. This necessitates the development of methods for collecting, processing, and analyzing accident data. However, the volume of data stored in vehicle-side data recording devices is enormous, and the data structure cannot be directly viewed. Furthermore, the data recording period is often very long, resulting in complex and diverse data. This presents a challenge for public security traffic management departments in extracting key clues and evidence during accident handling. The extraction and analysis methods for these key clues and evidence are cumbersome, time-consuming, and labor-intensive, leading to inefficient law enforcement. Therefore, it is crucial to develop a method that can quickly and effectively parse and visualize the data stored in autonomous vehicle data recording devices during an accident.

[0004] The existing technology, "A Multi-Dimensional Information Viewing System for Autonomous Vehicles," proposes collecting existing autonomous vehicle data scattered across various systems, using stream segmentation to simultaneously receive and play it, and displaying it as data blocks on a timeline, enabling synchronized playback of multi-dimensional data at the timeline cursor position. However, this technology only lists a subset of data types, primarily focusing on viewing and playback, lacking data analysis and visualization capabilities. Its limited functionality fails to meet the time synchronization requirements for data and in-vehicle and in-vehicle video surveillance.

[0005] The prior art "Method and Device for Analyzing Accident Responsibility for Autonomous Vehicles" proposes an accident responsibility analysis method based on cloud-based data analysis and simulation technology. This method analyzes and integrates data uploaded to a cloud platform, using simulation software to reconstruct the road conditions, vehicle trajectories, and information about traffic participants at the time of the accident, creating an accident scenario. Based on the accident scenario and relevant data from the autonomous vehicle, the method determines the driving mode at the time of the accident, analyzes the cause of the accident, and determines responsibility. However, this patent determines driver responsibility solely based on driving mode, making the method overly simplistic and unable to meet the practical needs of accident handling and liability determination. Furthermore, data upload to the cloud is based on trigger rules, which can easily omit relevant accident data.

[0006] The prior art "A road traffic accident data mining server, method and system" proposes an accident data mining method and system, which includes a data preprocessing module, an analysis module, a mining module and a visualization module to address the shortcomings of traditional vehicle traffic accident statistical analysis methods. However, this invention conducts macro-mining analysis and point statistics on historical data, rather than micro-analysis of a single autonomous driving vehicle accident, and uses macro-statistical indicators such as mortality rate, without involving specific parameters required for analysis.

[0007] The existing technology, "A Traffic Accident Data Integration and Analysis Method and System," proposes integrating and analyzing historical traffic accident data to establish traffic accident trend prediction models, traffic accident frequency models, and traffic accident impact models. These models calculate accident data values, traffic accident frequency, and traffic accident impact values, and issue warnings when these values ​​exceed preset thresholds. This invention relies on historical traffic accident data for analysis and provides warnings of potential accidents. The model used is simple and lacks practical traffic accident analysis methods.

[0008] The prior art "A Natural Driving Data Collection Device" proposes a natural driving data collection device that includes scene data collection equipment, vehicle-mounted data collection equipment, and vehicle-mounted intelligent screening equipment. This device uses multiple algorithms, including vehicle-following, overtaking, and lane-changing algorithms, to process and analyze the collected data. It primarily collects scene data required for intelligent connected vehicle testing, rather than accident data. The data items may differ from those required for accident analysis, and it lacks correlation analysis between vehicle trajectory data and accident environment data.

[0009] In summary, existing technical solutions primarily focus on managing historical accident data and conducting macro-level mining and analysis, rather than analyzing data from a single autonomous vehicle accident. Therefore, the present invention provides a method and system for analyzing and visualizing traffic accident data for autonomous vehicles to address the aforementioned technical issues. Summary of the Invention

[0010] The purpose of the present invention is to provide a method and system for analyzing and visualizing traffic accident data of autonomous driving vehicles, which is used to analyze the causes of autonomous driving vehicle accidents, realize accident data analysis and visualization, and mark key time points. It can automatically generate and save scientific and standardized accident analysis reports, guide every link of autonomous driving vehicle accident handling, and make various data analysis indicators traceable.

[0011] To achieve the above object, the present invention provides the following solutions:

[0012] A method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, comprising:

[0013] Collecting original data related to vehicle traffic accidents and standardizing and integrating the original data;

[0014] Clean, verify, time-calibrate, and synchronize the standardized and integrated data to obtain the processed data;

[0015] Obtain the collision time point based on the speed and path of the accident vehicle;

[0016] Analyzing the driving mode at the time of collision, the vehicle state before the collision, the traffic environment before the collision, and the vehicle state after the collision based on the processed data and the collision time point to obtain the key time point;

[0017] The analysis results are visualized based on the processed data and the key time points.

[0018] Optionally, collecting raw data related to vehicle traffic accidents and standardizing and integrating the raw data includes:

[0019] The raw data is collected by an on-board terminal device and manual recording, wherein the on-board terminal device collects data from the CAN bus, positioning and monitoring video, and the manual recording collects environmental information, safety officer information, collision area and casualty data;

[0020] The raw data is separated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data and audio and video data according to a standardized format.

[0021] Optionally, the standardized and integrated data is cleaned, verified, time-calibrated, and synchronized, including:

[0022] The raw data is pre-processed to eliminate redundancy, smooth conflicts, and fill gaps, the pre-processed data is arranged in time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format;

[0023] Obtaining different data sampling frequencies, presetting a time axis according to a minimum frequency and performing frequency filling, and performing data frequency alignment on the preprocessed data;

[0024] The pre-processed data are aligned for data length according to a preset reference point of a time frame.

[0025] Optionally, obtaining the collision time based on the speed and path of the accident vehicle includes:

[0026] The three-axis acceleration values ​​of the accident vehicle are obtained. The three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​first change suddenly is obtained, which is recorded as T. a ;

[0027] Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three axes and the time interval is greater than the first-order difference sum, which is recorded as T b ;

[0028] Get the time point when the accident vehicle deviates from the expected path, recorded as T c ;

[0029] Select T a ,T b ,T c The point that is closest in time to the collision point is taken as the collision time point.

[0030] Optionally, the key time points include: maximum acceleration time point, maximum speed time point, speed 0 time point, driving mode switching time point, manual intervention type switching time point, steering state switching time point, steering wheel angle is 0 for the first time, steering wheel angle lowest peak time point, steering wheel angle highest peak time point, automatic driving control braking is switched to safety officer stepping on the brake pedal time point, safety officer stepping on the brake pedal is switched to automatic driving control braking time point, braking lowest peak time point, braking highest peak time point, automatic driving control throttle is switched to safety officer stepping on the accelerator pedal time point, throttle lowest peak time point, throttle highest peak time point, light switching time point, horn switching time point, wiper switching time point, vehicle failure time point, parking state switching time point, reversing state switching time point.

[0031] Optionally, analyzing the vehicle state before the collision includes analyzing the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, and the vehicle faults before the collision;

[0032] Analyzing the vehicle pedals before the collision includes: obtaining the braking status and throttle status based on the vehicle status data, and determining the safety officer's takeover method based on the time point when the steering wheel angle first reaches 0, the time point when the braking status first switches from a negative number to a positive number, and the time point when the throttle status first reaches zero.

[0033] On the other hand, it also provides a traffic accident data analysis and visualization processing system for autonomous driving vehicles, which includes a data reading and integration module, a data cleaning and verification module, a time calibration and synchronization module, a data analysis module, and a data visualization module;

[0034] The data reading and integration module is used to collect raw data related to vehicle traffic accidents and perform standardized integration on the raw data;

[0035] The data cleaning and verification module is used to clean and verify the normalized and integrated data;

[0036] The time calibration and synchronization module is used to perform time calibration and synchronization processing on the data after cleaning and verification processing to obtain the processed data;

[0037] The data analysis module is used to obtain the collision time point based on the speed and path of the accident vehicle, and analyze the driving mode at the time of collision, the vehicle state before collision, the traffic environment before collision, and the vehicle state after collision based on the processed data and the collision time point to obtain the key time point;

[0038] The data visualization module is used to visualize the analysis results based on the processed data and the key time points.

[0039] Optionally, the system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode was switched during the accident, whether there was a control system failure or communication delay, whether the vehicle was driving in accordance with the automatic driving system, whether excessive decision-making behavior was taken or relevant laws and regulations were violated based on the vehicle's motion status, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

[0040] The beneficial effects of the present invention are as follows: the present invention is used to analyze the causes of autonomous vehicle accidents, realize accident data analysis and visualization, realize key time point marking, and can automatically generate and save scientific and standardized accident analysis reports to guide every link in the handling of autonomous vehicle accidents, so that various data analysis indicators can be traced, basically meeting the needs of traffic police in handling autonomous vehicle traffic accidents, and effectively helping traffic police to improve their interactive understanding level, scientifically and clearly interpret key accident information, enrich the display form of accident reports, and comprehensively and objectively investigate accidents, providing support for traffic management departments to handle autonomous vehicle accidents efficiently and with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a flow chart of a method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to an embodiment of the present invention;

[0043] Figure 2 A comparison chart of the same time axis of the data display window of an embodiment of the present invention;

[0044] Figure 3 This is an example diagram of an accident timeline according to an embodiment of the present invention;

[0045] Figure 4 This is an example diagram of an accident restoration animation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] like Figure 1 As shown, this embodiment provides a method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, including:

[0049] Collect raw data related to vehicle traffic accidents and standardize and integrate the raw data;

[0050] Clean, verify, time-calibrate, and synchronize the standardized and integrated data to obtain the processed data;

[0051] Obtain the collision time point based on the speed and path of the accident vehicle;

[0052] Based on the processed data and the time of collision, the driving mode at the time of collision, the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, the vehicle faults before the collision, the traffic signals and road markings before the collision, and the vehicle status after the collision are analyzed to obtain the key time points. The key time points are the time points when the motion data of the accident vehicle changes and the time points when the vehicle status changes;

[0053] The analysis results are visualized based on the processed data and key time points.

[0054] Furthermore, the original data related to vehicle traffic accidents is collected and standardized and integrated, including:

[0055] Collect raw data through on-board terminal devices and manual recording. The on-board terminal devices collect data from the CAN bus, positioning, and surveillance video, while manual recording collects environmental information, safety officer information, collision area, and casualty data.

[0056] The raw data is integrated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data, and audio and video data according to a standardized format.

[0057] Specifically, the original data includes: data from the CAN bus, positioning and monitoring video collected by the on-board terminal equipment, as well as environmental information, safety officer information, collision area and degree of casualties collected through manual recording.

[0058] CAN bus data refers to data such as vehicle speed, braking, throttle, steering, driving mode, fault codes, etc. collected through the vehicle's CAN bus. Positioning data refers to the current location of the vehicle, which can usually be converted into longitude and latitude or coordinates, and is not limited to positioning technologies such as the Global Navigation Satellite System (GNSS), Real-Time Kinematic Positioning (RTK), Inertial Navigation (INS), Laser Detection and Ranging (LiDAR), or a combination of positioning technologies. Surveillance video refers to in-vehicle video of an autonomous vehicle collected by cameras and monitoring devices, including steering wheel and instrument panel perspectives, driver posture, and surrounding environment video, including forward perspectives, rear perspectives, and side perspectives. Manually recorded data refers to information directly obtained by traffic police during accident scene investigations, as well as information supplemented by safety officers and the companies involved.

[0059] The above raw data is standardized in a format, and the single accident data is divided into 8 types of data and audio and video data, including vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, and path planning data, and stored as 8+N data files, where N is the number of audio and video data files. Audio and video data can be stored in mp4, avi, or mov format, and other data except audio and video can be stored in csv, xlsx, xls, txt, or dat format. All data files are named according to fixed rules: naming prefix + 17-digit vehicle identification code (VIN) + 10-digit time (such as 10:00 on October 21, 2024 is expressed as 2024102110), separated by underscores.

[0060] Vehicle status data, named with the prefix STATUS, includes the autonomous vehicle's driving status, manual intervention type, braking status, steering status, throttle status, parking status, reverse status, lighting status, vehicle fault codes, horn and wiper status, and other information derived from the CAN bus at each point in time. Vehicle motion data, named with the prefix MOVE, includes the autonomous vehicle's latitude and longitude, speed (three-axis velocity in the ego vehicle coordinate system), acceleration (three-axis acceleration in the ego vehicle coordinate system), heading angle, pitch angle, and roll angle at each point in time, and other information derived from the positioning system. Traffic participant data, named with the prefix OTHER, includes the type, number, lateral and longitudinal distance relative to the autonomous vehicle, speed, heading angle, and dimensions (length, width, and height) of objects other than the autonomous vehicle at each point in time, and other information derived from the CAN bus and positioning. Environmental data, named with the prefix ENV, includes information derived from the CAN bus and manual recordings, such as brightness (lux), visibility range, rain, snow, fog, and temperature. Traffic light recognition data, named with the prefix LIGHT, includes the traffic light type, number, phase light status (red, yellow, and green corresponding to left turn, straight ahead, and right turn) at each time point, as well as the distance between the traffic light and the vehicle, and other information derived from the CAN bus and positioning. Traffic sign recognition data, named with the prefix SIGN, includes the traffic sign number, type, value, and distance between the traffic sign and the vehicle, and other information derived from the CAN bus and positioning. Lane line recognition data, named with the prefix LANE, includes the lane number, lane type, lane curvature, lane width, distance between the autonomous vehicle and the left lane line, and the type and color of the left and right lane lines, and other information derived from surveillance video conversion, at each time point. Path planning data, named with the prefix PLAN, includes the autonomous vehicle's expected planned longitude and latitude position, expected planned heading, expected planned speed, and the time difference between the expected planned position and the start planned position, and other information derived from the CAN bus and positioning. Audio and video data are named using prefixes based on video content: W (panoramic video), DV (perception interface video), DR (in-vehicle safety officer and steering wheel video), BK (pedal video), F (front view), LF (left front view), RF (right front view), B (back view), LB (left back view), RB (right back view), L (left view), and R (right view). The number of audio and video data files (N) depends on the number of cameras installed in the autonomous vehicle and should include at least six video channels: DV, DR, F, B, L, and R.

[0061] Furthermore, the standardized and integrated data is cleaned, verified, time-calibrated, and synchronized, including:

[0062] The raw data is pre-processed to eliminate redundancy, smooth conflicts and fill gaps, the pre-processed data is arranged in time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format;

[0063] Obtain the frequency of different data, preset the time axis according to the minimum frequency and perform frequency filling, and align the data frequency of the preprocessed data;

[0064] The pre-processed data is aligned with the data length according to the preset reference point of the time frame.

[0065] Specifically, because autonomous vehicles may encounter unusual road conditions such as potholes and speed bumps, and may engage in sudden deceleration, acceleration, and sharp turns, data denoising methods are required to reduce the impact of these conditions on data analysis. Furthermore, given that accidents may occur due to hardware damage, software defects, or abnormal vehicle conditions, these conditions can lead to missing or erroneous data, which require processing.

[0066] Eliminating redundant data refers to processing non-standard data, specifically automatically identifying duplicate data records, allowing customization of redundant data identification rules (such as the exact same values ​​of certain key fields at different time points), and choosing to delete or merge them.

[0067] Smoothing conflicting data refers to processing inconsistent data. Specifically, it automatically identifies outliers by calculating the standard deviation, allowing you to choose whether to retain or process outliers, and using methods such as moving smoothing or exponential smoothing. Moving smoothing takes a fixed-size window at the conflict point, calculates the average of all data points in the window, and replaces the original data. Exponential smoothing uses a smoothing factor α to weight the current conflict value. and the previous smoothed value is the smoothed value at time t2.

[0068]

[0069] Filling missing data means filling missing values, specifically detecting null values ​​in the data. If the data in the corresponding field exists, you can choose methods such as neighbor method or interpolation regression to fill the missing values. and The missing value at time t2 in the interval [t1,t3] Neighbor method usage Previous location or the latter To fill. Linear interpolation regression uses first-order spline curve interpolation filling,

[0070]

[0071] The quadratic interpolation uses the second-order spline curve interpolation filling to construct the quadratic interpolation function f(t) so that For i=0,1,3, the Lagrange interpolation basis functions L0(t), L1(t), L3(t) are defined as follows:

[0072]

[0073] When reading a data file, the visual interface needs to output the field name, number of redundant data, number of outlier data, number of missing data, and number of valid data for each field in the file one by one.

[0074] When reading a data file, the visual interface needs to output the field name, number of redundant data, number of outlier data, number of missing data, and number of valid data for each field in the file one by one.

[0075] Calibrate and align timestamps of multi-source data to provide accurate time parameters for subsequent time series analysis. This is accomplished through three steps: determining the timestamp format, aligning data frequency, and aligning data length.

[0076] The first step is to determine the timestamp format. For each data file, arrange each frame of data in time sequence, and form a complete video frame data sequence based on the timestamp of the audio and video data. UNIX (The number of milliseconds since 00:00 on January 1, 1970) is converted to Beijing time.

[0077]

[0078] The second step is to align the data frequency. Since different data files are collected from different devices, there may be different collection frequencies, so time frame alignment is required to ensure that the data corresponding to different fields at that moment can be found in a certain key time frame. Detect the sampling frequencies of different data files, set a unified time axis according to the minimum frequency, and fill in the corresponding frequencies. The filling method should be configured in advance according to different fields. For non-continuously changing fields such as driving status, manual intervention type, lighting status, target number, etc., where different values ​​represent different meanings, the preceding neighbor method should be used to fill. For continuously changing fields such as speed, acceleration, heading angle, target distance, etc., the interpolation method can be used to fill. At the same time, the filled data and non-filled data should be marked, and different colors should be used to distinguish and display them in the data visualization module.

[0079] The third step is to align data lengths. Because the time of the accident is unknown before data analysis, the pre- and post-crash data submitted by the company may have inconsistent data collection lengths, especially the lengths of data files and audio and video files. Therefore, a certain time frame is set as a reference point, and the data and audio and video files are cut 20 seconds before and after the reference point.

[0080] The default reference point is the collision time, and you can also customize the input time as the reference point (input format is); you can also manually select the reference point, that is, drag the audio and video progress bar to a certain position in the audio and video display window of the visual interface, select a time point in the data display window, and after confirmation, the system will use this time point as the reference point and automatically synchronize the data and audio and video before and after the reference point.

[0081] For audio and video files with timeframe annotations, you can directly match the timeframe of the data file to find the reference point and cut the video length to match the data file's timeframe. For audio and video files without timeframe annotations, use the manual reference point selection method described above.

[0082] Furthermore, according to the speed and path of the accident vehicle, obtaining the collision time point includes:

[0083] The three-axis acceleration values ​​of the accident vehicle are obtained. The three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​first change suddenly is obtained, which is recorded as T. a ;

[0084] Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three axes and the time interval is greater than the first-order difference sum, which is recorded as T b ;

[0085] Get the time point when the accident vehicle deviates from the expected path, recorded as T c ;

[0086] Select T a ,T b ,T c The point with the earliest time in the data is taken as the collision time point.

[0087] Specifically, define A t =(accx t ,accy t ,accz t) represents the acceleration values ​​of the three axes from the sensor at time t. Usually, a collision will cause a sharp change in the acceleration of the autonomous vehicle. The acceleration of the three axes will have significant peaks and oscillations. The threshold a of the acceleration change is set to search for the acceleration mutation point. The threshold can be customized. The sliding window mechanism is used to detect A at each moment. t ,||A t ||≥a, capture A t Time of first mutation appearance T a , T a It must exist.

[0088] If the change in the actual speed of the autonomous vehicle per unit time Δt is equal to the acceleration sensor value A t The difference is large and can also be considered as a collision. The time series velocity data v(t1), v(t2), ... v(t3), its first-order difference is Δv i , the sum of first-order differences is

[0089] Δv i =v(t i+1 )-v(t i )

[0090]

[0091] At discrete time points, the acceleration calculated based on the velocity change rate should be approximated as the velocity difference divided by the time interval Δt,

[0092]

[0093] Under normal motion conditions, the acceleration of an autonomous vehicle should be approximately equal to the rate of change of the actual velocity, that is, the sum of the first-order differences of the velocities.

[0094]

[0095] If the actual acceleration sensor value multiplied by the time interval is greater than the sum of the first-order differences of the velocity,

[0096]

[0097] Capture the actual acceleration sensor value multiplied by the time interval greater than the first-order difference of the velocity and the corresponding time T b If it does not exist, then T b =NULL.

[0098] In addition, the normal motion trajectory should be relatively smooth, but collisions can cause sudden turns or jumps in the trajectory. The autonomous vehicle may deviate from the expected path (except for hardware failures or other environmental changes). If the deviation between its position and the planned path is too large, it means there is a probability of collision. Define the deviation loss between the expected path and the actual driving path. The expected path coordinate is (x i ,y i ), the actual driving path coordinates are

[0099]

[0100] If there is a loss that does not converge at a certain moment,

[0101]

[0102] It is considered that the autonomous driving vehicle deviates from the expected path at this moment, and the time point T is captured. c If it does not exist, then T c =NULL.

[0103] Collision time T collision The default is T a ,T b ,T c The earliest point in time,

[0104] T collision =min(T a ,T b ,T c )

[0105] Furthermore, the key time points include: the maximum acceleration time point, the maximum speed time point, the speed is 0 time point, the driving mode switching time point, the manual intervention type switching time point, the steering state switching time point, the steering wheel angle is 0 for the first time, the steering wheel angle lowest peak time point, the steering wheel angle highest peak time point, the automatic driving control braking is switched to the safety officer stepping on the brake pedal time point, the safety officer stepping on the brake pedal is switched to the automatic driving control braking time point, the braking lowest peak time point, the braking highest peak time point, the automatic driving control throttle is switched to the safety officer stepping on the accelerator pedal time point, the throttle lowest peak time point, the throttle highest peak time point, the light switching time point, the horn switching time point, the wiper switching time point, the vehicle failure time point, the parking state switching time point, and the reversing state switching time point.

[0106] The driving mode at the time of collision, the vehicle state before collision, the traffic environment before collision and the vehicle state after collision are analyzed. The analysis of the vehicle state before collision includes: analysis of the vehicle steering before collision, the vehicle pedals before collision, the vehicle accessories before collision and the vehicle fault before collision. The analysis of the traffic environment before collision includes: analysis of traffic signals and road markings before collision and analysis of other targets before collision. Specifically,

[0107] (1) Analysis of driving patterns at the time of collision:

[0108] Based on the vehicle status data, search for the switching time points of the driving mode (drive_mode) and the manual intervention type (takeover_mode). A drive_mode value of 0 indicates manual driving, 1 indicates autonomous driving, 2 indicates remote driving, 4 indicates disengaged autonomous driving, and 9 indicates other. A takeover_mode value of 0 indicates no takeover, 1 indicates driver's position taking over, 2 indicates other positions taking over, and 3 indicates remote takeover.

[0109] Search whether drive_mode changes from 1 to other values ​​at a certain moment T drive_mode , whether takeover_mode changes from 0 to other values ​​at a moment T takeover_mode .

[0110] If there is no mutation, search drive_mode(T collision ) and takeover_mode(T collision ) corresponds to the driving status and manual intervention type, and determines whether the vehicle is in automatic driving state, automatic driving disengaged state, or manual takeover (main driver's position, other position or remote takeover) when a collision occurs.

[0111] If there is a mutation, compare the mutation time T drive_mode 、T takeover_mode and collision time T collision ,

[0112] If the car was in autonomous driving mode at the time of the collision, the system's response timeliness and driving strategy were evaluated, indicating that the car did not brake in time. If the car was in manual driving mode at the time of the collision, the safety officer did not brake in time.

[0113] (2) Analysis of vehicle steering before collision:

[0114] Based on the vehicle status data, search for the switching time point of the steering status (steering_status). A steering_status value in the range [-1000, 1000] indicates manual steering. The number represents the steering angle, with left turns being negative and right turns being positive. A value in the range (1000, 3000) indicates the steering angle of the automated driving system. Subtracting 2000 degrees produces the same steering angle as the value in the range [-1000, 1000].

[0115] Search steering_status to see if it has a value in (1000, 3000). If all of them are, the vehicle is in the automatic driving state during the accident. If a value is in (1000, 3000] and subsequently mutates to [-1000, 1000], record the mutation time point T steering status At this moment, the steering wheel is switched from the automatic driving system to manual control, and the above driving mode judgment is verified. All values ​​belonging to (1000, 3000] are mapped to [-1000, 1000], and the first time point T where steering_status = 0 is searched. steering＝0 , as well as the lowest peak steering_min, the highest peak steering_max and the corresponding time point T steering min 、T steering max , which is used to determine the stability of vehicle control during an accident, whether the autonomous driving system made a sharp turn to avoid a collision, and whether the safety officer took any collision avoidance action.

[0116] (3) Analyze the vehicle pedals before the collision:

[0117] Based on the vehicle status data, search for the switching time points between the braking state (brake_status) and the throttle state (accelerator_status). A brake_status value of 0 indicates a non-braking state, [1, 100] indicates a pedal-pressed braking state, and [-100, -1] indicates an autonomous driving system braking state. A larger absolute value indicates a stronger braking force. An accelerator_status value of 0 indicates no throttle application, [1, 100] indicates accelerator pedal application force, and [-100, -1] indicates an autonomous driving system automatically applying throttle force. A larger absolute value indicates a stronger throttle force.

[0118] Search brake_status to see if there is a positive number. If not, the vehicle is in automatic driving mode during the accident. If there is a negative number, switch to a positive number and record the time point T of the first switch. brake_positive , which means that at this moment the braking control is switched from the automatic driving to the safety driver stepping on the brake pedal. Also record the time point T when the number first switches from positive to negative. brake_negative, the lowest braking peak brake_min, the highest braking peak brake_max and the corresponding time point T brake_min 、T brake_max .

[0119] Search for a positive number in accelerator_status. If not, the vehicle is in automatic driving mode throughout the accident. If a negative number is switched to a positive number, record the time point T when it first becomes 0. accelerator_0 , which means that at this moment the automatic driving control brake is switched to the safety driver stepping on the accelerator pedal. Also record the lowest throttle peak value T accelerator_min and the throttle peak value T accelerator_max .

[0120] By comparing T steering＝0 、T brake_positive 、T accelerator_0 Determine the safety officer's takeover method:

[0121]

[0122] (4) Analyze vehicle accessories before collision:

[0123] According to the vehicle status data, search for the switching time point of the light status (light_status). light_status is represented in binary format. The first bit is the hazard warning flasher, the second bit is the left turn signal, the third bit is the right turn signal, the fourth bit is the front fog light, the fifth bit is the rear fog light, the sixth bit is the backup light, the seventh bit is the brake light, the eighth bit is the low beam, and the ninth bit is the high beam. Each bit 0 represents the off state and 1 represents the on state. Since the lights change frequently, light_status is sliced ​​by bit and saved into 9 fields respectively. The time set of each field switching {T light_status}, used to determine whether the autonomous vehicle used lights in compliance with regulations during the accident, such as whether the turn signal was turned on when changing lanes, and whether the brake lights were turned on when braking.

[0124] According to the vehicle status data, search for the switching time point of the horn status (horn_status) and wiper status (wiper_status). horn_status is 0 for no horn status, 1 for horn on. wiper_status is 0 for no wiper status, 1 for wiper on. Search for the time point T when horn_status and wiper_status switch from 0 to 1. horn_status and T wiper_status , determine whether to take preventive measures and emergency measures, promptly convey warning information to surrounding targets, and combine environmental data to determine the weather conditions when the accident occurred.

[0125] (5) Analyze vehicle failures before collision:

[0126] According to the vehicle status data, search for the switching time point of the fault code (faultcode). Faultcode 0 indicates no fault, 1 indicates vehicle fault, 2 indicates perception fault, 3 indicates control fault, 4 indicates planning fault, and 5 indicates other faults. Search for the moment T when the faultcode suddenly changes from 0 to other values faultcode If there is a mutation, the automatic driving vehicle fails at all times. If there is no mutation, there is no vehicle failure.

[0127] (6) Analyze traffic signals and road markings before the collision:

[0128] Traffic light recognition data is used to determine whether the autonomous vehicle violated traffic regulations, such as running a red light or speeding, by analyzing the phase light status and traffic sign type and value. Audio and video data are also used to determine if there are any issues with the perception system. Lane line recognition data is used to determine the distance between the wheels and lane lines, as well as the color of the lane lines, to determine when the autonomous vehicle changed lanes or overtook, and whether it was driving over the lane line during the accident.

[0129] (7) Analysis of other targets before collision:

[0130] Based on the horizontal and vertical distances, heading angles, and sizes of the target objects relative to the autonomous vehicle in the traffic participant data, as well as the vehicle motion data, the area overlap method is used to find the number of the target object that collided with the autonomous vehicle, and the IoU (intersection over union) of the bounding box between the target object and the autonomous vehicle is calculated.

[0131]

[0132] By iterating through the object IDs and searching for those with an IoU greater than 0, we can determine which object collided with the autonomous vehicle. Based on the type, we can determine whether the object was a four-wheeled vehicle, a two-wheeled vehicle, a pedestrian, or another obstacle. Because autonomous driving systems are complex, decisions aren't solely based on a single object. Even objects that haven't collided can still affect the autonomous vehicle before a collision. Therefore, we calculate risk indicators such as TTC and THW for objects that are close to the autonomous vehicle at the time of collision to determine whether the autonomous driving system's decision is reasonable.

[0133] (8) Analyze the vehicle status after the collision:

[0134] Search speed based on vehicle motion data Moment

[0135] According to the vehicle status data, search for the switching time points of the parking state (park_status) and the reversing state (back_status). If park_status is 0, it indicates the non-parking state, and 1 indicates the parking state (including electronic parking and parking with handbrake). If back_status is 0, it indicates the non-reversing state, and 1 indicates the reversing state.

[0136] Search for the moment T at which park_status changes from 0 to 1. park_status , whether back_status changes from 0 to 1 at the moment T back_status .

[0137] If there is no sudden change, it is determined that the vehicle is not parked or reversed after the collision.

[0138] If there is a mutation, compare the mutation time T park_status 、T back_status The moment the vehicle comes to a complete stop after the collision if Indicates that the vehicle is in parked state when it comes to a complete stop after a collision. The representative performed a reverse maneuver before the vehicle came to a complete stop after the collision.

[0139] (9) Record key time points

[0140] Recording the values ​​and corresponding time points of key field changes facilitates visualization in the module. Table 1 shows the key field records. By sorting key time points and creating a timeline, we can clearly visualize the entire accident process and the specific operations of each module. Recording the time points of key vehicle motion data changes and vehicle status changes allows for accurate vehicle operation analysis.

[0141] Table 1

[0142]

[0143] In this embodiment, the data visualization module is used to intuitively display the changing trends, key points and accident restoration animations of accident data, and realize the visualization of the accident data analysis process and analysis results. Specifically, it includes several window interfaces for data display, audio and video display, key indicator display, accident timeline display and accident restoration animation display.

[0144] (1) Data display window

[0145] It is used to visually display the time series change trends of various fields in vehicle status data, vehicle movement data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data and path planning data in the form of line graphs, scatter graphs, etc., and supports checking multiple fields for comparison on the same time axis. For fields with numerical switching in the data analysis module, such as brake and accelerator pedals, it supports highlighting the switching time points with markers of different colors and shapes to distinguish different driving modes. At the same time, the data statistics chart supports timeline dragging, zooming and benchmark positioning, which is convenient for viewing specific time data. Figure 2 .

[0146] (2) Audio and video display window

[0147] Used to display at least six channels of audio and video data: DV, DR, F, B, L, and R. It supports zooming in and out of the screen, allows dragging the progress bar, and performs reference point calibration and time synchronization with the statistical chart data in the data display window.

[0148] (3) Key indicator display window

[0149] This tool displays the calculated values ​​of vehicle safety indicators such as TTC and THW, as well as the values ​​and corresponding time points of changes in key accident data fields in Table 1. It also supports exporting calculation results. Indicators support custom calculation logic.

[0150] (4) Accident timeline display window

[0151] It is used to show the key time points in the accident process from before the collision to the end of the collision in a time sequence, which is convenient for sorting out the accident timeline. Figure 3 .

[0152] (5) Accident restoration animation display window

[0153] Used to reproduce the accident process, using animation simulation based on audio and video data. Figure 4 shown.

[0154] After the visualization display, an accident analysis report is automatically generated. It supports the input of manually recorded information, combines the analysis process of the data analysis module and the visualization module, and clarifies whether the driving mode was switched during the accident, whether there are technical defects such as control system failures, perception failures such as communication delays, whether the operation is in accordance with the established design of the autonomous driving system, whether excessive decision-making behavior is taken or relevant laws and regulations are violated, etc., from multiple dimensions such as vehicle motion status, driving mode, perception results, decision analysis, control measures, and safety officer takeover. Finally, an accident analysis report in a standardized PDF format is generated. The output report supports previewing and editing part of the content before generating and downloading. The report contains the following content:

[0155] (1) Manually input basic accident information:

[0156] You need to fill in the basic information of the accident according to the prompts and present it in the report in a table format, as shown in Table 2.

[0157] Table 2

[0158]

[0159]

[0160] (2) Analysis process of data parsing module and visualization module:

[0161] The above analysis process is automatically integrated into the report in the form of text and pictures, and divided into three stages according to the time series: pre-collision, collision and post-collision.

[0162] Before the collision, it includes the basic situation of the safety officer (such as takeover due to incorrect operation) and driving status, analysis of road conditions (road conditions, environment), calculation of the time interval between each key time point and the collision, analysis of the decision-making of the safety officer or autonomous driving vehicle (such as estimating the behavior of other traffic participants), and the control of the safety officer or autonomous driving vehicle (whether the safety officer took over, whether the emergency response was improper, etc.).

[0163] The collision analysis includes speed, acceleration, steering, throttle, braking, and other factors at the time of collision. It also determines the driving mode at the time of collision, including whether the vehicle was operated by the autonomous driving system, the safety driver, or remote control. The collision position is combined with traffic lights, traffic signs, lane markings, and the collision location of other traffic participants to determine whether either party violated traffic rules.

[0164] After the collision, the final position of the autonomous vehicle when it was stationary and that of traffic participants, vehicle fault information, etc. are included to determine whether the cause of the accident involves safety hazards of the autonomous driving system.

[0165] (3) Report Conclusions:

[0166] The report's conclusions are used to analyze the causes of the accident and potential vehicle safety hazards. From the perspective of the autonomous vehicle, the vehicle's driving mode is used to clarify the transition between manual and autonomous driving modes during the accident. Vehicle perception results are used to verify the presence of perception failures such as communication delays. Vehicle decision-making is used to assess whether there are irrational, conservative or aggressive algorithmic logic. Vehicle control measures are used to determine whether there were risky driving behaviors such as emergency braking and obstacle avoidance. From the safety officer's perspective, observation errors due to poor road conditions, unclear traffic signs, or the size of the intersection are determined. Misjudgments regarding the movement of other vehicles, road shape and alignment, the driver's vehicle's performance and speed estimation, the speed of other vehicles, and the distance between the driver and the other vehicle are analyzed. Operational errors due to unfamiliarity with the vehicle and road, panic inability to respond in an emergency, or mechanical failures within the vehicle, such as brake failure, are assessed. Finally, a summary evaluation is provided based on the entire analysis process.

[0167] Example 2

[0168] A traffic accident data analysis and visualization processing system for autonomous driving vehicles, including a data reading and integration module, a data cleaning and verification module, a time calibration and synchronization module, a data analysis module, and a data visualization module;

[0169] Data reading and integration module, used to collect raw data related to vehicle traffic accidents and standardize and integrate the raw data;

[0170] Specifically, the data reading and integration module is used to perform regular reading and integration of accident-related raw data of different types and formats collected from multiple sources.

[0171] Data cleaning and verification module, used to clean and verify the standardized integrated data;

[0172] Time calibration and synchronization module time, used to calibrate and align the timestamps of multi-source data, providing accurate time parameters for subsequent timing analysis;

[0173] The data analysis module is used to obtain the collision time point based on the speed and path of the accident vehicle. Based on the processed data and the collision time point, the module analyzes the driving mode at the time of collision, the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, the vehicle fault before the collision, the traffic signals and road markings before the collision, and the vehicle status after the collision to obtain the key time point;

[0174] Specifically, the data analysis module is used to comprehensively consider data collected from multiple sources, find a set of key time points by comparing and analyzing the changing characteristics of each field, identify factors that can affect the collision at a specific time, and assist in understanding the course of the accident.

[0175] The data visualization module is used to visualize the analysis results based on the processed data and key time points.

[0176] The system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode was switched during the accident, whether there was a control system failure or communication delay, whether the vehicle was driving in accordance with the automatic driving system, whether excessive decision-making behavior was taken or relevant regulations were violated based on the vehicle's motion status, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

[0177] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, characterized in that: include: Collecting original data related to vehicle traffic accidents and standardizing and integrating the original data; Clean, verify, time-calibrate, and synchronize the standardized and integrated data to obtain the processed data; Obtain the collision time point based on the speed and path of the accident vehicle; According to the speed and path of the accident vehicle, the collision time is obtained including: The three-axis acceleration values ​​of the accident vehicle are obtained. The three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​first change suddenly is obtained, which is recorded as T. a ; Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three axes and the time interval is greater than the first-order difference sum, which is recorded as T b ; Get the time point when the accident vehicle deviates from the expected path, recorded as T c ; Select T a , T b , T c The point closest to the collision time is taken as the collision time point; Analyzing the driving mode at the time of collision, the vehicle state before the collision, the traffic environment before the collision, and the vehicle state after the collision based on the processed data and the collision time point to obtain the key time point; The key time points include: maximum acceleration time point, maximum speed time point, speed 0 time point, driving mode switching time point, manual intervention type switching time point, steering state switching time point, steering wheel angle 0 time point for the first time, steering wheel angle lowest peak time point, steering wheel angle highest peak time point, automatic driving control braking turns into safety officer stepping on the brake pedal time point, safety officer stepping on the brake pedal turns into automatic driving control braking time point, braking lowest peak time point, braking highest peak time point, automatic driving control throttle turns into safety officer stepping on the accelerator pedal time point, throttle lowest peak time point, throttle highest peak time point, light switching time point, horn switching time point, wiper switching time point, vehicle failure time point, parking state switching time point, reverse state switching time point; Analysis of the vehicle status before the collision includes analysis of the vehicle steering, vehicle pedals, vehicle accessories, and vehicle faults before the collision; Analyzing the vehicle pedals before the collision includes: obtaining the braking state and the throttle state based on the vehicle state data, and determining the safety officer's takeover method based on the time point when the steering wheel angle first reaches 0, the time point when the braking state first switches from a negative number to a positive number, and the time point when the throttle state first reaches zero; The analysis results are visualized based on the processed data and the key time points.

2. The method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to claim 1, wherein: Collecting raw data related to vehicle traffic accidents and standardizing and integrating the raw data includes: The raw data is collected by an on-board terminal device and manual recording, wherein the on-board terminal device collects data from the CAN bus, positioning and monitoring video, and the manual recording collects environmental information, safety officer information, collision area and casualty data; The raw data is separated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data and audio and video data according to a standardized format.

3. The method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to claim 2, wherein: Cleaning, verification, time calibration and synchronization of standardized and integrated data include: The raw data is pre-processed to eliminate redundancy, smooth conflicts, and fill gaps, the pre-processed data is arranged in time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format; Obtaining different data sampling frequencies, presetting a time axis according to a minimum frequency and performing frequency filling, and performing data frequency alignment on the preprocessed data; The pre-processed data are aligned for data length according to a preset reference point of a time frame.

4. A traffic accident data analysis and visualization processing system for an autonomous driving vehicle, characterized in that: Data reading and integration module, data cleaning and verification module, time calibration and synchronization module, data analysis module, data visualization module; The data reading and integration module is used to collect raw data related to vehicle traffic accidents and perform standardized integration on the raw data; The data cleaning and verification module is used to clean and verify the normalized and integrated data; The time calibration and synchronization module is used to perform time calibration and synchronization processing on the data after cleaning and verification processing to obtain the processed data; The data analysis module is used to obtain the collision time point based on the speed and path of the accident vehicle, and analyze the driving mode at the time of collision, the vehicle state before collision, the traffic environment before collision, and the vehicle state after collision based on the pre-processed data and the collision time point to obtain the key time point; According to the speed and path of the accident vehicle, the collision time is obtained including: The three-axis acceleration values ​​of the accident vehicle are obtained. The three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​first change suddenly is obtained, which is recorded as T. a ; Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three axes and the time interval is greater than the first-order difference sum, which is recorded as T b ; Get the time point when the accident vehicle deviates from the expected path, recorded as T c ; Select T a , T b , T c The point closest to the collision time is taken as the collision time point; The key time points include: maximum acceleration time point, maximum speed time point, speed 0 time point, driving mode switching time point, manual intervention type switching time point, steering state switching time point, steering wheel angle 0 time point for the first time, steering wheel angle lowest peak time point, steering wheel angle highest peak time point, automatic driving control braking turns into safety officer stepping on the brake pedal time point, safety officer stepping on the brake pedal turns into automatic driving control braking time point, braking lowest peak time point, braking highest peak time point, automatic driving control throttle turns into safety officer stepping on the accelerator pedal time point, throttle lowest peak time point, throttle highest peak time point, light switching time point, horn switching time point, wiper switching time point, vehicle failure time point, parking state switching time point, reverse state switching time point; Analysis of the vehicle status before the collision includes analysis of the vehicle steering, vehicle pedals, vehicle accessories, and vehicle faults before the collision; Analyzing the vehicle pedals before the collision includes: obtaining the braking state and the throttle state based on the vehicle state data, and determining the safety officer's takeover method based on the time point when the steering wheel angle first reaches 0, the time point when the braking state first switches from a negative number to a positive number, and the time point when the throttle state first reaches zero; The data visualization module is used to visualize the analysis results based on the processed data and the key time points.

5. The traffic accident data analysis and visualization processing system for an autonomous driving vehicle according to claim 4, characterized in that: The system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode was switched during the accident, whether there was a control system failure or communication delay, whether the vehicle was driving according to the automatic driving system, whether excessive decision-making behavior was taken or relevant laws and regulations were violated based on the vehicle's motion status, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

Images