Traffic accident data analysis and visualization processing method and system for autonomous vehicle

Abstract

The invention relates to a traffic accident data analysis and visualization processing method and system for an automatic driving vehicle, and the method comprises the steps: collecting the related original data of a vehicle traffic accident, and carrying out the standardized integration of the original data; performing cleaning and verification processing, time calibration and synchronous processing on the data after the standardized integration to obtain processed data; acquiring a collision time point according to the speed and the path of the accident vehicle; according to the processed data and the collision time point, a collision moment driving mode, vehicle steering before collision, vehicle pedals before collision, vehicle accessories before collision, vehicle faults before collision, traffic signals before collision, road marking before collision and a vehicle state after collision are analyzed, and a key time point is obtained; and according to the processed data and the key time point, carrying out visual display on an analysis result. According to the invention, key time point marking can be realized, and a scientific and standardized accident analysis report can be automatically generated and stored.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic accident data processing for autonomous driving vehicles, and in particular to a method and system for analyzing and visualizing traffic accident data for autonomous driving vehicles. Background Art

[0002] With the gradual development of domestic autonomous driving technology, the number of autonomous driving vehicles on road tests continues to increase. From the actual road tests of autonomous driving vehicles, traffic violations and traffic accidents occur from time to time. Unlike traditional vehicles, autonomous driving vehicles have both "vehicle" and "driver" characteristics. The vehicle control subject is changed from a single human driver to an autonomous driving system joint driver. The traditional accident handling investigation and evidence collection methods are no longer applicable to autonomous driving vehicle accident investigations, which has brought new demands and challenges to road traffic management. At present, the relevant road test rules issued by autonomous driving test areas and regulatory units in various parts of the country require that autonomous driving test vehicles should have vehicle status recording, storage and online monitoring functions, and can transmit vehicle control mode, vehicle position, vehicle speed, acceleration and other motion states in real time, and automatically record and store environmental perception and response status, vehicle lights and signals real-time status, vehicle external 360-degree video monitoring, in-vehicle video and voice monitoring reflecting the tester and human-machine interaction status, remote control commands received by the vehicle, and vehicle failure conditions at least 90 seconds before the vehicle accident or failure occurs.

[0003] To investigate road traffic accidents involving autonomous vehicles and determine the illegal responsibility, it is necessary to conduct multi-source data fusion analysis of perception, decision-making, control, human-computer interaction, etc. of the autonomous driving system based on the actual operation data of the vehicle during the accident, and to determine whether there are safety hazards in the autonomous driving system of the accident vehicle, so as to determine the cause of the accident and the illegal responsibility. It is urgent to develop accident data collection, processing and analysis methods. However, the amount of data in the vehicle-side data recording device is huge, and the data structure cannot be directly viewed. At the same time, the time period of data recording is often very long, resulting in complex and diverse data, which has become a difficulty for the public security traffic management department to extract key clues and evidence in the process of handling accidents. The extraction and analysis method of these key clues and evidence is very cumbersome, consuming a lot of manpower and time, resulting in low law enforcement efficiency. Therefore, it is very important to develop a method that can quickly and effectively parse and visualize the data in the data recording device of the autonomous driving vehicle during the accident.

[0004] The existing technology "A multi-dimensional information viewing system for autonomous driving vehicles" proposes to collect the existing autonomous driving vehicle data scattered in different systems, use stream segmentation technology to achieve simultaneous reception and playback, and display it on the timeline in the form of data blocks to achieve synchronous playback of multi-dimensional data at the location of the timeline cursor. However, only some data types are listed, mainly viewing and playing data, lacking data analysis, visualization, etc., and the functions are single, which cannot meet the time synchronization requirements of data and video surveillance inside and outside the vehicle.

[0005] The prior art "Accident responsibility analysis method and device for autonomous driving vehicles" proposes an accident responsibility analysis method based on cloud data analysis and simulation technology. By analyzing and integrating the data uploaded to the cloud platform, the simulation software is used to restore the road conditions, vehicle driving trajectory, traffic participant information, etc. at the time of the accident to form an accident scene. According to the accident scene and the relevant data of the autonomous driving vehicle, the driving mode at the time of the accident is judged, the cause of the accident is analyzed, and the responsibility for the accident is determined. This patent only determines whether it is the driver's responsibility based on the driving mode. The method is too simple and cannot meet the needs of accident handling and responsibility determination in actual work. In addition, the data is uploaded to the cloud by setting trigger rules, which is easy to miss relevant accident data.

[0006] The prior art "A road traffic accident data mining server, method and system" proposes an accident data mining method and system, which includes a data preprocessing module, an analysis module, a mining module and a visualization module to solve the shortcomings of traditional vehicle traffic accident statistical analysis methods. However, this invention conducts macro mining analysis and point statistics on historical data, rather than micro analysis of a single autonomous driving vehicle accident, and uses macro statistical indicators such as mortality rate, without involving specific parameters required for analysis.

[0007] The existing technology "A traffic accident data integration and analysis method and system" proposes to integrate and analyze historical traffic accident data, set up traffic accident trend prediction models, traffic accident frequency models and traffic accident impact models, calculate accident data values, traffic accident frequencies and traffic accident impact values, and issue warnings when they exceed preset thresholds. This invention relies on historical traffic accident data for analysis and issues warnings when accidents may occur. The model used is simple and lacks actual traffic accident processing and analysis methods.

[0008] The prior art "A natural driving data collection device" proposes a natural driving data collection device, which includes scene data collection equipment, vehicle-mounted data collection equipment, vehicle-mounted intelligent screening equipment, etc. The device uses a variety of algorithms such as vehicle following algorithm, overtaking algorithm, lane changing algorithm, etc. to process and analyze the collected data. It is mainly used to collect scene data required for intelligent networked vehicle testing, rather than accident data. The data items used for accident analysis may be different, and there is a lack of correlation analysis between vehicle driving trajectory data and accident environment data.

[0009] In summary, the existing technical solutions mainly focus on the management and macro-mining analysis of historical accident data, rather than the data analysis of a single accident of an autonomous driving vehicle. Therefore, the present invention provides a method and system for analyzing and visualizing traffic accident data of an autonomous driving vehicle to solve the above technical problems. Summary of the invention

[0010] The purpose of the present invention is to provide a method and system for analyzing the traffic accident data of an autonomous driving vehicle and visualizing it, which is used to analyze the causes of autonomous driving vehicle accidents, realize accident data analysis and visualization, realize key time point marking, and can automatically generate and save scientific and standardized accident analysis reports, guide every link of autonomous driving vehicle accident handling, and make various data analysis indicators traceable.

[0011] To achieve the above object, the present invention provides the following solutions:

[0012] A method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, comprising:

[0013] Collecting original data related to vehicle traffic accidents and standardizing and integrating the original data;

[0014] Clean and verify the standardized and integrated data, perform time calibration and synchronization, and obtain the processed data;

[0015] Obtain the collision time point based on the speed and path of the accident vehicle;

[0016] Analyze the driving mode at the time of collision, the vehicle state before collision, the traffic environment before collision, and the vehicle state after collision according to the processed data and the collision time point to obtain the key time point;

[0017] The analysis results are visualized based on the processed data and the key time points.

[0018] Optionally, collecting raw data related to vehicle traffic accidents and standardizing and integrating the raw data includes:

[0019] The raw data is collected by the vehicle terminal device and manual recording, wherein the vehicle terminal device collects data from the CAN bus, positioning and monitoring video, and the manual recording collects environmental information, safety officer information, collision area and casualty degree data;

[0020] The raw data is integrated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data and audio and video data according to a standardized format.

[0021] Optionally, the standardized integrated data is cleaned and verified, time-calibrated, and synchronized, including:

[0022] The raw data is preprocessed to eliminate redundancy, smooth conflicts and fill gaps, the preprocessed data is arranged according to time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format;

[0023] Acquire different data adoption frequencies, preset a time axis according to a minimum frequency and perform frequency filling, and perform data frequency alignment on the preprocessed data;

[0024] The pre-processed data is aligned for data length according to a preset reference point of the time frame.

[0025] Optionally, according to the speed and path of the accident vehicle, obtaining the collision time point includes:

[0026] The three-axis acceleration values ​​of the accident vehicle are obtained, and the three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​change for the first time is obtained, which is recorded as T a ;

[0027] Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three-axis direction and the time interval is greater than the first-order difference sum, which is recorded as T b ;

[0028] Get the time point when the accident vehicle deviates from the expected path, denoted as T c ;

[0029] Select T a ,T b ,T c The point with the earliest time in the collision is taken as the collision time point.

[0030] Optionally, the key time points include: the maximum acceleration time point, the maximum speed time point, the speed is 0 time point, the driving mode switching time point, the manual intervention type switching time point, the steering state switching time point, the steering wheel angle is 0 for the first time, the steering wheel angle lowest peak time point, the steering wheel angle highest peak time point, the automatic driving control braking is transformed into the safety officer stepping on the brake pedal time point, the safety officer stepping on the brake pedal is transformed into the automatic driving control braking time point, the braking lowest peak time point, the braking highest peak time point, the automatic driving control throttle is transformed into the safety officer stepping on the accelerator pedal time point, the throttle lowest peak time point, the throttle highest peak time point, the light switching time point, the horn switching time point, the wiper switching time point, the vehicle failure time point, the parking state switching time point, and the reversing state switching time point.

[0031] Optionally, analyzing the vehicle state before the collision includes: analyzing the vehicle steering before the collision, the vehicle pedal before the collision, the vehicle accessories before the collision, and the vehicle fault before the collision;

[0032] Analyzing the vehicle pedals before the collision includes: obtaining the braking status and the throttle status based on the vehicle status data, and determining the safety officer's takeover method based on the time point when the steering wheel angle first reaches 0, the time point when the braking status first switches from a negative number to a positive number, and the time point when the throttle status first reaches zero.

[0033] On the other hand, it also provides a traffic accident data analysis and visualization processing system for an autonomous driving vehicle, a data reading and integration module, a data cleaning and verification module, a time calibration and synchronization module, a data analysis module, and a data visualization module;

[0034] The data reading and integration module is used to collect raw data related to vehicle traffic accidents and perform standardized integration on the raw data;

[0035] The data cleaning and verification module is used to clean and verify the standardized integrated data;

[0036] The time calibration and synchronization module is used to perform time calibration and synchronization processing on the data after cleaning and verification processing to obtain the processed data;

[0037] The data analysis module is used to obtain the collision time point according to the speed and path of the accident vehicle, and analyze the driving mode at the collision time, the vehicle state before the collision, the traffic environment before the collision and the vehicle state after the collision according to the processed data and the collision time point to obtain the key time point;

[0038] The data visualization module is used to visualize the analysis results based on the processed data and the key time points.

[0039] Optionally, the system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode is switched during the accident, whether there is a control system failure or communication delay, whether the vehicle is driving in accordance with the automatic driving system, whether excessive decision-making behavior is taken or relevant laws and regulations are violated according to the vehicle's motion state, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

[0040] The beneficial effects of the present invention are as follows: the present invention is used to analyze the causes of accidents of autonomous driving vehicles, realize accident data analysis and visualization, realize key time point marking, and can automatically generate and save scientific and standardized accident analysis reports, guide every link in the handling of autonomous driving vehicle accidents, and make various data analysis indicators traceable, basically meeting the needs of traffic police in handling traffic accidents of autonomous driving vehicles, and effectively helping traffic police to improve their interactive understanding level, scientifically and clearly interpret key accident information, enrich the display form of accident reports, and comprehensively and objectively investigate accidents, thereby providing support for traffic management departments to handle autonomous driving vehicle accidents efficiently and with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flow chart of a method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to an embodiment of the present invention;

[0043] Figure 2 A comparison diagram of the same time axis of the data display window of an embodiment of the present invention;

[0044] Figure 3 This is an example diagram of an accident timeline according to an embodiment of the present invention;

[0045] Figure 4 This is an example diagram of an accident restoration animation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] like Figure 1 As shown, this embodiment provides a method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, including:

[0049] Collect raw data related to vehicle traffic accidents and integrate the raw data in a standardized manner;

[0050] Clean and verify the standardized and integrated data, perform time calibration and synchronization, and obtain the processed data;

[0051] Obtain the collision time point based on the speed and path of the accident vehicle;

[0052] According to the processed data and the time of collision, the driving mode at the time of collision, the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, the vehicle fault before the collision, the traffic signal and road markings before the collision and the vehicle status after the collision are analyzed to obtain the key time points, which are the time points when the motion data of the accident vehicle changes and the time points when the vehicle status changes;

[0053] The analysis results are visualized based on the processed data and key time points.

[0054] Furthermore, the original data related to vehicle traffic accidents is collected and the original data is standardized and integrated, including:

[0055] The original data is collected through the vehicle terminal equipment and manual recording. The vehicle terminal equipment collects data from the CAN bus, positioning and monitoring video, and the manual recording collects environmental information, safety officer information, collision area and casualty degree data;

[0056] The raw data is integrated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data and audio and video data according to a standardized format.

[0057] Specifically, the original data includes: data from the CAN bus, positioning and monitoring video collected by the on-board terminal equipment, as well as environmental information, safety officer information, collision area and degree of casualties collected through manual recording.

[0058] CAN bus data refers to data such as vehicle speed, braking, throttle, steering, driving mode, fault code, etc. collected by connecting to the vehicle's CAN bus. Positioning data refers to the current location of the vehicle, which can usually be converted into longitude and latitude or coordinates, and is not limited to being obtained through positioning technologies such as the Global Navigation Satellite System (GNSS), Real-Time Kinematic Positioning (RTK), Inertial Navigation (INS), and Laser Radar (LiDAR) or combined positioning technologies. Monitoring video refers to the in-vehicle video of an autonomous driving vehicle collected by cameras and monitoring devices, including the steering wheel and instrument panel perspectives, driver posture, and surrounding environment video, including forward perspectives, rear perspectives, and side perspectives. Manually recorded data refers to information directly obtained by traffic police during accident site investigations, as well as information supplemented by safety officers and the companies involved.

[0059] The above raw data is in a standardized format, and the single accident data is integrated into 8 types of data and audio and video data, including vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane recognition data, and path planning data, and stored as 8+N data files, where N is the number of audio and video data files. Audio and video data can be stored in mp4, avi, or mov formats, and other data except audio and video can be stored in csv, xlsx, xls, txt, or dat formats. All data files are named according to fixed rules: naming prefix + 17-digit vehicle identification code (VIN) + 10-digit time (such as 2024102110 at 10:00 on October 21, 2024), separated by an underscore.

[0060] Vehicle status data, named with the prefix STATUS, includes the driving status, manual intervention type, braking status, steering status, throttle status, parking status, reversing status, lighting status, vehicle fault code, horn and wiper information from the CAN bus corresponding to each time point. Vehicle motion data, named with the prefix MOVE, includes the latitude and longitude of the autonomous vehicle at each time point, speed (three-axis speed in the vehicle coordinate system), acceleration (three-axis acceleration in the vehicle coordinate system), heading angle, pitch angle, roll angle and other information from the positioning system. Traffic participant data, named with the prefix OTHER, includes the type, number, horizontal and vertical distance relative to the autonomous vehicle, speed, heading angle, size (length, width and height) and other information from the CAN bus and positioning of the target object other than the autonomous vehicle at each time point. Environmental data, named with the prefix ENV, includes brightness (lux), visibility distance, rain, snow, fog, temperature and other information from the CAN bus and manual records. Traffic light recognition data, named with the prefix LIGHT, includes the type, number, phase light status (red, yellow, and green for left turn, straight and right turn) of the traffic light at each time point, as well as the distance between the traffic light and the vehicle, and other information derived from the CAN bus and positioning. Traffic sign recognition data, named with the prefix SIGN, includes the number, type, value of the traffic sign, as well as the distance between the traffic sign and the vehicle, and other information derived from the CAN bus and positioning. Lane line recognition data, named with the prefix LANE, includes the number of the lane where the autonomous driving vehicle is located at each time point, lane type, lane curvature, lane width, the distance between the autonomous driving vehicle and the left lane line, and the type and color of the lane lines on the left and right sides, and other information derived from the monitoring video conversion. Path planning data, named with the prefix PLAN, includes the expected longitude and latitude position, expected heading, expected speed, and the time difference between the expected planning position and the start planning position, and other information derived from the CAN bus and positioning. Audio and video data, named with prefixes based on video content: W (panoramic video), DV (perception interface video), DR (in-vehicle safety officer and steering wheel video), BK (pedal video), F (front video), LF (left front video), RF (right front video), B (rear video), LB (left rear video), RB (right rear video), L (left video), R (right video). The number of audio and video data files N depends on the number of cameras installed on the autonomous vehicle, and should include at least six channels of video: DV, DR, F, B, L, and R.

[0061] Furthermore, the standardized and integrated data is cleaned, verified, time-calibrated and synchronized, including:

[0062] The raw data is preprocessed to eliminate redundancy, smooth conflicts and fill gaps, the preprocessed data is arranged in time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format;

[0063] Obtain the frequency of different data, preset the time axis according to the minimum frequency and perform frequency filling, and align the data frequency of the preprocessed data;

[0064] According to the preset reference point of the time frame, the preprocessed data is aligned for data length.

[0065] Specifically, because it is necessary to consider that autonomous driving vehicles on actual roads may pass through abnormal road surfaces such as potholes and speed bumps, and take driving behaviors such as sudden deceleration, sudden acceleration, and sharp turns, data denoising methods are needed to reduce the noise impact of the above situations on data analysis. In addition, considering that hardware damage, software defects, abnormal vehicle status, etc. may occur during the accident process, resulting in missing or incorrect data, which needs to be processed.

[0066] Eliminating redundant data refers to processing irregular data, specifically automatically identifying duplicate data records, allowing for custom rules for identifying redundant data (such as when the values ​​of certain key fields are exactly the same at different time points), and choosing to delete or merge them.

[0067] Smoothing conflicting data refers to processing inconsistent data. Specifically, it automatically identifies outliers by calculating the standard deviation, allowing you to choose whether to keep or process outliers, and using methods such as moving smoothing or exponential smoothing. Moving smoothing is to take a fixed-size window at the conflict point, calculate the average of all data points in the window and replace the original data. Exponential smoothing is to use a smoothing factor α to weight the current conflict value. and the previous smoothed value is the smoothed value at time t2.

[0068]

[0069] Filling missing data refers to filling missing values, specifically detecting null values ​​in the data. If the corresponding field data exists, you can choose methods such as neighbor method or interpolation regression to fill the missing values. and The missing value of time t2 in the interval [t1, t3] Neighbor Method Use Previous location or the latter To fill. Linear interpolation regression uses first-order spline curve interpolation filling.

[0070]

[0071] The quadratic interpolation uses the second-order spline curve interpolation to construct the quadratic interpolation function f(t), so that For i=0,1,3, the Lagrangian interpolation basis functions L0(t), L1(t), L3(t) are defined as:

[0072]

[0073] When reading a data file, the field name, number of redundant data, number of outlier data, number of missing data, and number of valid data of each field in the file need to be output one by one in the visual interface.

[0074] When reading a data file, the field name, number of redundant data, number of outlier data, number of missing data, and number of valid data of each field in the file need to be output one by one in the visual interface.

[0075] Calibrate and align the timestamps of multi-source data to provide accurate time parameters for subsequent timing analysis. This includes three steps: determining the timestamp format, aligning the data frequency, and aligning the data length.

[0076] The first step is to determine the timestamp format. For each data file, arrange each frame of data in time sequence, and form a complete video frame data sequence based on the timestamp of the audio and video data. UNIX (The number of milliseconds from 0:00 on January 1, 1970) converted to Beijing time.

[0077]

[0078] The second step is to align the data frequency. Since different data files are collected from different devices, there may be different collection frequencies, so time frame alignment is required to ensure that the data corresponding to different fields at that moment can be found in a certain key time frame. Detect the sampling frequency of different data files, set a unified time axis according to the minimum frequency, and fill the corresponding frequency. The filling method should be configured in advance according to different fields. For non-continuously changing fields such as driving status, manual intervention type, lighting status, target number, etc., where different values ​​represent different meanings, the previous neighbor method should be used for filling. For continuously changing fields such as speed, acceleration, heading angle, target distance, etc., the interpolation method can be used to fill. At the same time, the filled data and non-filled data should be marked, and different colors should be used to distinguish and display them in the data visualization module.

[0079] The third step is to align the data length. Since the time of the accident is unknown before data analysis, the data collected before and after the collision submitted by the company may have inconsistent data collection lengths, especially the lengths of data files and audio and video files are basically inconsistent. Therefore, it is necessary to set a certain time frame as the reference point and cut the data and audio and video files 20 seconds before and after the reference point.

[0080] The default reference point is the collision time, and you can also customize the input time as the reference point (input format is ). You can also manually select the reference point, that is, drag the audio and video progress bar to a certain position in the audio and video display window of the visual interface, select a time point in the data display window, and after confirmation, the system will use this time point as the reference point and automatically synchronize the data and audio and video before and after the reference point.

[0081] For audio and video files with time frame annotations, you can directly match the time frame of the data file, find the reference point, and cut the video length to the same length as the data file. For audio and video files without time frame annotations, use the above method of manually selecting the reference point.

[0082] Furthermore, according to the speed and path of the accident vehicle, obtaining the collision time point includes:

[0083] The three-axis acceleration values ​​of the accident vehicle are obtained, and the three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​change for the first time is obtained, which is recorded as T a ;

[0084] Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration values ​​of the three axes in different directions and the time interval is greater than the first-order difference sum, which is recorded as T b ;

[0085] Get the time point when the accident vehicle deviates from the expected path, denoted as T c ;

[0086] Select T a ,T b ,T c The point with the earliest time in the collision is taken as the collision time point.

[0087] Specifically, define A t =(accx t ,accy t ,accz t) represents the acceleration values ​​of the three axes from the sensor at time t. Usually, a collision will cause a sharp change in the acceleration of the autonomous vehicle. The acceleration of the three axes will have significant peaks and oscillations. The threshold a of the acceleration change is set to search for the acceleration mutation point. The threshold allows custom settings. The sliding window mechanism detects A at each moment. t ,||A t ||≥a, capture A t Time T of first mutation appearance a , T a It must exist.

[0088] If the change Δv of the actual speed of the autonomous vehicle per unit time Δt is equal to the acceleration sensor value A t The difference is large and can also be regarded as a collision. The time series velocity data v(t1), v(t2), ... v(t3), its first-order difference is Δv i , the sum of first-order differences is

[0089] Δv i =v(t i+1 )-v(t i )

[0090]

[0091] At discrete time points, the acceleration calculated from the velocity change rate should be approximated as the velocity difference divided by the time interval Δt,

[0092]

[0093] In normal motion, the acceleration of an autonomous vehicle should be approximately equal to the rate of change of the actual velocity, that is, the sum of the first-order differences of the velocities.

[0094]

[0095] If the actual acceleration sensor value multiplied by the time interval is greater than the sum of the first-order differences of the velocity,

[0096]

[0097] Capture the actual acceleration sensor value multiplied by the time interval greater than the first-order difference of the velocity and the corresponding time T b If it does not exist, then T b =NULL.

[0098] In addition, the normal motion trajectory should be relatively smooth, but collisions can cause the trajectory to suddenly turn or jump. The autonomous vehicle may deviate from the expected path (except for hardware failures or other environmental changes). If the deviation between its position and the planned path is too large, it means that there is a probability of collision. Define the deviation loss between the expected path and the actual driving path. The expected path coordinate is (x i ,y i ), the actual driving path coordinates are

[0099]

[0100] If there is a loss that does not converge at a certain moment,

[0101]

[0102] It is considered that the autonomous driving vehicle deviates from the expected path at this moment, and the time point T is captured c If it does not exist, then T c =NULL.

[0103] Collision time T collision The default value is T a ,T b ,T c The earliest point in time,

[0104] T collision =min(T a ,T b ,T c )

[0105] Furthermore, the key time points include: the maximum acceleration time point, the maximum speed time point, the speed is 0 time point, the driving mode switching time point, the manual intervention type switching time point, the steering state switching time point, the steering wheel angle is 0 for the first time, the steering wheel angle lowest peak time point, the steering wheel angle highest peak time point, the automatic driving control braking is transferred to the safety officer stepping on the brake pedal time point, the safety officer stepping on the brake pedal is transferred to the automatic driving control braking time point, the braking lowest peak time point, the braking highest peak time point, the automatic driving control throttle is transferred to the safety officer stepping on the accelerator pedal time point, the throttle lowest peak time point, the throttle highest peak time point, the light switching time point, the horn switching time point, the wiper switching time point, the vehicle failure time point, the parking state switching time point, and the reversing state switching time point.

[0106] The driving mode at the time of collision, the vehicle status before collision, the traffic environment before collision and the vehicle status after collision are analyzed. The analysis of the vehicle status before collision includes: the analysis of the vehicle steering before collision, the vehicle pedals before collision, the vehicle accessories before collision, and the vehicle fault before collision. The analysis of the traffic environment before collision includes: the analysis of the traffic signals and road markings before collision and the analysis of other targets before collision. Specifically,

[0107] (1) Analysis of driving patterns at the time of collision:

[0108] According to the vehicle status data, search for the switching time points of the driving state (drive_mode) and the manual intervention type (takeover_mode). The drive_mode value of 0 indicates manual driving, 1 indicates automatic driving, 2 indicates remote driving, 4 indicates automatic driving disengagement, and 9 indicates others. The takeover_mode value of 0 indicates no takeover, 1 indicates takeover from the main driver's position, 2 indicates takeover from other positions, and 3 indicates remote takeover.

[0109] Search whether drive_mode changes from 1 to other values ​​at a certain moment T drive_mode , whether takeover_mode changes from 0 to other values ​​at a certain moment T takeover_mode .

[0110] If there is no mutation, search drive_mode(T collision ) and takeover_mode(T collision ) corresponds to the driving status and manual intervention type, determining whether the vehicle is in automatic driving, automatic driving disengaged, or manually taken over (main driver's position, other position, or remote takeover) when a collision occurs.

[0111] If there is a mutation, compare the mutation time T drive_mode , T takeover_mode and collision time T collision ,

[0112] The collision was in automatic driving mode, which evaluates the system's response timeliness and driving strategy, and the car did not brake in time. If the collision was in manual driving mode, it means that the safety officer did not brake in time.

[0113] (2) Analysis of vehicle steering before collision:

[0114] According to the vehicle status data, search for the switching time point of the steering status (steering_status). Steering_status in [-1000, 1000] indicates manual steering of the steering wheel. The number represents the steering angle, with left turn as a negative number and right turn as a positive number. (1000, 3000] degrees represents the steering angle of the automatic driving system. After subtracting 2000 degrees, it is consistent with the steering wheel steering angle of [-1000, 1000].

[0115] Search whether steering_status has a value in (1000, 3000]. If all of them are, the vehicle is always in the automatic driving state during the accident. If a value is in (1000, 3000] and there is a subsequent mutation to [-1000, 1000], record the mutation time point T steering status At this moment, the steering wheel is switched from the automatic driving system to manual control, and the above judgment of the driving mode is verified. Map all values ​​belonging to (1000, 3000] to [-1000, 1000], and search for the first time point T where steering_status = 0 steering＝0 , as well as the lowest peak steering_min, the highest peak steering_max and the corresponding time point T steering min , T steering max , which is used to determine the stability of vehicle control during an accident, whether the autonomous driving system made a sharp turn to avoid a collision, and whether the safety officer took any collision avoidance action.

[0116] (3) Analysis of the vehicle pedals before the collision:

[0117] According to the vehicle status data, search for the switching time points of the brake status (brake_status) and the accelerator status (accelerator_status). Brake_status of 0 indicates the non-braking state, [1, 100] indicates the pedal-operated braking state, and [-100, -1] indicates the automatic driving system braking. The larger the absolute value, the greater the braking force. Accelerator_status of 0 indicates the non-acceleration state, [1, 100] indicates the accelerator pedal-operated force, and [-100, -1] indicates the automatic acceleration force of the automatic driving system. The larger the absolute value, the greater the acceleration force.

[0118] Search brake_status to see if there is a positive number. If not, the vehicle is in automatic driving mode during the accident. If a negative number is switched to a positive number, record the time point T of the first switch. brake_positive , which means that at this moment, the braking control of the automatic driving is switched to the safety officer stepping on the brake pedal. Also record the time point T when the number first switches from positive to negative. brake_negative, the minimum braking peak brake_min, the maximum braking peak brake_max and the corresponding time point T brake_min , T brake_max .

[0119] Search for a positive number in accelerator_status. If not, the vehicle is in automatic driving mode during the accident. If a negative number is switched to a positive number, record the time point T when it first becomes 0. accelerator_0 , which means that at this moment, the driver is no longer in control of the brakes, but is instead in control of the accelerator pedal. Also record the lowest throttle peak value T accelerator_min and the throttle peak value T accelerator_max .

[0120] By comparing T steering＝0 , T brake_positive , T accelerator_0 Determine the way the safety officer takes over:

[0121]

[0122] (4) Analysis of vehicle accessories before collision:

[0123] According to the vehicle status data, search for the switching time point of the light status (light_status). Light_status is represented in binary, with the first bit being the hazard warning flasher, the second bit being the left turn signal, the third bit being the right turn signal, the fourth bit being the front fog light, the fifth bit being the rear fog light, the sixth bit being the reverse light, the seventh bit being the brake light, the eighth bit being the low beam, and the ninth bit being the high beam. Each bit 0 represents the off state, and 1 represents the on state. Since the lights change frequently, light_status is sliced ​​by bit and saved into 9 fields respectively, and the time set for each field switching is searched {T light_status}, used to determine whether the autonomous driving vehicle used lights in compliance with regulations during the accident, such as whether the turn signal was turned on when changing lanes, whether the brake lights were turned on when braking, etc.

[0124] According to the vehicle status data, search for the switching time point of the horn status (horn_status) and the wiper status (wiper_status). A horn_status of 0 indicates no horn status, and a 1 indicates the horn is on. A wiper_status of 0 indicates no wiper status, and a 1 indicates the wiper is on. Search for the time point T when horn_status and wiper_status switch from 0 to 1 horn_status and T wiper_status , determine whether to take preventive measures and emergency measures, promptly convey warning information to surrounding targets, and combine environmental data to determine the weather conditions when the accident occurred.

[0125] (5) Analysis of vehicle failures before collision:

[0126] According to the vehicle status data, search for the switching time point of the fault code (faultcode). Faultcode 0 indicates no fault, 1 indicates vehicle fault, 2 indicates perception fault, 3 indicates control fault, 4 indicates planning fault, and 5 indicates other faults. Search for the moment T when the faultcode suddenly changes from 0 to other values faultcode If there is a mutation, the automatic driving vehicle fails at all times. If there is no mutation, there is no vehicle failure.

[0127] (6) Analysis of traffic signals and road markings before the collision:

[0128] Based on the phase light status of traffic light recognition data and the type and value of traffic signs, it is determined whether the autonomous vehicle violated traffic regulations such as running a red light or speeding during the accident, and combined with audio and video data to determine whether there is a problem with the perception system. Based on the distance between the wheel and the lane line and the color of the lane line data, it is determined when the autonomous vehicle changes lanes or overtakes, and whether it drove on the lane line during the accident.

[0129] (7) Analysis of other targets before collision:

[0130] According to the horizontal and vertical distances, heading angles and sizes of the target objects relative to the autonomous vehicle in the traffic participant data, as well as the vehicle motion data, the area overlap method is used to find the number of the target object that collides with the autonomous vehicle, and the IoU of the bounding box of the target object and the autonomous vehicle is calculated.

[0131]

[0132] Traverse the target object numbers and search for the number with IoU>0 to determine the target object that collides with the autonomous vehicle. Then, according to the type, the target object can be a four-wheeled vehicle, a two-wheeled vehicle, a pedestrian, or other obstacles. Since the autonomous driving system is relatively complex, the decision does not depend solely on a certain target object. Even targets that have not collided may have an impact on the autonomous driving vehicle before the collision. Therefore, the TTC, THW and other risk indicators of the target object that is close to the autonomous driving vehicle at the time of collision are calculated to determine whether the decision of the autonomous driving system is reasonable.

[0133] (8) Analysis of vehicle status after collision:

[0134] Based on the vehicle motion data, search speed Moment

[0135] According to the vehicle status data, search for the switching time points of the parking state (park_status) and the reversing state (back_status). When park_status is 0, it indicates the non-parking state, and 1 indicates the parking state (including electronic parking and handbrake parking). When back_status is 0, it indicates the non-reversing state, and 1 indicates the reversing state.

[0136] Search for the moment T at which park_status changes from 0 to 1 park_status , whether back_status changes from 0 to 1 at a moment T back_status .

[0137] If there is no mutation, it is determined that the vehicle is not parked or reversed after the collision.

[0138] If there is a mutation, compare the mutation time T park_status , T back_status The moment when the vehicle comes to a complete stop after the collision if Indicates that the vehicle is parked when it comes to a complete stop after a collision. The representative performed a reverse maneuver before the vehicle came to a complete stop after the collision.

[0139] (9) Record key time points

[0140] The values ​​and corresponding time points of key field changes are recorded to facilitate visualization module display. The key field records are shown in Table 1. By sorting the key time points and sorting out the timeline, the whole process of the accident and the specific operations of each module can be clearly and intuitively seen. The key vehicle motion data change time points and vehicle status change time points can be recorded to judge vehicle operations.

[0141] Table 1

[0142]

[0143] In this embodiment, the data visualization module is used to intuitively display the changing trends, key points and accident restoration animations of accident data, and realize the visualization of the accident data analysis process and analysis results. Specifically, it includes several window interfaces for data display, audio and video display, key indicator display, accident timeline display and accident restoration animation display.

[0144] (1) Data display window

[0145] It is used to visually display the time series change trends of various fields in vehicle status data, vehicle movement data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, and path planning data in the form of line graphs, scatter graphs, etc., and supports selecting multiple fields for comparison on the same time axis. For fields with numerical switching in the data analysis module, such as brake and accelerator pedals, it supports highlighting the switching time points with markers of different colors and shapes to distinguish different driving modes. At the same time, the data statistics chart supports timeline dragging, zooming, and benchmark positioning to facilitate viewing of data at specific times. Figure 2 .

[0146] (2) Audio and video display window

[0147] It is used to display at least six channels of audio and video data: DV, DR, F, B, L, and R. It supports zooming in and out of the screen, allows dragging of the progress bar, and performs reference point calibration and time synchronization with the statistical chart data in the data display window.

[0148] (3) Key indicator display window

[0149] It is used to display the calculated values ​​of indicators such as TTC and THW that characterize vehicle operation safety, as well as the values ​​and corresponding time points of changes in the key fields of accident data in Table 1, and supports the export of calculation results. Indicators support custom calculation logic.

[0150] (4) Accident timeline display window

[0151] It is used to show the key time points in the accident process from before the collision to the end of the collision in a time sequence, which is convenient for sorting out the accident timeline. Figure 3 .

[0152] (5) Accident restoration animation display window

[0153] Used to reproduce the accident process, using animation simulation based on audio and video data. Figure 4 shown.

[0154] After the visualization, an accident analysis report is automatically generated. It supports the input of manually recorded information, and combines the analysis process of the data analysis module and the visualization module to clarify whether the driving mode was switched during the accident, whether there are technical defects such as control system failures, perception failures such as communication delays, whether the automatic driving system is running according to the established design, whether excessive decision-making behavior is taken or relevant laws and regulations are violated, etc., from multiple dimensions such as vehicle motion status, driving mode, perception results, decision analysis, control measures, and safety officer takeover. Finally, a standardized PDF format accident analysis report is generated. The output report supports previewing and editing part of the content before generating a download. The report contains the following:

[0155] (1) Manually input basic accident information:

[0156] You need to fill in the basic information of the accident according to the prompts and present it in the report in a table format, as shown in Table 2.

[0157] Table 2

[0158]

[0159]

[0160] (2) Analysis process of data parsing module and visualization module:

[0161] The above analysis process is automatically integrated into the report in the form of text and pictures, and divided into three stages: pre-collision, collision and post-collision according to the time series.

[0162] Before the collision, it includes the basic situation of the safety officer (such as takeover due to misoperation) and driving status, analysis of road conditions (road conditions, environment), calculation of the time interval between each key time point and the collision, analysis of the decision-making of the safety officer or the autonomous driving vehicle (such as predicting the behavior of other traffic participants), and the control of the safety officer or the autonomous driving vehicle (whether the safety officer takes over, whether the emergency response is improper, etc.).

[0163] The collision includes speed, acceleration, steering, throttle, braking, etc. at the time of collision, and determines the driving mode at the time of collision, whether the vehicle is operated by the autonomous driving system, safety officer, or remote control. Combined with traffic lights, traffic signs, lane lines and the collision position of other traffic participants, it is determined whether one party violated the traffic rules.

[0164] After the collision, the final position of the autonomous driving vehicle when it was stationary and that of traffic participants, vehicle fault information, etc. are used to determine whether the cause of the accident involves safety hazards of the autonomous driving system.

[0165] (3) Report Conclusions:

[0166] The conclusion of the report is used to analyze the cause of the accident and the potential safety hazards of the vehicle. From the perspective of the autonomous vehicle, the switching process between manual driving and autonomous driving modes during the accident is clarified based on the vehicle driving mode. The perception results of the vehicle are used to verify whether there are perception failures such as communication delays. The vehicle decision-making is used to evaluate whether there are conservative or radical unreasonable algorithm logics. The vehicle control measures are used to determine whether there are risky driving behaviors such as emergency braking and obstacle avoidance operations. From the perspective of the safety officer, it is determined whether there are observation errors caused by poor road conditions, unclear traffic signs, and too large intersection areas. It is analyzed whether there are errors in the judgment of the movement of other vehicles, the shape and line type of the road, the performance and speed estimation of the vehicle, the speed of other vehicles, and the distance between the vehicle and the other vehicle. It is evaluated whether there are operational errors caused by mechanical failures of the vehicle itself, such as unfamiliarity with the vehicle and the road, panic in an emergency, or brake failure. Finally, a summary evaluation is given based on the entire analysis process.

[0167] Embodiment 2

[0168] A traffic accident data analysis and visualization processing system for an autonomous driving vehicle, comprising a data reading and integration module, a data cleaning and verification module, a time calibration and synchronization module, a data analysis module, and a data visualization module;

[0169] Data reading and integration module, used to collect raw data related to vehicle traffic accidents and standardize and integrate the raw data;

[0170] Specifically, the data reading and integration module is used to perform regular reading and integration of accident-related raw data of different types and formats collected from multiple sources.

[0171] Data cleaning and verification module, used to clean and verify the data after normalization and integration;

[0172] Time calibration and synchronization module time, used to calibrate and align the timestamps of multi-source data, and provide accurate time parameters for subsequent timing analysis;

[0173] The data analysis module is used to obtain the collision time point according to the speed and path of the accident vehicle, and analyze the driving mode at the time of collision, the vehicle steering before the collision, the vehicle pedal before the collision, the vehicle accessories before the collision, the vehicle fault before the collision, the traffic signal and road marking before the collision, and the vehicle status after the collision according to the processed data and the collision time point to obtain the key time point;

[0174] Specifically, the data analysis module is used to comprehensively consider data collected from multiple sources, find a set of key time points by comparing and analyzing the change characteristics of each field, identify factors that can affect the collision at a specific time, and assist in understanding the course of the accident.

[0175] The data visualization module is used to visualize the analysis results based on the processed data and key time points.

[0176] The system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode was switched during the accident, whether there was a control system failure or communication delay, whether the vehicle was driving in accordance with the automatic driving system, whether excessive decision-making behavior was taken or whether relevant regulations were violated, based on the vehicle's motion status, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

[0177] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for analyzing and visualizing traffic accident data of an autonomous driving vehicle, characterized in that: include: Collecting original data related to vehicle traffic accidents and standardizing and integrating the original data; Clean and verify the standardized and integrated data, perform time calibration and synchronization, and obtain the processed data; Obtain the collision time point based on the speed and path of the accident vehicle; Analyzing the driving mode at the time of collision, the vehicle state before collision, the traffic environment before collision, and the vehicle state after collision according to the processed data and the collision time point to obtain the key time point; The analysis results are visualized based on the processed data and the key time points.

2. The method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to claim 1, characterized in that: Collecting raw data related to vehicle traffic accidents and standardizing and integrating the raw data include: The raw data is collected by the vehicle terminal device and manual recording, wherein the vehicle terminal device collects data from the CAN bus, positioning and monitoring video, and the manual recording collects environmental information, safety officer information, collision area and casualty degree data; The raw data is integrated into vehicle status data, vehicle motion data, traffic participant data, environmental data, traffic light recognition data, traffic sign recognition data, lane line recognition data, path planning data and audio and video data according to a standardized format.

3. The traffic accident data analysis and visualization processing method of the autonomous driving vehicle according to claim 2, characterized in that: Cleaning, verification, time calibration and synchronization of standardized and integrated data include: The raw data is preprocessed to eliminate redundancy, smooth conflicts and fill gaps, the preprocessed data is arranged according to time sequence, the audio and video data are formed into a complete video frame data sequence according to the timestamp, and the timestamp is converted into a preset time format; Acquire different data adoption frequencies, preset a time axis according to a minimum frequency and perform frequency filling, and perform data frequency alignment on the preprocessed data; The pre-processed data is aligned for data length according to a preset reference point of the time frame.

4. The method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to claim 1, characterized in that: According to the speed and path of the accident vehicle, the collision time point is obtained including: The three-axis acceleration values ​​of the accident vehicle are obtained, and the three-axis acceleration values ​​at each moment are detected through the sliding window mechanism and the threshold of acceleration change. The time point when the three-axis acceleration values ​​change for the first time is obtained, which is recorded as T a ; Obtain the time series speed data of the accident vehicle, perform first-order difference summation on the time series speed data, and obtain the time point when the product of the acceleration value of the three-axis direction and the time interval is greater than the first-order difference sum, which is recorded as T b ; Get the time point when the accident vehicle deviates from the expected path, denoted as T c ; Select T a ,T b ,T c The point with the earliest time in the collision is taken as the collision time point.

5. The method for analyzing and visualizing traffic accident data of an autonomous driving vehicle according to claim 1, characterized in that: The key time points include: the maximum acceleration time point, the maximum speed time point, the speed is 0 time point, the driving mode switching time point, the manual intervention type switching time point, the steering state switching time point, the steering wheel angle is 0 for the first time, the steering wheel angle lowest peak time point, the steering wheel angle highest peak time point, the automatic driving control braking is converted to the safety officer stepping on the brake pedal time point, the safety officer stepping on the brake pedal is converted to automatic driving control braking time point, the braking lowest peak time point, the braking highest peak time point, the automatic driving control throttle is converted to the safety officer stepping on the accelerator pedal time point, the throttle lowest peak time point, the throttle highest peak time point, the light switching time point, the horn switching time point, the wiper switching time point, the vehicle failure time point, the parking state switching time point, and the reversing state switching time point.

6. The traffic accident data analysis and visualization processing method of the autonomous driving vehicle according to claim 1, characterized in that: The analysis of the vehicle status before the collision includes: analysis of the vehicle steering before the collision, the vehicle pedals before the collision, the vehicle accessories before the collision, and the vehicle faults before the collision; Among them, analyzing the vehicle pedals before the collision includes: obtaining the braking status and throttle status according to the vehicle status data, and determining the safety officer's takeover method according to the time point when the steering wheel angle first reaches 0, the time point when the braking status first switches from a negative number to a positive number, and the time point when the throttle status first reaches zero.

7. A traffic accident data analysis and visualization processing system for an autonomous driving vehicle, characterized in that: Data reading and integration module, data cleaning and verification module, time calibration and synchronization module, data analysis module, data visualization module; The data reading and integration module is used to collect raw data related to vehicle traffic accidents and perform standardized integration on the raw data; The data cleaning and verification module is used to clean and verify the standardized integrated data; The time calibration and synchronization module is used to perform time calibration and synchronization processing on the data after cleaning and verification processing to obtain the processed data; The data analysis module is used to obtain the collision time point according to the speed and path of the accident vehicle, and analyze the driving mode at the collision time, the vehicle state before the collision, the traffic environment before the collision and the vehicle state after the collision according to the pre-processed data and the collision time point to obtain the key time point; The data visualization module is used to visualize the analysis results based on the processed data and the key time points.

8. The traffic accident data analysis and visualization processing system for an autonomous driving vehicle according to claim 7, characterized in that: The system also includes a report generation module, which is used to combine the data analysis module and the data visualization module to determine whether the driving mode is switched during the accident, whether there is a control system failure or communication delay, whether the vehicle is driving in accordance with the automatic driving system, whether excessive decision-making behavior is taken or relevant regulations are violated according to the vehicle's motion state, driving mode, perception results, decision analysis, control measures and safety officer takeover, and finally generate a standardized accident analysis report.

Images