Automatic driving accident handling method and related device

Abstract

This application discloses an autonomous driving accident handling method and related equipment. One autonomous driving accident handling method includes: acquiring raw data uploaded from the vehicle and / or cloud, and processing the raw data, wherein the raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packets, ROSbag, and CAN logs; visualizing the processed data to obtain a visualization interface; storing the editing operations performed by analysts in the visualization interface, wherein the editing operations include at least one of the following: marking the accident occurrence time, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the 3D model state; and packaging and sharing the visualization interface and / or the editing operations in response to the analyst's sharing instruction.

Description

Technical Field

[0001] This application relates to the field of combined driver assistance technology, and in particular to an autonomous driving accident handling method and related equipment. Background Technology

[0002] In related technologies, vehicles operating under combined assisted driving and conditional autonomous driving systems perceive their surroundings through various sensors (such as cameras, LiDAR, and millimeter-wave radar), and a decision-making system performs path planning and behavioral decisions. In actual operation, sensor malfunctions, errors in environmental perception data, and anomalies in the decision-making system may occur. To ensure the optimization of subsequent autonomous vehicle decisions, timely and accurate analysis and accountability for these issues are necessary. This data comes in diverse formats and is scattered across different systems. Existing analytical tools have limited functionality and struggle to meet the needs of multi-departmental collaboration, leading to complex data retrieval, low analysis efficiency, and impacting the safety and reliability of the autonomous driving system.

[0003] According to the "Technical Specification for Accident Delineation and Data Collaboration of Combined Driving Assistance and Conditional Automated Driving" (T-CAAMTB287—2025), an accident handling system for autonomous vehicles has become a necessity. End-to-end data integration capabilities are crucial for ensuring the safe operation of autonomous vehicles. Specific requirements include: • Vehicle-side data storage: 48 hours of high-frequency data local storage (≥10Hz).

[0004] • Cloud data transmission: Event-triggered transmission (data ≥ 10 seconds before / after collision).

[0005] • Responsibility determination: Data analysis is used to determine and match 12 types of activation scenarios and 7 types of exception scenarios.

[0006] • Compliance assurance: Data fingerprinting prevents tampering and supports judicial-grade evidence collection (vehicle-side physical interface + cloud API).

[0007] • Data stakeholders: automakers, suppliers of combined driver assistance and conditional autonomous driving systems, insurance companies, third-party organizations, and judicial departments.

[0008] These requirements set higher standards for data management and analysis of autonomous vehicles, and existing software and technologies are insufficient to fully meet these needs.

[0009] Existing analytics tools suffer from numerous shortcomings when processing the complex data from autonomous vehicles, including operational complexity, data fragmentation, inconsistent analytical methods, difficulties in multi-person collaboration, and insufficient compliance guarantees. These issues not only increase operational complexity and learning costs but also lead to complex data retrieval and low analytical efficiency, impacting the safety and reliability of autonomous driving systems. Furthermore, existing systems struggle to meet national standards for data integrity and authenticity, and cannot effectively support forensic investigations. Summary of the Invention

[0010] This application provides an autonomous driving accident handling method and related equipment to at least solve one of the above-mentioned technical problems.

[0011] In a first aspect, embodiments of this application provide an autonomous driving accident handling method, comprising: acquiring raw data uploaded from the vehicle and / or cloud, and processing the raw data, wherein the raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packets, ROSbag, and CAN logs; visualizing the processed data to obtain a visualization interface, wherein the visualization interface includes accident-related data records, accident video frame playback and images, a 3D model obtained by combining data abstraction, and / or a predicted visualization trajectory; storing the editing operations of the analyst in the visualization interface, wherein the editing operations include at least one of the following: marking the accident occurrence time, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the 3D model state; and packaging and sharing the visualization interface and / or the editing operations in response to the analyst's sharing instruction.

[0012] Secondly, embodiments of this application provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described autonomous driving accident handling methods of this application.

[0013] Thirdly, embodiments of this application provide a storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-mentioned autonomous driving accident handling methods of this application.

[0014] Fourthly, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-mentioned autonomous driving accident handling methods.

[0015] The method of this application acquires raw data uploaded from the vehicle and / or the cloud, integrates data in different formats, supports multiple data sources, simplifies the data acquisition process, and then provides various visualization methods to present the processed data. It supports real-time interactive operation, enabling users to intuitively analyze the data. Finally, by storing the operations of the analysts, it supports one-click sharing, supports multi-person collaborative work, and provides real-time data sharing, annotation, discussion and other functions to improve team collaboration efficiency, enabling users to better collect, analyze and process data related to autonomous driving accidents. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The conditional autonomous driving accident definition process in the relevant technical specifications; Figure 2 A flowchart illustrating an autonomous driving accident handling method provided in one embodiment of this application; Figure 3 A system framework diagram of a specific example of an autonomous driving accident handling method provided in an embodiment of this application; Figure 4 A data sharing flowchart provided for one embodiment of this application; Figure 5 A product architecture diagram provided for one embodiment of this application; Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In related technologies, data formats are diverse. Functions such as after-sales, R&D, and testing focus on data in various formats, requiring different software for analysis, increasing operational complexity and learning costs. Furthermore, data is scattered across various systems, making data retrieval complex and time-consuming, impacting work efficiency. In addition, inconsistent analytical methods across departments lead to high costs of mutual understanding and hinder effective communication and collaboration. Moreover, complex multi-person collaboration makes it difficult to work collaboratively on the same problem, resulting in low problem-solving efficiency. Further, related technologies lack automated accountability processes; data classification, accountability, and report generation all rely on manual execution, leading to inefficiency. Finally, traditional analytical tools are complex to operate and costly to use. Different tools have inconsistent operational logic and supporting systems, requiring the use of multiple software and systems to analyze a single problem, placing a heavy mental burden on users.

[0020] Figure 1 The conditional automated driving accident definition process is shown in the specification document mentioned in the background art.

[0021] The methods for determining the time of the first collision include: 1) determining based on data records directly related to the collision (such as collision sensor data or airbag deployment records); 2) determining based on collision frames in video footage (such as collision frame timestamps); 3) determining based on vehicle operation data (such as automatic braking function activation data); the time of the collision can be determined by any of the above methods, or by combining two or three methods.

[0022] Situations where the conditional automated driving system was activated before the accident include: 1) the time of the first collision being the moment the conditional automated driving system was activated; 2) the conditional automated driving system was performing vehicle motion control when a collision risk occurred, and a collision still occurred despite the driver's reasonable intervention to avoid the danger. Furthermore, the activation moments of the conditional automated driving system include: 1) when the conditional automated driving system is performing vehicle motion control; 2) when the conditional automated driving system is issuing an intervention request while still performing vehicle motion control; 3) when the conditional automated driving system is escalating an intervention request while still performing vehicle motion control; 4) when the conditional automated driving system is implementing a minimum risk strategy (MRM) or taking measures to mitigate vehicle risk; 5) when the conditional automated driving system is indicating insufficient takeover capability while still performing vehicle motion control; 6) when the dynamic driving task support user intervenes in longitudinal motion control, but the conditional automated driving system does not disengage; 7) when the dynamic driving task support user's intervention in lateral motion control does not exceed the reasonable thresholds declared by the vehicle manufacturer to prevent misuse; 8) when the conditional automated driving system triggers a temporary disengagement while still performing vehicle motion control; 9) when the conditional automated driving system reduces or suppresses the intervention of the dynamic driving task support user; 10) 11) In the event of a serious vehicle failure or a serious failure of the conditional automated driving system, the conditional automated driving system executes other safety exit control strategies declared by the vehicle manufacturer; 12) In the event of a serious vehicle failure or a serious failure of the conditional automated driving system, the conditional automated driving system exhibits unexpected behavior or defects.

[0023] Furthermore, if the accident meets the aforementioned conditions of conditional autonomous driving system activation before the accident, and there are exceptions to the conditional autonomous driving accident scenario, then it does not meet the conditional autonomous driving accident scenario; if the accident meets the aforementioned conditions of conditional autonomous driving system activation before the accident, and there are no exceptions to the following conditional autonomous driving accident scenarios, then it meets the conditional autonomous driving accident scenario. Exceptions to Conditional Automated Driving (CDA) accidents include: 1) The driver or dynamic driving support user is driving under the influence of alcohol, drugs, or without a license; 2) Activating the CDA system or impairing its normal operation by damaging vehicle parts, sensors, or replacing substandard or illegally modified parts; 3) Activating or maintaining the CDA system by interfering with or deceiving sensors; 4) Activating or maintaining the CDA system illegally beyond its designed operating range, such as tampering with the location to bypass the geofence set by the manufacturer; 5) Activating the CDA system or impairing its normal operation by attacking the software system, writing illegal programs, or tampering with data; 6) Accidents where the vehicle is not at fault; 7) Accidents caused by unreasonable behaviors such as forcibly opening doors, hoods, tailgates, leaning out of the vehicle, throwing objects, extending foreign objects, dangerous driving, or intentionally causing a collision during the activation of the CDA system; 8) Accidents occurring during track driving, closed-course testing, or while the vehicle is seized or impounded by courts, police, or other authorities; 9) 10) Accidents that occurred while the driver or the user supporting the dynamic driving task was illegally in control of the vehicle through theft, robbery, snatching, or other means.

[0024] The standard document also mentions the following compliance and trust requirements for vehicle data collaboration: 1) Vehicle manufacturers should take measures to ensure the security, integrity, and trustworthiness of data related to the determination of combined driving assistance and conditional autonomous driving during the processes of collection, storage, processing, transmission, and calculation both locally on the vehicle and in the cloud. 2) After the data related to the determination of accidents involving combined driving assistance and conditional autonomous driving is transmitted back to the vehicle manufacturer's vehicle networking platform, a data fingerprint should be generated, and anti-tampering measures should be applied to the data fingerprint using technologies such as blockchain and data encryption. The blockchain should be distributed among collaborating parties, and multi-party data verification should be achieved using encryption and other technologies. 3) For intelligent connected vehicles with on-vehicle data processing capabilities, data fingerprint calculation can be completed on the vehicle itself. Data fingerprints should be retained for at least 6 months. 4) The accident analysis and determination process should employ encryption and anti-tampering methods to apply consistency verification mechanisms to raw data, intermediate process data, and output data. 5) In cases where personal information is processed in the determination of accidents involving combined driving assistance and conditional autonomous driving, the default setting should be that sensitive personal information is not collected, and individual consent should be obtained for each piece of sensitive information. If personal information outside the vehicle is collected without consent and then provided to the outside of the vehicle, it should be anonymized.

[0025] In response to one or more problems existing in related technologies, and following the conditional autonomous driving accident definition process in the specification documents, this application proposes an autonomous driving accident handling method.

[0026] Please refer to Figure 2 The diagram illustrates a flowchart of an autonomous driving accident handling method provided in an embodiment of this application. The method of this embodiment can be applied to autonomous driving after-sales problem visualization and analysis software. Furthermore, the autonomous driving accident handling method may exist in the form of an algorithm in a computer; in subsequent embodiments, it may be referred to as an autonomous driving accident handling device, such as a piece of program code or software, etc., and this application does not impose any limitations herein.

[0027] like Figure 2 As shown, in step 201, the raw data uploaded from the vehicle and / or the cloud is obtained and processed. In step 202, the processed data is visualized to obtain a visualization interface; In step 203, the storage analyst performs editing operations in the visualization interface; In step 204, the visualization interface and / or the editing operation are packaged and shared in response to the analyst's sharing instruction.

[0028] In this embodiment, for step 201, the autonomous driving accident handling device acquires raw data uploaded from the vehicle and / or the cloud. This raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packets, ROSbag, and CAN logs. Specifically, fleet learning data is not the entire dataset, but rather "valuable fragments" selected by triggers. The collection scenario can be trigger events during the daily driving of the user's fleet. Autonomous driving data recording system data packets (DSSAD packets, Data Storage System for Automated Driving) can include timestamps, vehicle status, autonomous driving system status, system behavior, driver status, external conditions, and event markers. It is a compliant "accident black box," and its collection scenario is saving data only when a specific event triggers (collision, system failure, etc.). ROSbag (Robot Operating System Bag) records complete communication data for all topics in the ROS system (such as images, point clouds, perception results, planned trajectories, etc.). CAN logs (Controller Area Network Log) include the raw sequence of CAN frames, including ID, data bytes, and timestamps. Subsequently, the autonomous driving accident handling device processes these raw data, which may include encryption and compliance processing to ensure that the data complies with relevant regulations. For example, a consistency verification mechanism can be applied to the raw data, intermediate data, and output data to achieve encryption and tamper-proofing. In addition, sensitive data should be desensitized or anonymized in accordance with relevant regulations.

[0029] Subsequently, in step 202, the autonomous driving accident processing device visualizes the processed data to obtain a visualization interface. This visualization interface includes at least one of the following: records of data directly related to the accident, playback of accident video frames and images, and a 3D model and / or predicted visualized trajectory derived from data abstraction. For example, relevant text and data (such as vehicle status data for a period before and after the accident) are directly recorded through logs and tables to determine the accident situation; video frame playback and images record the accident scene; and the abstracted 3D model and algorithm-predicted trajectory visualization simulate the entire physical scene, combining software and algorithm data to provide decision-making support for data analysis. It can also provide heatmaps, time-series graphs, interactive dashboards, etc., satisfying data recording requirements while presenting the data in a more intuitive form and supporting real-time interactive operation. Presenting the data through a visualization interface allows for a more intuitive and clearer presentation of accident data, enabling analysts to better analyze the accident.

[0030] Then, for step 203, the autonomous driving accident handling device stores the editing operations performed by the analyst in the visualization interface. These editing operations include at least one of the following: marking the accident occurrence time, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the 3D model state. Specifically, marking the accident occurrence time refers to the analyst (e.g., R&D user) marking the time point of the accident in the visualization interface; adding annotations refers to the R&D user adding corresponding annotations in the relevant interface; annotations for the same data by collaborators can also be synchronized to other R&D users; adding discussions refers to the R&D user adding relevant questions to be discussed in the relevant interface, or creating discussion groups for multiple R&D users to discuss data processing business, etc.; commenting on question types refers to examples of R&D users' commenting actions on the data, which can include commenting on question types, classifying data into questions, or adding other annotations. Analyzing data also refers to further analysis actions by the R&D user. In the aforementioned embodiment, the 3D model was used to simulate and abstract the physical world; adjusting the model perspective refers to the user being able to drag and interact within the software to adjust the model perspective during data analysis to achieve higher analysis efficiency, and is also an operation performed by the user during data analysis. Adjusting the model status refers to the ability to show / hide the model, highlight / darken it, etc., to facilitate data analysis. The adjustments can be saved and shared downstream for further analysis based on the existing model status. Furthermore, editing operations can also include result and status confirmation; that is, some analysts (such as operations users) can further compare and confirm existing analytical data and results when performing qualitative analysis.

[0031] Finally, in step 204, in response to the analyst's sharing instruction, the visualization interface and / or the editing operation are packaged and shared. Specifically, this can be achieved through a secure link, allowing for one-click sharing of the current analysis status, incident markers, and data changes, ensuring real-time data synchronization between different users.

[0032] This application embodiment acquires raw data uploaded from the vehicle and / or cloud, integrates data in different formats, supports multiple data sources, simplifies the data acquisition process, and then provides various visualization methods to present the processed data. It supports real-time interactive operation, enabling users to intuitively analyze the data. Finally, by storing the operations of the analysts, it supports one-click sharing, supports multi-person collaborative work, and provides real-time data sharing, annotation, discussion and other functions to improve team collaboration efficiency, enabling users to better collect, analyze and process data related to autonomous driving accidents.

[0033] In some optional embodiments, processing the raw data includes encrypting and complying with regulations for the raw data uploaded from the vehicle and / or the cloud. Specifically, in cases involving the processing of sensitive personal information in the determination of accidents related to combined driver assistance and conditional automated driving, the default setting is to not collect sensitive personal information, and separate consent is obtained for each piece of sensitive personal information. If personal information outside the vehicle is collected without obtaining individual consent and provided to the outside of the vehicle, it should be anonymized, including deleting images that can identify natural persons, or partially outlining facial information in the images. Personal information in collected vehicle operation data and other data should be subject to access control measures to prevent unauthorized access.

[0034] In some optional embodiments, after the compliance processing of the raw data uploaded from the vehicle and / or cloud, the method further includes: automatically parsing the time boundaries of the compliant data to create at least one trip slip with a unique trip ID; and returning at least one compliant trip slip in response to the trip retrieval by the analyst. The time boundary parsing can be achieved through the collaborative work of multiple detectors, such as ignition / shutdown detection, AD (Automated Driving) mode switching detection, and long-term stationary detection, to obtain multiple vehicle start-stop times as multiple candidate boundaries. Then, a preset fusion strategy is used to pair and weight the multiple candidate boundaries with confidence levels, and the corresponding time boundaries are output, which may include start and end times, start and end reasons, and confidence levels. After parsing the time boundaries, a trip slip can be obtained, and a unique trip ID can be created based on the date, vehicle, and day number. This trip ID may also include a checksum to prevent tampering. For example, trip data corresponding to the time boundaries, such as vehicle (or license plate number), vehicle type, city, time range, vehicle identification number, driving trajectory, vehicle speed, and surrounding environmental information, can be integrated to obtain the trip slip. Then, when a trip retrieval request is received from an analyst, one or more trips that meet the criteria are output. Furthermore, when searching for trips later, analysts can provide information from multiple dimensions such as vehicle type, vehicle, city, time range, and VIN (Vehicle Identification Number) for retrieval and arbitrary combinations of cross-referencing, facilitating the rapid retrieval of trip data that meets the criteria from massive amounts of data based on specified features and search scope. This application's embodiment automatically segments massive, disorganized, and anonymized vehicle time-series data into standardized trip units with unique IDs, and supports multi-dimensional, second-level retrieval, providing analysts with a one-stop solution "from data to trip".

[0035] In some optional embodiments, after obtaining the raw data uploaded from the vehicle and / or cloud, the method further includes: retrieving the encrypted raw data in response to a judicial-grade evidence retrieval request. Specifically, judicial-grade evidence retrieval is the process of securely transferring encrypted raw data held by the automaker to judicial personnel in a manner that is admissible in court and verifiable by technology, under the four ironclad principles of legal authorization, minimum necessity, end-to-end encryption, and on-chain evidence storage.

[0036] Specifically, the system has a built-in permission configuration and verification process for judicial-level evidence collection requests to ensure legal authorization and adherence to the minimum necessary requirements. For example, the system only grants administrators or a few high-level users permission to access raw data, and only users with the corresponding permissions can initiate judicial-level evidence collection requests. Furthermore, to enhance security, when a relevant user discovers a judicial-level evidence collection request, the system can require the user to upload verification materials (such as user authorization for the system to extract their biometric information, user-uploaded official judicial-level evidence collection authorization documents, user's special permission password, etc.). The system can analyze and identify these verification materials and verify their authenticity before responding to the request; alternatively, after a user uploads relevant materials / initiates an evidence collection request, the system will only respond to the request after confirmation / approval by pre-defined approvers. Furthermore, to achieve the minimum necessary principle, the system will retrieve encrypted data within the scope of authorization identified by the verification materials or the pre-defined permission range. The system can use blockchain or other legally compliant encryption schemes for data, and this application does not impose any restrictions. For example, the system (such as a compliant cloud) encrypts and de-identifies the raw data from the vehicle and then uploads it to the vehicle manufacturer's cloud. After authorization verification, it responds to judicial-level evidence collection requests and, after authorization verification is passed, retrieves the necessary encrypted raw data from the vehicle manufacturer's cloud according to the authorized scope, thereby achieving judicial-level evidence collection.

[0037] In some optional embodiments, the method further includes applying a consistency verification mechanism to the original data, the processed data, the intermediate process data, and the final output data. For example, the consistency verification mechanism establishes a "hash passport" (such as a Merkle tree, a digital identity solution built using hashing and cryptography techniques) for each bit of data from its creation to delivery. Through step-by-step signing, chain verification, and on-chain evidence storage, any tampering at any stage can be immediately detected, accurately located, and legally verified. This ensures that the original data → intermediate process → final output remains unchanged, intact, and authentic throughout its entire lifecycle of storage, transmission, and processing, forming an undeniable chain of evidence. Furthermore, checksums (such as MD5 / CRC), Raft / Paxos consensus algorithms, version vectors, or Bloom filters can also be used to ensure the consistency and integrity of data across different nodes or replicas.

[0038] In some optional embodiments, the method further includes: performing anomaly scene mining on the processed raw data using preset trigger conditions, wherein the preset trigger conditions can be activation conditions built into the rule engine, such as AEB trigger, lateral error > 0.4 m, and longitudinal impact > 3 m / s. 3 When pre-set abnormal scenario conditions are met, the corresponding original data needs to be extracted as abnormal scenario data. Self-supervised clustering (e.g., using Isolation Forest (an unsupervised machine learning algorithm) + PCA) is performed on each abnormal scenario. When an unknown abnormal cluster is discovered, at least the start and end timestamps and confidence scores of the abnormal segments corresponding to the unknown abnormal cluster are output. Self-supervised clustering is an advanced machine learning method combining self-supervised learning and unsupervised clustering. Its goal is to automatically divide data into meaningful clusters without manual labeling. An abnormal segment refers to a subset of data with a continuous time span in continuous time series data or log streams that the algorithm determines does not belong to a normal behavior pattern. The abnormal segments are labeled, and the labeling results are stored and corresponding semantic version numbers are generated. This allows for the automatic discovery of unknown abnormal scenarios (long-tail problems) in massive amounts of compliant data, the identification of novel failure modes through self-supervised clustering, and the labeling and versioning management of abnormal segments, forming a continuously evolving scenario library.

[0039] In some optional embodiments, the method further includes: classifying the processed raw data within a preset time period, performing accident rate statistics and / or customer complaint cause statistics; and generating a journal report based on the classification and statistical results. This allows for multi-dimensional classification and statistics of compliant data within a preset time period, transforming scattered after-sales issues, incidents, and customer complaint feedback into a structured, readable, and distributable journal report to support quality review and decision-making.

[0040] Optionally, the method further includes: obtaining analysis results based at least on the visualization interface and / or the editing operation; automatically assigning responsibility for data where the analysis results indicate the existence of an exception scenario for conditionally automated driving accidents, or data where there is no conditionally automated driving activation scenario before the accident; and generating an analysis report based on the analysis results for data that conforms to a conditionally automated driving assistance accident scenario, wherein the analysis report includes a problem description, data details, cause analysis, and / or solution suggestions. Thus, based on the visualization analysis interface and manual editing operation, a final responsibility determination is made for the accident data, and the data is automatically categorized according to the determination results: Existing exception scenario / no AD activation → Automatic responsibility assignment (user / third party); Conforms to AD-assisted accident scenario → Generation of analysis report (technical attribution + solution). The system deeply integrates visual analysis with manual editing: For scenarios where the vehicle is not liable, it automatically determines responsibility in milliseconds, diverting judicial / insurance processes. For example, the system can link to insurance platforms, traffic bureau / vehicle manufacturer platforms, and / or vehicle owner terminals, directly pushing analysis results to the corresponding platforms and / or vehicle owner terminals, thus automatically diverting responsibility after determination. For scenarios where the vehicle is liable, it automatically generates structured chemical report forms, providing a complete closed loop from problem description to solution, achieving accident handling from liability determination to product improvement. Exceptions to these scenarios can be found in the aforementioned regulatory documents and will not be elaborated upon here.

[0041] Please refer to Figure 3 This diagram illustrates a system framework of a specific example of an autonomous driving accident handling method provided in an embodiment of this application. The autonomous driving problem visualization and analysis software system is a specific example of the autonomous driving accident handling method.

[0042] This invention relates to the field of integrated driver assistance systems, and in particular to the problems of diverse data formats, scattered data, inconsistent analysis methods, difficulties in multi-person collaboration, and complex operation of traditional analysis tools in the after-sales, R&D, and testing functions of conditionally automated vehicles. It provides an integrated visualization analysis software system for autonomous driving problems.

[0043] like Figure 3 As shown, the OEM (Original Equipment Manufacturer) cloud environment includes an interface and services. The interface includes multiple modules such as trip retrieval, trip viewing, data playback, and operational data dashboards. Services include OEM-owned services, trip management services, data mining services, and operational dashboard services.

[0044] The system includes the following features: Trip Search: Provides a multi-dimensional search interface based on vehicle type, city, time range, VIN code, and other dimensions, allowing for arbitrary combinations and cross-searches. This facilitates rapid data retrieval from massive datasets based on specified features and search scopes. Trip View: After retrieving the corresponding data, users can view trip details with a single click, displaying a driving details interface. Driving details include overall vehicle information, start and end times, and start and end locations. Data Replay (Visualization Analysis Module): This module is the main visualization and interactive module. It offers various visualization formats for scenario data, DTC (Diagnostic Trouble Code) data, DSSAD (Data Storage System for Automated Driving) data, and other thematic data, integrated into this module to provide a simulated data playback visualization interface. For example, it provides logs and tables to directly record relevant text and data, which can be used to determine the accident situation; it provides video frame playback and images to record accident scene data; combined with an abstracted 3D model, it provides functions such as algorithm prediction trajectory visualization and drawing, which can simulate the entire physical scene and combine software and algorithm data to provide decision-making for data analysis. It also provides heat maps, time series graphs, interactive dashboards, etc. It ensures that the visualization types meet the data recording requirements for the first accident, are presented in a relatively intuitive form, and support real-time interactive operation. Furthermore, in addition to visualization forms such as 3D models, heat maps, and time series graphs, the system can also use virtual reality (VR) or augmented reality (AR) technology for visualization and interaction, further enhancing the user's immersion and interactive experience; this application has no restrictions. Operational data dashboard: It performs operation and analysis on vehicle data within a time period, automatically classifies data, calculates accident rates, identifies customer complaint reasons, etc., and provides periodic reports.

[0045] Furthermore, the trip management service includes: 1. Trip creation: Receiving FL (Fleet Learning) data, DSSAD packets, ROSbag, CAN logs, etc. uploaded from the vehicle / cloud → automatically parsing time boundaries → generating a unique TripID (trip ID). 2. Trip cascading storage: For hot data (≤7 days) → Kafka → ClickHouse, ensuring second-level retrieval; for cold data → object storage + EC redundancy, reducing costs. 3. Indexing and retrieval: Inverted index, including: VIN, start and end time, start and end location, scene tag, event type, DTC code; supports combined queries (e.g., "Shenzhen area last week + parking failure + steering wheel vibration"). 4. Permissions and compliance: Filtering based on OAuth2.0 returned fields to set different permissions for different user roles, such as after-sales roles only being able to view authorized area data, and judicial evidence collection roles being able to pull the original encrypted packets. 5. External Interface: Supports RESTful + GraphQL dual protocols: the front-end interface uses GraphQL to retrieve fields on demand; third-party systems use RESTful for batch retrieval. Provides a "find itinerary" experience in seconds; at the same time, it provides a unique, complete, and reusable data entry point for the upper-level "itinerary viewing / playback / dashboard".

[0046] Data mining service: Transforming petabyte-level logs dormant in the data lake into a "high-value scenario library," achieving an automatic anomaly detection → labeling → storage → operational closed loop. This service includes a data mining pipeline, such as an anomaly scenario mining pipeline: The rule engine has built-in multiple activation conditions (such as AEB trigger, lateral error > 0.4 m, vertical impact > 3 m / s). 3 The service utilizes self-supervised clustering (Isolation Forest + PCA) to identify unknown anomaly clusters after the data meets the activation conditions, outputting the start and end timestamps and confidence scores of the anomaly segments. It also includes data classification and labeling functionality: the front end embeds a four-step operation of "playback-pause-box selection-label selection," providing conditions for manual labeling; simultaneously, it automatically recommends labels using a historical labeling model, requiring only manual confirmation / modification, improving efficiency by 5x. The service also includes database entry and versioning functionality: label results are written to "Classification Result Storage" and a semantic version number is generated, allowing for traceability and rollback. This improves data mining efficiency, proactively intercepts potential incidents / after-sales risks, and achieves a closed loop of data-driven iteration.

[0047] Operational data dashboard service: Update operational dashboard data based on stored operational data, and host operational journals based on journal reports.

[0048] Specifically, the autonomous driving problem visualization and analysis software system of this application embodiment may include the following main modules: Data Integration Module: Integrates data in different formats and supports multiple data sources, including vehicle-side and cloud-based data. In addition to integrating data through data adapters and API interfaces, it can also utilize data warehouse technology to uniformly store data of different formats in a data warehouse, facilitating subsequent analysis and processing.

[0049] Data Management Module: Centrally manages data, providing functions such as data search, filtering, and classification, simplifying the data acquisition process. It acquires vehicle lateral and longitudinal motion data, driver control data, vehicle combined assisted driving and conditional automated driving request and upgrade signals, system and hardware data, and accident occurrence time (TriggerTime) and ±10s before and after the incident. It ensures that the data acquisition types meet the requirements to determine the type of accident / after-sales issue.

[0050] The visualization analysis module, like the data playback module mentioned above, will not be described in detail here.

[0051] The shared collaboration module enables data sharing through a secure connection, allowing users to share the current analysis status (the content after the visualization analysis module presents various data visually), incident markers, and data status with a single click, ensuring real-time data synchronization across different users. It supports multi-user collaboration, providing real-time data sharing, annotation, and discussion features. For example, team members can view and edit the same data in real time through the online collaboration platform, adding annotations and engaging in discussions, improving team collaboration efficiency. Furthermore, in addition to the online collaboration platform, multi-user collaboration can also be achieved through instant messaging tools or project management software; this application does not impose any restrictions.

[0052] Report Generation Module: Automatically generates reports based on analysis results, with built-in data models and workflows required by specifications. For example, based on data feedback, it automatically assigns responsibility and rejects after-sales processes for cases involving exceptions to conditionally automated driving accidents or where there were no pre-accident conditionally automated driving activation scenarios, significantly reducing the workload of manual analysis. For data that meets the criteria for conditionally automated driving assistance accidents, it generates detailed analysis reports based on the user's analysis results, including problem descriptions, data details, cause analysis, and solution suggestions. Furthermore, in addition to the built-in responsibility assignment system and specifications, data classification and responsibility assignment can also be achieved through manual data analysis and module categorization; this application does not impose any restrictions on this.

[0053] Compliance Cloud Synchronization: To ensure safety and regulatory compliance, data generated during the accident analysis and judgment process is encrypted, fingerprinted, and tamper-proofed through the compliance cloud's compliance services. A consistency verification mechanism is applied to raw data, intermediate process data, and output data to ensure data integrity and authenticity, supporting forensic-level evidence collection. Visualization results are processed by removing images containing natural persons or partially outlining facial information. Access control measures are implemented for vehicle operation data and information recorded by systems such as DVR, DSSAD, and EDR to prevent unauthorized access. Location trajectory data, external video, and external image data are all stored domestically. The data collaboration compliance requirements, data collaboration trust requirements, data storage requirements, and data retrieval requirements are met as specified in regulatory documents. Furthermore, in addition to data fingerprinting technology, blockchain technology can be used to further enhance the immutability and trustworthiness of the data; this application does not impose any restrictions on this.

[0054] Further reference Figure 4 The diagram illustrates a data sharing flowchart provided in one embodiment of this application. In a specific application scenario, the accident handling method / system / module provided in the above embodiment of this application can achieve the following functions: sharing of analysis processes and annotations, as well as data sharing.

[0055] The analysis process and annotation sharing include: R&D users can mark the time point of an accident, recording data 10 seconds before and 10 seconds after the collision. Team members sharing the data can quickly locate the moment of the accident and view the data changes before and after it. This function allows for rapid and focused analysis of sensor data, environmental perception data, and decision system data within this time period. In a specific example, user A can annotate the time point of the accident (triggerTime) or comment on the issue type, and then generate a sharing link A to send to user B. This sharing will include the aforementioned information, allowing user B to continue the analysis. During the analysis, user A can also drag the progress bar to locate keyframes, adjust the 3D scene to a special perspective, and / or adjust the model state, then generate a sharing link B. This sharing will include this information, facilitating user C's continued analysis. Furthermore, user B and user C can be the same user, and two sharing links can be merged into one. This application does not limit the number of sharing links, which will not be elaborated further here. Data sharing: The shared data includes not only basic sensor and environmental perception data, but also complete data status, such as timestamps, camera viewpoints, and 3D model positions. Members whose data is shared can click a link to instantly access and restore all analyzed data, ensuring consistency across departments and among multiple individuals. This completeness and consistency of data is crucial for ensuring the accuracy of the analysis.

[0056] like Figure 4As shown, "Annotate triggerTime" refers to marking the time point when the incident occurred. "Comment on issue type" refers to examples of comments made by R&D users on the data, which can be comments on issue types, classifying the data into issues, or adding other annotations. "Analyze data" also refers to further analysis actions by R&D users, such as viewing data before and after the incident. Adjusting the model perspective refers to the 3D scene in the visualization analysis module of the aforementioned embodiment. The 3D model simulates and abstracts the physical world. Users can drag and interact to adjust the model perspective within the software when analyzing data to achieve higher analysis efficiency. Adjusting the model perspective is also an operation performed by users when analyzing data. Adjusting the model state refers to the ability to show / hide, highlight / darken, etc., the model to facilitate data analysis. The adjustment results can be saved and shared downstream for further analysis based on the existing model state. Result and state confirmation refers to the ability of some operational users to further compare and confirm the results based on existing analysis data when qualitatively analyzing the data. All operations used in data analysis can come from the same or different users; this application does not impose any restrictions. Furthermore, the final results generated after data analysis can also be shared via links to data / result users (such as car manufacturers, car owners, judicial evidence collection units, etc.), and this application does not impose any restrictions on this.

[0057] Further reference Figure 5 The diagram illustrates the product architecture of an autonomous driving accident handling system according to an embodiment of this application.

[0058] like Figure 5 As shown in the embodiments of this application, the autonomous driving accident handling visualization analysis software is an integrated solution based on a modular product architecture, consisting of four main layers: application layer, service layer, data layer, and infrastructure layer. Each layer undertakes key functions to support the operation of the entire system.

[0059] The application layer is the user interface for interacting with the system, providing various functional modules to meet the needs of different users. These include: Trip management: Tracking and managing the driving data of autonomous vehicles.

[0060] Document Management: Manage documents and records related to incidents / after-sales service.

[0061] Operations Dashboard: Provides real-time operational data and statistics.

[0062] Data playback: Allows users to replay historical data for analysis and problem diagnosis.

[0063] Data analytics: Provides advanced data analytics tools to help users gain insights into patterns and issues behind their data.

[0064] The service layer is the core of the software, providing a series of backend services to support the functions of the application layer: Remote data access: Allows users to access and manipulate data from remote locations.

[0065] User management: Manage system users and permissions to ensure data security.

[0066] Managed layout services: Manage user interface layouts and provide a personalized user experience.

[0067] Data tags: Classify and label data to facilitate searching and analysis.

[0068] Data retrieval: Provides powerful data retrieval functions to quickly locate the information you need.

[0069] Data mining: using data mining techniques to discover potential value and patterns in data.

[0070] The data layer is responsible for storing and managing various types of data, including: Fleet Data: This data records the operational information of the entire fleet.

[0071] CAN data: Internal vehicle communication network data used to diagnose problems inside the vehicle.

[0072] Visual Data: Such as images and videos captured by a camera.

[0073] RosBag V2: A format for storing ROS (Robot Operating System) data, suitable for autonomous vehicles.

[0074] The infrastructure layer is the basic infrastructure of the entire system, providing basic functions for data storage and message passing: Database: Stores structured data and supports efficient data querying and management.

[0075] Storage: Provides large-scale data storage solutions to ensure data persistence and reliability.

[0076] Kafka: A distributed messaging system for processing real-time data streams and supporting high-throughput data transmission.

[0077] Through this layered architecture, the embodiments of this application provide a powerful, flexible, and easily scalable system platform for the visual analysis of after-sales issues related to autonomous vehicles. It not only improves the speed and accuracy of problem diagnosis but also provides in-depth business insights through data mining and analysis, helping enterprises optimize operations and improve service quality.

[0078] By constructing an integrated autonomous driving problem visualization and analysis software system, key issues such as data fragmentation, difficulties in team collaboration, and high user barriers are effectively addressed. Firstly, the system unifies multi-source data, supporting multiple data formats such as Rosbag and CAN, breaking down data silos and ensuring smooth data integration and efficient processing. Secondly, the system supports cross-functional team collaboration across platforms, providing process tracking functionality, bridging barriers between different functions, and significantly improving team collaboration efficiency. It also offers an intelligent analysis experience ranging from out-of-the-box to deep customization. Through an intuitive interface and flexible configuration options, it reduces the cognitive load on users, allowing both novice and experienced users to quickly get started and deeply customize the system to meet the needs of different users.

[0079] The present invention provides a one-stop data display and analysis tool that conforms to the combined assisted driving liability determination standard. It automatically collects and analyzes similar data through an intelligent data platform dashboard, and displays the problem flow nodes in real time in combination with a closed-loop tracking mechanism. This significantly improves data analysis efficiency and enhances process observability, thereby providing users with an efficient, transparent and intelligent data processing solution.

[0080] Specifically, it integrates multiple data formats: The data integration module integrates data of different formats, supports multiple data sources, and simplifies the data acquisition process. It centrally manages data: The data management module centrally manages data, providing functions such as data searching, filtering, and classification, improving data acquisition efficiency. It offers multiple visualization formats: The visualization analysis module provides multiple visualization formats, supports real-time interactive operations, and enables users to intuitively analyze data. It supports multi-user collaborative work: The collaborative work module supports multi-user collaborative work, providing functions such as real-time data sharing, annotation, and discussion, improving team collaboration efficiency. It provides detailed analysis reports: The report generation module generates detailed analysis reports based on the user's analysis results, providing strong support for problem-solving. It ensures data compliance: The compliance assurance module ensures the integrity and authenticity of data, supports judicial-level evidence collection, and meets the requirements of relevant regulatory documents.

[0081] The system provided in this application embodiment has its built-in functions and analysis process fully compliant with relevant specifications, offering clear and reasonable support for accident handling / after-sales problem analysis processes. It also achieves seamless integration of different types of data. Furthermore, it provides users with an intelligent analysis experience ranging from out-of-the-box to deep customization, intelligently generating problem analysis reports and automatically extracting key data. In addition, data analysis and accountability processes support one-click sharing of keyframe locations, accelerating the analysis, processing, and accountability determination process.

[0082] In other embodiments, this application also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the autonomous driving accident handling method in any of the above method embodiments; As one implementation, the non-volatile computer storage medium of this application stores computer-executable instructions, which are configured as follows: Acquire raw data uploaded from the vehicle and / or cloud, and process the raw data, wherein the raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packet, ROSbag, and CAN log; The processed data is visualized to obtain a visualization interface, which includes data records directly related to the accident, accident video frame playback and images, 3D models and / or predicted visualization trajectories obtained by combining data abstraction. The storage analyst's editing operations in the visualization interface include at least one of the following: marking the time of the incident, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the state of the 3D model; In response to the analyst's sharing instruction, the visualization interface and / or the editing operation are packaged and shared.

[0083] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the autonomous driving accident handling device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and this remote memory may be connected to the autonomous driving accident handling device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0084] This application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform any of the above-described autonomous driving accident handling methods.

[0085] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6As shown, the device includes one or more processors 610 and a memory 620. Figure 6 Taking a processor 610 as an example, the device for handling autonomous driving accidents may further include an input device 630 and an output device 640. The processor 610, memory 620, input device 630, and output device 640 can be connected via a bus or other means. Figure 6 Taking a bus connection as an example, the memory 620 is the aforementioned non-volatile computer-readable storage medium. The processor 610 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 620, thereby implementing the autonomous driving accident handling method described in the above embodiment. The input device 630 can receive input digital or character information and generate key signal inputs related to user settings and function control of the autonomous driving accident handling device. The output device 640 may include a display device such as a screen.

[0086] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0087] In one embodiment, the above-described electronic device is used in an apparatus for an autonomous driving accident handling method, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire raw data uploaded from the vehicle and / or cloud, and process the raw data, wherein the raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packet, ROSbag, and CAN log; The processed data is visualized to obtain a visualization interface, which includes data records directly related to the accident, accident video frame playback and images, 3D models and / or predicted visualization trajectories obtained by combining data abstraction. The storage analyst's editing operations in the visualization interface include at least one of the following: marking the time of the incident, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the state of the 3D model; In response to the analyst's sharing instruction, the visualization interface and / or the editing operation are packaged and shared.

[0088] The electronic devices described in this application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0089] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0090] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (e.g., iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0091] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0092] (5) Other electronic devices with data interaction functions.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for handling autonomous driving accidents, comprising: Acquire raw data uploaded from the vehicle and / or cloud, and process the raw data, wherein the raw data includes at least one of the following: fleet learning data, autonomous driving data recording system data packet, ROSbag, and CAN log; The processed data is visualized to obtain a visualization interface, which includes data records directly related to the accident, accident video frame playback and images, 3D models and / or predicted visualization trajectories obtained by combining data abstraction. The storage analyst's editing operations in the visualization interface include at least one of the following: marking the time of the incident, adding annotations, adding discussions, analyzing data, adjusting the perspective, and adjusting the state of the 3D model; In response to the analyst's sharing instruction, the visualization interface and / or the editing operation are packaged and shared.

2. The method according to claim 1, characterized in that, The processing of the raw data includes: Encrypt and comply with regulations the raw data uploaded from the vehicle and / or the cloud.

3. The method according to claim 2, characterized in that, After performing compliance processing on the raw data uploaded from the vehicle and / or the cloud, the method further includes: The system automatically parses the time boundaries of the compliant data and creates at least one itinerary with a unique itinerary ID. In response to the analyst's trip retrieval, at least one trip form that meets the requirements is returned.

4. The method according to claim 2, characterized in that, After acquiring the raw data uploaded from the vehicle and / or the cloud, the method further includes: In response to a judicial-level evidence request, the encrypted original data is retrieved.

5. The method according to claim 1, characterized in that, The method further includes: A consistency verification mechanism is applied to the original data, the processed data, the intermediate process data, and the final output data.

6. The method according to claim 1, characterized in that, The method further includes: Use preset trigger conditions to mine abnormal scenarios in the processed raw data; Perform self-supervised clustering on each abnormal scenario. When an unknown abnormal cluster is found, at least the start and end timestamps and confidence scores of the abnormal segments corresponding to the unknown abnormal cluster are output. The abnormal segments are labeled, the labeled results are stored, and corresponding semantic version numbers are generated.

7. The method according to claim 1, characterized in that, The method further includes: Perform data classification, accident rate statistics, and / or customer complaint reason statistics on the processed raw data within a preset time period; Journal reports are generated based on the results of classification and statistics.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The analysis results are obtained at least based on the visualization interface and / or the editing operations; For data whose analysis results indicate the existence of exceptions to conditional autonomous driving accidents, or for data where there are no pre-accident conditionsal autonomous driving activation scenarios, liability will be automatically determined. For data whose analysis results match the conditional automated driving assistance accident scenario, an analysis report is generated based on the analysis results. The analysis report includes a problem description, data details, cause analysis, and / or solution suggestions.

9. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

Images