Automatic driving hazard scene detection method and system, electronic equipment and vehicle

Abstract

The invention provides an automatic driving hazard scene detection method and system, electronic equipment and a vehicle, and relates to the technical field of automatic driving. According to the method, a multi-dimensional hazard scene detection rule system is established, so that accurate identification and response to key security events are realized. In the automatic driving operation process, the system continuously collects and compares a vehicle state signal with a preset triggering condition in real time, once any hazard scene condition is met, vehicle burying point data in a preset time window are collected and uploaded to a cloud end, and the vehicle burying point data are stored in the cloud end. Accident scene restoration, system risk assessment and fault root cause analysis are carried out based on the collected data, then a targeted safety repair scheme is generated, vehicle software is remotely upgraded in an OTA mode, closed-loop processing of safety problems and continuous optimization of the system are achieved, the safety, traceability and remote operation and maintenance capacity of the automatic driving system are effectively improved, and the safety and reliability of the automatic driving system are improved. And the running safety of the vehicle is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a hazard scene detection method and system for automatic driving, an electronic device and a vehicle. BACKGROUND

[0002] Traditional automatic driving systems usually lack a perfect vehicle monitoring mechanism, resulting in the inability to timely and effectively collect and record data when a hazard scene occurs. This technical deficiency makes it difficult to accurately restore the vehicle state, environmental information and system decision-making process at the time of the event, and thus it is difficult to clearly determine the cause of the accident and the responsibility. At the same time, due to the lack of real-time monitoring and data backhaul functions, the vehicle manufacturers and technical personnel cannot immediately grasp the long-tail problems and potential safety risks encountered by the vehicle in actual operation, affecting the rapid response and repair of system defects. In addition, the lack of targeted data accumulation also restricts the continuous optimization and positive development of the automatic driving system, reducing the overall safety and user trust. SUMMARY

[0003] The purpose of the present application is to provide a hazard scene detection method and system for automatic driving, an electronic device and a vehicle, to solve one or more technical problems existing in the prior art, and at least to provide a beneficial choice or create conditions.

[0004] The solution to the technical problem of the present application is: on the one hand, the present application provides a hazard scene detection method for automatic driving, comprising the following steps: setting a plurality of hazard scene detection rules, each type of rule corresponding to at least one trigger condition and a monitoring signal; continuously collecting vehicle signals during automatic driving; comparing the vehicle signals with the trigger conditions of the hazard scene detection rules to determine whether the trigger conditions are met; when the trigger conditions are met, collecting vehicle embedded point data in a time period around the trigger time according to a preset time window, and uploading to the cloud; based on the vehicle embedded point data, performing scene restoration, risk assessment and problem analysis in the cloud, generating a safety repair scheme, and upgrading the vehicle software through OTA.

[0005] Further, the hazard scene detection rules include a driver takeover rule, an emergency system exit rule, an accident rule, an exit operation design domain rule and a minimum risk rule; The trigger conditions of the driver takeover rule include but are not limited to: automatic driving function activated and steering wheel speed exceeding a preset speed threshold; The triggering conditions of the emergency system exit rule include but are not limited to: the automatic driving function is activated and the active safety system is activated. The triggering conditions of the accident rule include but are not limited to: the automatic driving function is activated and the passive safety system sends a collision signal. The triggering conditions of the exit operation design domain rule include but are not limited to: the environmental conditions exceed the automatic driving design domain. The triggering conditions of the minimum risk rule include but are not limited to: the automatic driving function is activated and the system sends the highest level alarm.

[0006] Further, the whole vehicle buried point data includes: Vehicle basic information, including vehicle identification number and location; Vehicle state and dynamic information, including ignition information; Driver state and operation information, including driver state monitoring information; Environment perception related information, including camera related video, light intensity information; Automatic driving system running information, including requested steering angle; Instrument and HMI display information, including DTC diagnostic fault code related information; New energy information, including SOC value at the start and end of charging.

[0007] Further, when the triggering condition is met, according to a preset time window, the whole vehicle buried point data in the time period near the triggering time is collected and uploaded to the cloud, including the following steps: Taking the timestamp of the triggering time as the origin, mark it as the reference time point of data collection; Based on the reference time point, the time window for data collection is set as [-15s, 5s] of the triggering time, covering the whole vehicle buried point data from 15 seconds before triggering to 5 seconds after triggering; Real-time store the whole vehicle buried point data in the time window; When the data volume of the collected whole vehicle buried point data accumulates to a preset data volume threshold, the cached whole vehicle buried point data is packaged and uploaded to the cloud within the user authorization range; After the cloud receives the whole vehicle buried point data, it is stored in association with the corresponding hazard scene type and triggering timestamp, supporting subsequent scene restoration and safety analysis.

[0008] Further, based on the whole vehicle buried point data, the scene restoration, risk assessment and problem analysis are performed in the cloud to generate a safety repair scheme, and the vehicle software is upgraded through OTA, including the following steps: After receiving the vehicle embedded data, the cloud verifies the data integrity and timing consistency of the vehicle embedded data, and removes outliers and redundant information from the vehicle embedded data through data cleaning processing. Based on the timestamp, the vehicle embedded data within the time window [-15s, 5s] is aligned to construct a time-series visualization model of the hazardous scenario, reconstruct the vehicle state change process, and generate scenario reconstruction data; By invoking a preset risk assessment algorithm and combining the scene reconstruction data, the risk level is calculated, potential safety hazards are identified, and a risk assessment result is generated. Based on the risk assessment results, the problem module is located using a root cause analysis algorithm; For the identified problematic modules, generate corresponding software security remediation solutions; The security repair solution is packaged into an OTA upgrade package, pushed to the target vehicle, and the upgrade progress and success rate of the corresponding software are monitored.

[0009] Furthermore, the process of aligning the vehicle's embedded data within the time window [-15s, 5s] based on timestamps to construct a time-series visualization model of the hazardous scenario, reconstructing the vehicle's state change process, and generating scenario reconstruction data includes the following steps: Using the timestamp of the triggering time as a reference point, the time coordinate system of the vehicle embedded data is unified; Based on the time coordinate system, align the vehicle embedded data within the time window [-15s, 5s]; The integrated and aligned vehicle embedded data is used to define the temporal association rules between the data and construct the temporal visualization model containing a three-dimensional spatiotemporal coordinate system. Using the time-series visualization model, key state nodes before and after the triggering time are extracted, the change process of the whole vehicle embedded data is reproduced, the interaction process between the vehicle and the environment under the hazardous scenario is restored, and the initial scenario restoration data is generated. The initial scene reconstruction data is subjected to integrity verification. Missing key data fragments are supplemented by interpolation algorithm to generate the scene reconstruction data in a standardized format, and the hazardous scene type and trigger timestamp are associated and stored.

[0010] Furthermore, the risk assessment algorithm includes the FMEA (Failure Mode and Effects Analysis) algorithm; the root cause analysis algorithm includes, for example, fault tree analysis, fishbone diagram analysis, Bayesian network, or knowledge graph-based analysis algorithms.

[0011] On the other hand, this application provides a hazardous scene detection system for autonomous driving, including a rule setting module, a signal acquisition module, a hazardous scene detection module, and a safety repair module; The rule setting module is configured to set multiple hazard scene detection rules, each type of rule corresponding to at least one trigger condition and a monitoring signal. The signal collection module is configured to continuously collect the vehicle signal during automatic driving. The hazard scene detection module is configured to compare the vehicle signal with the trigger condition of the hazard scene detection rule, determine whether the trigger condition is met, when the trigger condition is met, collect the vehicle buried point data in the time period around the trigger time according to a preset time window, and upload to the cloud. The safety repair module is configured to perform scene restoration, risk assessment and problem analysis based on the vehicle buried point data in the cloud, generate a safety repair scheme, and upgrade the vehicle software through OTA.

[0012] In another aspect, the application provides an electronic device comprising a processor and a memory; the memory stores a computer program, and the processor executes the computer program to implement the hazard scene detection method steps of the automatic driving according to any one of claims 1 to 7.

[0013] In another aspect, the application provides a vehicle integrated with the hazard scene detection system of the automatic driving according to claim 8.

[0014] The application provides a hazard scene detection method for automatic driving, which realizes accurate identification and response to key safety events by establishing a multi-dimensional hazard scene detection rule system. During the operation of automatic driving, the system continuously collects and compares the vehicle state signal with the preset trigger condition in real time. Once any hazard scene condition is met, the vehicle buried point data in the preset time window is collected and uploaded to the cloud. In the cloud, the collected data is used for accident scene restoration, system risk assessment and fault root cause analysis, and then a targeted safety repair scheme is generated. The vehicle software is remotely upgraded through OTA, realizing closed-loop processing and continuous optimization of safety problems, effectively improving the safety, traceability and remote operation and maintenance capability of the automatic driving system. The application also provides a corresponding system, electronic device and vehicle. The beneficial effects of the system, electronic device and vehicle are similar to those of the method, which will not be repeated here.

[0015] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0017] Figure 1 This is a flowchart of the hazard scene detection method for autonomous driving provided in this application; Figure 2 This is a structural diagram of the hazard scene detection system for autonomous driving provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] With the rapid development of autonomous driving technology, vehicles are becoming increasingly intelligent and automated, especially in Level 3 and higher autonomous driving systems, where vehicles can completely take over driving tasks under specific conditions. However, as system complexity increases, ensuring safety, traceability, and liability determination after accidents during autonomous driving has become a significant challenge for the industry. Currently, although many autonomous vehicles are equipped with basic data recording functions, such as Event Data Recorders (EDRs) or black box systems, these systems still have many limitations in practical applications and cannot meet the refined, intelligent, and systematic safety monitoring requirements of advanced autonomous driving.

[0023] Currently, the mainstream technologies in the field of autonomous driving mainly include the following categories: Traditional vehicles are generally equipped with EDR (Electronic Data Recording) devices to record key parameters such as vehicle speed, braking status, and seatbelt usage during a collision. These systems are typically only activated after a serious collision, and the recorded data dimensions are limited, primarily used by traffic police or insurance companies to determine liability after an accident. Some smart cars are equipped with dashcams or surround-view camera systems to continuously record video streams of the vehicle's surroundings. While these systems can provide visual information, they involve large amounts of data, high storage costs, and lack deep integration with vehicle control signals, making it impossible to achieve intelligent identification and selective recording of specific "hazardous scenarios."

[0024] Furthermore, modern intelligent connected vehicles generally possess remote communication capabilities, enabling them to upload vehicle status information to the cloud via an in-vehicle T-Box and support software updates via over-the-air (OTA). However, these systems are primarily used for routine status reporting and function iteration, lacking proactive triggering mechanisms for safety events and refined data collection strategies. Some Advanced Driver Assistance Systems (ADAS), such as AEB (Automatic Emergency Braking) and LDW (Lane Departure Warning), possess certain active safety capabilities, issuing warnings or taking intervention measures when potential risks are detected. However, these systems are mostly independent functional modules, lacking coordination among themselves, and their triggering does not necessarily accompany a complete vehicle data recording and uploading process. Moreover, during the development and testing phases of autonomous driving algorithms, companies typically deploy high-frequency, full-volume data collection systems to collect data on various driving scenarios to optimize models. However, due to cost, bandwidth, and privacy considerations, such systems are often significantly simplified or disabled in mass-produced vehicles, failing to achieve long-term, routine, and effective monitoring.

[0025] While the aforementioned technologies have improved vehicle safety and intelligence to some extent, they still reveal several shortcomings when dealing with complex "hazardous scenarios" in advanced autonomous driving. Most existing technologies respond to single events (such as collisions or sudden braking), lacking a systematic approach to identifying and classifying various potential risk scenarios during autonomous driving operation. For example, situations such as the driver's failure to respond promptly to a system takeover request, exiting the ODD (Operation Design Domain) due to weather conditions, or the system triggering the Minimum Risk Management (MRM) strategy to pull over, while not resulting in direct accidents, still represent significant safety boundary breaches. Traditional systems lack the ability to identify such "soft" hazard scenarios, leading to the omission of crucial data and impacting subsequent analysis and improvement.

[0026] Furthermore, most existing systems either collect data continuously and in full (which is costly and wasteful of resources) or only begin recording after extreme events (such as airbag deployment) occur (resulting in information lag). This leads to the inability to obtain complete contextual data before and after the event in many intermediate or progressive failure scenarios. For example, when AEB is triggered in autonomous driving mode but no collision occurs, such "near-accident" data is easily lost and difficult to use for algorithm optimization if there is no dedicated mechanism to record the process.

[0027] Traditional EDR or DVR systems collect data with limited dimensions, often only covering vehicle dynamics parameters or local video clips. They lack the synchronous collection and fusion of multi-source heterogeneous data, such as the internal state of the autonomous driving system (e.g., perception results, decision logic, control commands), driver state (e.g., attention, takeover readiness), and environmental information (e.g., lighting, weather, and traffic participant behavior). This makes it difficult to reconstruct the complete interaction process after an accident, limiting the accuracy of root cause analysis.

[0028] Even though some vehicles have data upload capabilities, these are mostly passive reporting or periodic transmissions, lacking a real-time upload mechanism based on event triggers. More importantly, existing systems generally lack a complete closed-loop capability from "data collection → cloud analysis → problem localization → repair solution generation → OTA upgrade." This means that even if a problem is discovered, it cannot be quickly fed back to the vehicle for systemic repair, leading to the recurrence of similar risks.

[0029] With the implementation of Level 3 and above autonomous driving, legal responsibility is gradually shifting from drivers to OEMs or system suppliers. Therefore, auditable and traceable safety monitoring capabilities are essential. However, existing systems have shortcomings in data integrity, time synchronization, and tamper resistance, making it difficult to meet the stringent requirements of future regulations (such as UN-R157's type certification requirements for automated lane keeping systems) regarding "autonomous driving event recording."

[0030] During data collection and uploading, ensuring effective security monitoring while respecting user privacy and complying with data protection regulations such as GDPR and CCPA is a major technological challenge. Existing systems often adopt extreme strategies of "all data collection and transmission" or "no data collection and no transmission," lacking intelligent control capabilities based on user authorization, scenario sensitivity, and data anonymization mechanisms.

[0031] In summary, existing technologies have significant shortcomings in monitoring and responding to hazardous scenarios in autonomous driving: they lack both the ability to systematically model and identify diverse hazardous scenarios and the ability to achieve accurate, efficient, and closed-loop data collection and analysis mechanisms. Especially when facing challenges such as the transfer of responsibility, frequent long-tail problems, and dramatically increased system complexity brought about by L3 and above autonomous driving, traditional solutions are inadequate. Therefore, there is an urgent need for a comprehensive safety monitoring solution that can proactively identify multiple hazardous scenarios, accurately trigger data recording, comprehensively collect multi-dimensional information, and achieve problem analysis and OTA (Over-The-Air) repair via the cloud.

[0032] Against this backdrop, this application proposes a method, system, electronic device, and vehicle for detecting hazardous scenarios in autonomous driving, constructing an intelligent safety monitoring system covering the entire chain of "perception-judgment-recording-uploading-analysis-repair," thereby improving the reliability, interpretability, and continuous evolution capability of autonomous driving systems.

[0033] First, the method for detecting hazardous scenarios in autonomous driving provided in this application will be described in detail below with reference to the accompanying drawings.

[0034] Reference Figure 1 The implementation process of the hazard scene detection method for autonomous driving provided in this application embodiment includes, but is not limited to, the following steps.

[0035] Step S110: Set multiple hazard scenario detection rules, with each rule corresponding to at least one triggering condition and monitoring signal.

[0036] Among them, the detection rules for various hazard scenarios include driver takeover rules, emergency system exit rules, accident rules, exit operation design domain rules, and minimum risk rules.

[0037] In step S110, a systematic, structured, and comprehensive safety monitoring framework is constructed. By pre-defining five key hazard scenario detection rules—namely, driver takeover rules, emergency system exit rules, accident rules, exit operation design domain (ODD) rules, and minimum risk strategy rules—this application achieves accurate classification and identification of various potential safety risks during autonomous driving operation. Each rule corresponds to specific triggering conditions and vehicle signals that need to be monitored, such as steering wheel speed exceeding a threshold, AEB activation, airbag deployment, weather not meeting ODD requirements, or the system issuing the highest-level alarm. This multi-dimensional rule setting not only covers the entire spectrum of risk events from "soft anomalies" to "hard failures," but also ensures that the system can respond promptly at different levels, avoiding missed or false judgments due to a single judgment standard. In addition, this rule system has good scalability and can be continuously iterated and optimized based on actual operating data in the future, adding new scenario types to continuously improve the safety boundary recognition capability of the autonomous driving system.

[0038] Step S120: During the autonomous driving process, continuously collect vehicle signals.

[0039] In step S120, after the vehicle enters autonomous driving mode, the system needs to collect various status signals of the entire vehicle in real time and continuously, including but not limited to L3 function activation status, steering angle request, perception module output, driver monitoring status, environmental sensor data (such as cameras and radar), vehicle dynamic parameters, and the operating status of active / passive safety systems. These signals constitute the original input data source for determining whether a hazardous scenario has occurred. The significance of continuous collection lies in ensuring the continuity of monitoring and the integrity of data, especially in the critical few seconds before an event occurs, where any slight change in state may be a precursor to system anomalies. Through a high-frequency, low-latency data collection mechanism, the system can capture instantaneous abnormal behaviors, providing accurate and reliable information support for subsequent rule comparisons, thereby achieving a closed-loop monitoring capability of "early warning, in-process recording, and post-event traceability".

[0040] Step S130: Compare the vehicle signal with the trigger conditions of the hazard scene detection rules to determine whether the trigger conditions are met.

[0041] In step S130, the collected real-time status signals are logically matched with a preset rule base to determine whether a certain type of hazardous scenario has occurred. This process is essentially a real-time rule engine execution process, where the system checks whether the triggering conditions of each rule are met. For example, if the steering wheel speed suddenly exceeds 500° / s while the L3 function is activated, it is determined that a "driver takeover" scenario has been triggered; if the AEB system is activated in autonomous driving mode, it is determined that an "emergency system exit" scenario has been established. This condition-based judgment mechanism has high determinism and interpretability, capable of completing complex logical judgments within milliseconds, ensuring timely response. More importantly, it achieves "precise filtering" of truly noteworthy safety events from massive amounts of operational data, avoiding the resource waste caused by indiscriminate full recording, and improving the targeting and efficiency of data collection.

[0042] Step S140: When the triggering condition is met, collect the vehicle embedded data within a time period near the triggering time according to the preset time window, and upload it to the cloud.

[0043] In step S140, once the system determines that a hazardous scenario has been triggered, it immediately initiates a data acquisition mechanism. Using the timestamp of the trigger moment as the baseline, it comprehensively collects vehicle-wide embedded data within a preset time window. This time window design fully considers both the pre-event causal analysis (such as changes in driver behavior or environmental deterioration) and the post-event system response process (such as braking or pulling over), ensuring data integrity and contextual consistency. The collected data is extensive, covering multiple dimensions including basic vehicle information, dynamic status, driver operations, perception information, internal operating logs of the autonomous driving system, HMI display content, ambient light, and new energy SOC values, forming a multi-source fusion "event snapshot." Subsequently, with user authorization, the system packages this data and uploads it to the cloud server via the vehicle communication module, providing a high-quality data foundation for subsequent in-depth analysis, truly achieving "critical events must be recorded, and important data must be transmitted."

[0044] In step S150, on the cloud, based on the vehicle's embedded data, scene reconstruction, risk assessment and problem analysis are performed to generate a safety repair solution, and the vehicle software is upgraded via OTA.

[0045] In step S150, after receiving the uploaded data points on the cloud platform, the data is first cleaned, aligned, and its integrity is verified to ensure data quality. Then, using timestamp alignment technology, a three-dimensional time-series visualization model incorporating time, space, and system state is constructed to realistically recreate the interaction process between the vehicle and its environment before and after the accident, helping engineers intuitively understand the full picture of the event. Based on this, combined with root cause analysis algorithms such as FMEA (Failure Mode and Effects Analysis) and fishbone diagrams, the system can automatically identify potential safety hazards, assess risk levels, and pinpoint the root cause of the problem (such as perception misjudgment, decision-making logic defects, or control delays). Finally, targeted software repair solutions are generated for the identified problems and packaged into OTA upgrade packages for push to relevant vehicles, enabling remote repair and system optimization. This process not only significantly shortens the problem response cycle but also forms a positive iterative mechanism of "problem discovery—problem analysis—problem resolution—problem prevention," continuously improving the safety, robustness, and user experience of the autonomous driving system.

[0046] In some embodiments of this application, the triggering conditions for the driver takeover rule include, but are not limited to: the autonomous driving function is activated and the steering wheel speed exceeds a preset speed threshold.

[0047] The design of this trigger condition has a clear technical purpose and practical significance. Its core function is to accurately identify whether the driver actively and promptly intervenes manually during the operation of the autonomous driving system, i.e., the moment of "takeover" behavior. When the vehicle is in L3 or higher level autonomous driving mode, the system is theoretically responsible for driving tasks, but according to regulations and safety policies, the driver still needs to remain alert and prepared to take over the vehicle when requested by the system. However, in actual operation, there may be situations where the driver takes over the steering wheel without being explicitly prompted, such as due to distrust of the system's behavior or perception of potential risks leading to premature intervention. By monitoring the steering wheel speed—a sensitive and direct physical signal—and setting a reasonable threshold, normal fine-tuning can be effectively distinguished from substantial manual takeover actions, avoiding false triggers caused by slight disturbances.

[0048] Once this condition is met, the system can determine that a critical safety event, "driver takeover," has occurred, thereby activating subsequent data collection and uploading mechanisms to ensure a complete record of system state changes, environmental information, and human-machine interaction processes before and after the takeover. This is of great significance for analyzing the availability of autonomous driving systems, human-machine collaboration efficiency, and responsibility allocation, and also provides valuable data support for optimizing takeover alert strategies and improving user experience.

[0049] In some embodiments of this application, the triggering conditions for the emergency system exit rule include, but are not limited to: the autonomous driving function is activated, and the vehicle's active safety system (such as automatic emergency braking AEB, forward collision warning FCW, lane keeping assist LKA, etc.) is triggered and is activated.

[0050] The design of this trigger condition aims to identify whether, during autonomous driving mode operation, the vehicle senses an impending collision or significant deviation risk, prompting the active safety system to intervene, thus forcing the autonomous driving function to disengage or degrade. Although no actual collision occurs in this scenario, it represents a high-risk "quasi-accident" situation, reflecting that the autonomous driving system may not be able to independently complete safe driving tasks under certain conditions and must rely on traditional ADAS functions for backup protection. By performing a logical AND operation on the "autonomous driving activation" and "active safety system activation" signals, such critical safety events can be accurately identified, avoiding misjudging AEB triggering in non-autonomous driving mode as a hazardous scenario.

[0051] Once this condition is met, the system determines that an "emergency system exit" event has occurred and immediately initiates the collection and uploading of vehicle data within a preset time window. This not only helps to reconstruct the system's decision-making process before emergency intervention (such as perception misjudgment, unreasonable path planning, etc.), but also provides high-quality data support for subsequent analysis of the collaborative logic between autonomous driving and active safety systems, optimization of control priority strategies, and improvement of system redundancy and safety. It is an important technical means to achieve continuous iteration of autonomous driving systems and convergence of long-tail problems.

[0052] In some embodiments of this application, the triggering conditions for the accident rule include, but are not limited to: the autonomous driving function is activated and the vehicle's passive safety system (such as the airbag control unit ACU) issues a collision signal.

[0053] The core function of this triggering condition is to accurately identify whether a real physical collision event has occurred in autonomous driving mode, thereby activating the highest-priority safety response and data recording mechanism. Passive safety systems (especially airbag controllers) monitor changes in vehicle deceleration in real time through devices such as acceleration sensors. When a severe impact exceeding a preset threshold is detected, it determines that a collision has occurred and triggers the airbag ignition command, while simultaneously outputting a "collision signal." This signal has extremely high reliability and authority, and is a key basis for determining whether an actual accident has occurred. By performing a logical AND operation between this signal and the "autonomous driving function activated" state, ordinary traffic accidents in non-autonomous driving states can be effectively excluded, ensuring that only collisions occurring while the autonomous driving system is responsible for driving tasks are considered "accident" type hazards as defined in this application.

[0054] Once this condition is met, the system immediately determines that a serious safety incident has occurred and activates the emergency data collection process for the entire vehicle. This process fully records multi-dimensional data within a preset time window before and after the collision, including vehicle dynamics, perception results, decision logs, driver status, and environmental video. This data is crucial not only for reconstructing the entire accident process and analyzing the autonomous driving system's response behavior before the collision (such as whether it took timely deceleration or obstacle avoidance measures), but also for providing tamper-proof objective evidence for subsequent liability determination, insurance claims, product liability tracing, and system safety improvements. It is a key component in building a closed-loop management system for autonomous driving safety.

[0055] In some embodiments of this application, the triggering conditions for exiting the Operational Design Domain (ODD) rule include, but are not limited to, environmental conditions exceeding the design operating range defined by the autonomous driving system.

[0056] The core function of this trigger condition is to identify whether the autonomous driving system has encountered external environmental changes that exceed its functional capabilities during actual operation, thereby actively or passively exiting the autonomous driving mode to ensure vehicle operation safety. ODD is a set of operating conditions that are explicitly defined during the design of the autonomous driving system, which usually includes weather conditions (such as heavy rain, heavy fog, heavy snowfall), lighting conditions (such as nighttime, backlight), road type (such as highways, urban roads), traffic conditions, and geographical range (such as the area covered by high-precision maps).

[0057] When a vehicle is operating in autonomous driving mode, if the sensors detect that the current environmental parameters (such as visibility below a threshold, severely slippery road surface, lack of lane markings, or GPS signal loss) no longer meet the preset ODD requirements, it is considered an "ODD exit" event has occurred. This process may be initiated automatically by the system as a degradation request, or it may result in a functional interruption due to perception failure. By comparing environmental perception signals (such as camera fog detection, abnormal radar echoes, and meteorological information access) with the ODD boundary in real time, the system can predict or promptly identify deteriorating trends in the operating environment.

[0058] Once environmental conditions are determined to exceed the design domain, even without emergency braking or a collision, it should be considered a significant "soft hazard scenario." This is because it means the system can no longer guarantee continuous safe operation and must rely on driver takeover or the implementation of a minimum risk strategy (MRM). Triggering a data acquisition and uploading mechanism at this point helps to fully record the perception degradation process, decision logic evolution, and human-machine interaction state before the system exits. This provides crucial data support for subsequent optimization of ODD boundary definitions, improvement of environmental adaptability, refinement of degradation warning strategies, and verification of system fail-safe mechanisms. It is an important technical guarantee for achieving the robustness and reliability of autonomous driving systems.

[0059] In some embodiments of this application, the triggering conditions for the minimum risk rule include, but are not limited to: the autonomous driving function is activated and the system issues the highest level alarm signal (e.g., a level 4 alarm of the NOP function).

[0060] The purpose of this trigger condition is to identify whether the autonomous driving system has entered a seriously abnormal or unsustainable state during operation. Immediate measures must be taken to guide the vehicle to the Minimum Risk Condition (MRC) to ensure the safety of passengers and other road users. The highest level alarm typically indicates that the system has detected a critical situation such as a major malfunction, perception fusion failure, loss of localization, redundant system anomaly, or exceeding the Operating Design Domain (ODD) and is unable to continue operating safely. The system can no longer complete the driving task on its own and must activate emergency response strategies, such as automatic deceleration, lane changing, and pulling over.

[0061] By logically ANDing the conditions of "autonomous driving function activation" and "highest level alarm," the system can accurately distinguish between normal exit and emergency exit scenarios, avoiding misjudging driver-initiated function shutdown or low-level warnings as high-risk events. Once this condition is met, the system determines that the critical safety event of "minimum risk strategy execution" has occurred, and immediately triggers the emergency collection and uploading process of vehicle-wide embedded data.

[0062] The recorded data covers system state evolution before and after the alarm, fault codes (DTCs), perception inputs, decision outputs, vehicle dynamics, and driver response behavior. This data is crucial for analyzing system failure modes, evaluating the effectiveness of minimum risk strategies, optimizing alarm classification mechanisms, and improving overall vehicle functional safety (such as meeting ISO 26262 ASIL requirements). Furthermore, this mechanism provides OEMs with the ability to remotely monitor vehicle health, respond promptly to user requests, and implement proactive service interventions, making it an indispensable part of building a closed-loop safety management system for autonomous driving.

[0063] In some embodiments of this application, the vehicle embedded data includes basic vehicle information, vehicle status and dynamic information, driver status and operation information, environmental perception-related information, autonomous driving system operation information, instrument and HMI display information, and new energy information.

[0064] The vehicle basic information includes the vehicle identification number and location; vehicle status and dynamic information includes ignition information; driver status and operation information includes driver status monitoring information; environmental perception information includes camera-related video and light intensity information; autonomous driving system operation information includes the requested steering angle; instrument panel and HMI display information includes DTC diagnostic fault code information; and new energy information includes the SOC value at the start and end of charging.

[0065] Specifically, basic vehicle information includes the Vehicle Identification Number (VIN) and geolocation information. Its purpose is to uniquely identify the vehicle from which the data originates and to record the precise location at the time of the incident, facilitating vehicle tracking, regional risk analysis, and subsequent service response for OEMs. It is a prerequisite for achieving data traceability.

[0066] Vehicle status and dynamic information includes real-time dynamic parameters such as ignition status, vehicle speed, acceleration, gear position, and turn signal status. These parameters are used to characterize the vehicle's driving behavior before and after the triggering moment, helping to determine whether the vehicle is in a normal driving state, decelerating to the side of the road, or has stopped. This data is the core data for reconstructing the vehicle's motion trajectory and operating logic.

[0067] Driver status and operation information includes information collected by the driver monitoring system (DMS) such as attention status, line of sight, readiness to take over, whether hands are on the steering wheel, and whether pedals are pressed. Its significance lies in assessing the driver's level of participation and responsiveness during human-machine collaboration. In particular, when determining liability, it can clarify whether the driver is distracted, fatigued, or has failed to take over in a timely manner.

[0068] Environmental perception-related information includes front, rear, left, and right video streams recorded by cameras, radar point cloud data, lidar data, and information such as light intensity and weather perception. It is mainly used to reconstruct the real traffic environment around the vehicle and analyze whether the perception system has false detections, missed detections, or judgment delays due to poor lighting, occlusion, or complex traffic participants. It is a key basis for verifying the performance of autonomous driving perception modules.

[0069] The operational information of an autonomous driving system involves internal decision-making and control commands, such as requested steering angles, target vehicle speeds, acceleration planning, path planning results, target recognition and tracking lists, and ODD state judgments. This data reveals the thought process of the system's "brain," helps to deeply analyze whether there are defects in the decision-making logic, and whether the control execution is reasonable. It is the core input for root cause analysis and algorithm optimization.

[0070] The instrument panel and HMI display information includes the instrument panel display content, the central control screen interaction status, voice prompt records, and fault-related information such as DTC (diagnostic fault codes). It can reflect whether the system promptly issues a takeover request to the driver, whether the alarm level is appropriate, and whether the human-machine interaction is clear and effective. At the same time, DTC information can directly point to the abnormality of specific components or software modules, providing clues for fault location.

[0071] Information related to new energy focuses on the energy management status of electric vehicles, such as the battery SOC (State of Charge) value, battery temperature, and charging power at the start and end of charging. Although this type of data does not directly participate in driving decisions, it has important analytical value in specific scenarios (such as low battery causing system degradation or abnormalities occurring during charging). It can also be used to assess the impact of energy management strategies on the stability of autonomous driving functions.

[0072] In summary, the collaborative collection of various types of vehicle-wide embedded data constructs a panoramic data view covering "human-vehicle-environment-system," ensuring that after a hazardous scenario is triggered, the entire picture of the event can be reconstructed from multiple angles and in depth. This supports accurate scene reconstruction, risk assessment, and problem diagnosis in the cloud, ultimately achieving a closed-loop technology for safety repair via OTA, significantly improving the safety, interpretability, and continuous evolution capabilities of the autonomous driving system.

[0073] In some embodiments of this application, in step S140, when the triggering condition is met, the process of collecting vehicle embedded data within a time period near the triggering time according to a preset time window and uploading it to the cloud includes, but is not limited to, the following steps.

[0074] Step S210: Using the timestamp of the trigger moment as the origin, mark it as the reference time point for data collection.

[0075] In step S210, when the system determines through the rule engine that the triggering conditions for a certain hazardous scenario are met (such as driver takeover, AEB activation, airbag deployment, etc.), it immediately records the precise time point of the event and uses it as the "time origin" or "reference timestamp" for the entire data acquisition process. This operation ensures that all subsequently collected data can be organized and time-aligned with this time point as a reference. Since autonomous driving systems involve multiple ECUs, sensors, and software modules, their data output suffers from asynchronous timestamps; therefore, establishing a unified time reference is crucial. This reference time point not only defines the acquisition window but also serves as the core anchor point for reconstructing the event sequence and constructing the timeline during cloud data analysis, ensuring data temporal consistency and traceability, and providing precise time coordinate support for subsequent scenario reconstruction and root cause analysis.

[0076] Step S220: Based on the reference time point, define the data acquisition time window as [-15s, 5s] of the trigger time, covering the whole vehicle embedded data from 15 seconds before the trigger to 5 seconds after the trigger.

[0077] In step S220, the selection of the asymmetric time window [-15s, 5s] has clear technical considerations: the forward 15 seconds are used to capture "trigger information" such as the system's operating status, environmental change trends, and driver behavior evolution before the event occurs, such as whether the perception system has made a misjudgment, whether the driver has begun to prepare to take over, and whether road conditions have deteriorated; the backward 5 seconds are used to record the system's response behavior after the event is triggered, such as emergency braking execution, steering control, HMI alarm prompts, function degradation or exit processes, etc. This design can completely reconstruct the entire process of "event cause - critical state - system response - consequence evolution", which is especially suitable for analyzing progressive failure problems in long-tail scenarios. In addition, this time window takes into account both data integrity and storage / transmission efficiency, avoids the resource waste caused by full-time recording, and realizes a data acquisition strategy of "precise capture and highlighting key points".

[0078] Step S230: Real-time storage of vehicle embedded data within the time window.

[0079] In step S230, data acquisition is performed, continuously and stably collecting and caching various types of vehicle-wide embedded data within a defined time window. The system reads signals from various ECUs (Electronic Control Units) in real time via the vehicle data bus (such as CAN, CAN FD, Ethernet) and stores them in a formatted manner according to a preset data structure. This data includes, but is not limited to, vehicle dynamic information (vehicle speed, acceleration), driver operation and status, environmental perception results, internal decision logs of the autonomous driving system, HMI interaction content, and SOC changes of new energy vehicles. Data storage typically employs a ring buffer mechanism, continuously caching the data of the most recent 15 seconds before triggering. Once the triggering condition is met, the buffer is immediately frozen, and data for the next 5 seconds continues to be recorded, ensuring that historical information before the trigger is not lost.

[0080] This mechanism requires the system to have highly reliable, low-latency data acquisition capabilities to prevent data interruptions caused by packet loss or latency, which could affect the accuracy of subsequent analysis. Simultaneously, the data must be strictly sorted by timestamp to lay the foundation for multi-source data alignment in the cloud.

[0081] Step S240: When the amount of collected vehicle embedded data accumulates to a preset data volume threshold, the cached vehicle embedded data is packaged and uploaded to the cloud within the scope authorized by the user.

[0082] In step S240, due to the large amount of vehicle-wide embedded data (especially when it includes video or high-frequency signals), the system does not upload the data immediately after collection. Instead, it first packages and compresses the data locally and sets a "data volume threshold" as the upload trigger condition. This threshold can be dynamically adjusted according to network conditions (such as 4G / 5G signal strength and Wi-Fi connection status) to avoid forced uploads that could lead to failure or affect vehicle communication performance when the network is unstable. When the data packet reaches the threshold or the collection window ends, the system encrypts and packages the data and, with user authorization, uploads it to the cloud server via the T-Box or the vehicle communication module. The upload process supports breakpoint resumption and integrity verification to ensure data security and reliability. Furthermore, the setting of the "user authorization scope" reflects the emphasis on privacy protection, complies with data compliance requirements such as GDPR and CCPA, and allows users to choose whether to participate in data sharing programs, enhancing the system's legitimacy and user trust.

[0083] In step S250, after receiving the vehicle's embedded data, the cloud associates and stores it with the corresponding hazard scenario type and trigger timestamp to support subsequent scenario reconstruction and security analysis.

[0084] In step S250, after the cloud server successfully receives the uploaded data packet, it first decrypts, decompresses, and verifies its integrity. Then, it deeply binds the data with its metadata, including: the type of hazard scenario triggered (such as "driver takeover" or "emergency system exit"), the trigger timestamp, vehicle VIN, geographical location, and ODD status. This structured storage method makes each event record an independent, searchable, and analyzable "safety event archive." Based on this, the cloud platform can utilize big data analytics engines, visualization tools, and AI algorithms to perform three-dimensional reconstruction of the event, restoring the vehicle's driving trajectory, driver behavior, surrounding traffic flow, and system decision-making process. This allows for risk assessment, root cause analysis (RCA), and FMEA failure analysis. More importantly, this archived data can also be used to train autonomous driving models, optimize ODD boundaries, improve human-machine interaction design, and ultimately generate OTA repair solutions, driving continuous system evolution. Therefore, this step is not only the end point of data storage but also the starting point for intelligent analysis and safety improvement, truly achieving closed-loop management "from data to insight, from insight to action."

[0085] In some embodiments of this application, in step S150, the process of performing scene restoration, risk assessment and problem analysis based on the whole vehicle embedded data in the cloud, generating a safety repair solution, and upgrading the vehicle software via OTA includes, but is not limited to, the following steps.

[0086] In step S310, after receiving the vehicle data embedding data, the cloud verifies the data integrity and timing consistency of the vehicle data embedding data, and removes outliers and redundant information from the vehicle data embedding data through data cleaning processing.

[0087] In step S310, after the cloud server receives the vehicle-mounted data packet uploaded from the vehicle, it first performs an integrity check to ensure that no packet loss, truncation, or encryption errors occurred during transmission. This includes checking the data header, checksum, and file size for consistency. Simultaneously, it verifies timing consistency, confirming the continuity of timestamps for all collected signals and preventing data misalignment due to vehicle clock asynchrony or communication delays. Based on this, the system performs data cleaning, identifying and removing outliers (such as sensor mutations or values ​​outside the physical range), duplicate records, or irrelevant redundant information (such as periodic reports of non-critical states). Missing data is processed using interpolation or tagging. This process ensures the dataset upon which subsequent analysis relies has high reliability, high consistency, and high availability, laying a solid foundation for accurate scene reconstruction and risk assessment, and avoiding misjudgments caused by "garbage in, garbage out."

[0088] Step S320: Based on the timestamp-aligned vehicle embedded data within the time window [-15s, 5s], construct a time-series visualization model of the hazardous scenario, reconstruct the vehicle state change process, and generate scenario reconstruction data.

[0089] In step S320, after data cleaning, the system uses the timestamp of the trigger moment as the origin (t=0) to perform high-precision time alignment on multi-source heterogeneous data (such as CAN signals, video frames, DMS images, radar target lists, HMI logs, etc.) from different ECUs, different communication buses, and different sampling frequencies. Through alignment algorithms (such as linear interpolation, spline fitting, or event-driven synchronization), it ensures that all signals have a unified time reference within the [-15s, 5s] time window, thereby constructing a time-continuous and logically consistent "digital twin" timing model. This model can dynamically display the complete behavioral trajectory of the vehicle before and after the event, including speed change curves, steering angle requests and actual responses, driver attention drift, forward target recognition status, system alarm level evolution, and instrument display content switching. Through visualization tools (such as timeline players and 3D simulation interfaces), engineers can intuitively "replay" the entire hazardous scenario, as if watching a "driving black box video" with internal system status annotations. This ability to recreate scenarios greatly improves the efficiency and accuracy of accident analysis and is a key support for understanding complex human-computer interactions and system behavior.

[0090] Step S330: By calling a preset risk assessment algorithm and combining it with scene reconstruction data, the risk level is calculated, potential safety hazards are identified, and a risk assessment result is generated.

[0091] In step S330, the system invokes a preset risk assessment algorithm, comprehensively considering multiple dimensions of factors, including: vehicle dynamic parameters (such as deceleration and lateral offset), environmental complexity (such as traffic density and weather), system response time, driver takeover delay, and whether the collision threshold (TTC, Time to Collision) is approaching. The algorithm performs weighted calculations on these indicators, outputs a quantified risk level (such as low, medium, high, or very high), and marks key risk nodes.

[0092] For example, if the system detects an obstacle ahead 3 seconds before AEB is triggered but fails to decelerate in time, it is classified as a high-risk "decision delay" scenario; if the driver does not respond within 8 seconds of the system requesting takeover, it is marked as a "human-machine collaboration failure" risk. This assessment result is not only used for event classification and prioritization but also serves as input for subsequent root cause analysis and remediation strategy development, achieving a leap from "qualitative description" to "quantitative assessment."

[0093] Step S340: Based on the risk assessment results, locate the problem module using the root cause analysis algorithm.

[0094] In step S340, after obtaining the risk assessment results, the system combines the scene reconstruction data and runs a root cause analysis (RCA) algorithm to systematically investigate the technical aspects that may lead to the risk event. The analysis process covers all modules of the autonomous driving system, including the perception layer (whether the camera / radar misidentifies objects), the decision-making layer (whether the path planning is reasonable), the control layer (whether the actuator response is lagging), the human-machine interaction layer (whether the HMI prompts are clear), and functional safety mechanisms (whether the redundancy design is effective). For example, if the vehicle exits ODD in rainy weather, the system can trace back to specific aspects such as increased camera image blur, decreased target detection confidence, and improper weight adjustment of the fusion algorithm, ultimately pinpointing the fundamental problem of "insufficient perception robustness under adverse weather conditions." This process supports automatic annotation of suspicious signal paths and abnormal behavior patterns, assisting engineers in quickly identifying the source of the fault, significantly shortening the troubleshooting cycle, and improving problem-solving efficiency.

[0095] Step S350: For the identified problematic module, generate a corresponding software security repair solution.

[0096] In step S350, the system automatically generates a targeted software repair plan based on the specific modules identified (such as perception algorithms, decision logic, ODD judgment strategies, HMI prompting mechanisms, etc.). This plan may include: updating the training data of the object detection model to improve rainy weather recognition capabilities, optimizing the AEB trigger threshold to reduce false triggers, improving the takeover alert strategy (such as adding dual sound and vibration prompts), and adjusting the execution path of the minimum risk strategy (MRM), etc. The repair plan can be generated based on a preset rule base, historical repair case matching, or intelligently generated using an AI recommendation engine.

[0097] Simultaneously, the system assesses the impact of the remediation plan to ensure no new side effects are introduced, and generates detailed change documentation, including modifications, expected effects, and testing and verification suggestions. This process automates the transition from "problem identification" to "solution proposal," providing OEM R&D teams with clear technical guidance and accelerating problem closure.

[0098] Step S360: Package the security repair solution into an OTA upgrade package, push it to the target vehicle, and monitor the upgrade progress and success rate of the corresponding software.

[0099] In step S360, the system compiles and packages the generated security repair solution into an OTA upgrade package that conforms to the vehicle ECU communication protocol, and pushes it to affected vehicles (such as specific vehicle models, specific software versions, or vehicles that have experienced similar incidents) through the cloud management platform. The push process supports canary releases, version control, and upgrade strategy configuration (such as upgrading only while charging), and fully considers network security mechanisms (such as signature verification and tamper-proof encryption). The vehicle completes the download, verification, and flashing operations under user authorization and security conditions.

[0100] Meanwhile, the cloud continuously monitors the execution status of the upgrade task, including download progress, installation success rate, reasons for failure (such as power outages or communication interruptions), and post-upgrade system stability, generating statistical reports. For vehicles where the upgrade fails, the system can automatically retry or notify after-sales service for intervention. Through this mechanism, OEMs can quickly fix potential safety hazards and improve the overall vehicle safety level without requiring users to visit a dealership. Simultaneously, it accumulates a large amount of real-world verification data, feeding back into the positive development of next-generation autonomous driving systems, truly achieving the intelligent evolution goal of "getting smarter the more you drive and safer the more you use it."

[0101] In some embodiments of this application, after the software upgrade is completed, the repair effect is verified by collecting vehicle operation data of the target vehicle. If the risk is not eliminated, the above steps are repeated until the risk is eliminated.

[0102] Specifically, after the software is upgraded via OTA, the system does not simply assume that the problem has been solved. Instead, it introduces a "repair effect verification" mechanism, which continuously collects vehicle operation data of the target vehicle in actual operation to conduct closed-loop verification of the actual effectiveness of the safety repair solution.

[0103] The purpose of this step is to ensure that the pushed software update truly and effectively resolves the original security vulnerabilities, rather than merely remaining at the theoretical level or introducing new potential risks. Specifically, after the vehicle completes the upgrade, the system will focus on monitoring similar operating conditions (such as similar weather, road types, traffic environments, etc.) in the scenarios where hazards previously occurred, collecting key data such as the vehicle's perception results, decision-making behavior, control response, and human-machine interaction performance in the same or similar scenarios after the upgrade.

[0104] For example, if the original problem was accidental exit from the ODD during rainy weather, the system will focus on the stability of the sensing module, the continuity of the ODD's judgment logic, and whether unnecessary exit events are still frequently triggered under subsequent rainfall conditions. By comparing the data performance before and after the upgrade, the system can quantitatively assess whether risk indicators (such as event frequency, response latency, and number of takeover requests) have significantly decreased. If the analysis results show that the risk has been effectively suppressed, the system is considered to have successfully fixed the problem; if the risk has not been eliminated or has even worsened, the system will automatically trigger a new round of analysis to re-examine whether the root cause judgment is accurate, whether the repair strategy is appropriate, and iteratively generate new optimization solutions, and then push the update again via OTA.

[0105] This continuous iterative mechanism of "monitoring-evaluation-feedback-re-optimization" constructs a dynamic and adaptive safety evolution closed loop, which not only improves the reliability of problem solving, but also enhances the robustness of autonomous driving systems in dealing with long-tail scenarios and complex environments, truly realizing the leap from passive response to proactive evolution.

[0106] In some embodiments of this application, in step S320, the process of constructing a time-series visualization model of the hazardous scenario based on the vehicle embedded data within the time window [-15s, 5s] aligned with the timestamp, and reconstructing the vehicle state change process to generate scenario reconstruction data includes, but is not limited to, the following steps.

[0107] Step S410: Using the timestamp of the triggering moment as the reference point, unify the time coordinate system of the whole vehicle embedded data.

[0108] In step S410, within the autonomous driving system, various signals originate from different electronic control units (ECUs), such as the autonomous driving domain controller, body controller, driver monitoring system (DMS), perception sensors, and human-machine interface (HMI). These modules often operate under independent clock systems, exhibiting slight time deviations or even clock drift. Directly using the raw timestamps for analysis could lead to disordered event sequences or misjudgments of causal relationships. Therefore, it is essential to use the precise timestamp of the hazardous scenario trigger moment as the global reference origin (t=0) and remap all collected vehicle-wide embedded data to this unified time coordinate system.

[0109] This process involves not only time offset correction but also confirmation of clock synchronization mechanisms (such as PTP precision time protocol or GPS timing) to ensure that all signals are aligned with millisecond-level precision. By establishing this unified time reference framework, a solid foundation is laid for subsequent cross-system and cross-bus data alignment and visualization modeling, which is the primary guarantee for achieving realistic and reliable scene reproduction.

[0110] Step S420: Based on the time coordinate system, align the vehicle embedded data within the time window [-15s, 5s].

[0111] In step S420, multidimensional signals distributed across different communication networks (such as CAN, LIN, Ethernet) and sampling frequencies are precisely arranged according to timestamps to form a time-continuous and logically consistent data sequence. Because the update frequencies of various signals differ significantly—for example, vehicle speed may update every 10ms, while HMI display content may refresh every 100ms, video frame rate is 30fps, and DTC fault codes are event-triggered records—the system needs to employ techniques such as interpolation, resampling, or event matching to ensure that all signals are synchronized on the time axis within the [-15s, 5s] time window. For example, for low-frequency signals, intermediate states can be filled in using forward padding or linear interpolation; for high-frequency signals, downsampling is performed to match the analysis granularity. This process ensures that at any time slice (e.g., t=-8.3s), a snapshot of the state of all relevant systems at that moment can be obtained, thereby constructing a complete and seamless event development timeline and avoiding analytical blind spots or misjudgments caused by data asynchrony.

[0112] Step S430: Integrate the aligned vehicle embedded data, define the temporal association rules between the data, and construct a temporal visualization model containing a three-dimensional spatiotemporal coordinate system.

[0113] In step S430, the system semantically integrates the aligned vehicle data points and establishes logical relationships between data items based on preset temporal correlation rules. For example, rules such as "if the driver does not take over within 3 seconds after the system issues a level 4 alarm and the vehicle speed is higher than 60km / h, then the minimum risk strategy will be activated" are encoded as state transition paths in the model.

[0114] Building upon this foundation, a three-dimensional spatiotemporal coordinate system model is constructed, integrating the time dimension (t), spatial dimension (x, y, z), and system state dimension. This model not only includes the vehicle's own trajectory and attitude changes but also overlays surrounding traffic participants (targets identified by the perception system), road structure (high-precision map information), ambient lighting, weather conditions, and other external elements, forming a highly realistic virtual scene. The model supports multi-view playback, key event annotation, and linked display of signal curves, providing engineers with an intuitive and immersive analysis environment, significantly improving the efficiency of understanding complex events.

[0115] Step S440: Through the time-series visualization model, extract the key state nodes before and after the triggering time, reproduce the change process of the whole vehicle embedded data, restore the interaction process between the vehicle and the environment under the hazardous scenario, and generate the initial scene restoration data.

[0116] In step S440, the "dynamic playback and process reconstruction" function is executed using the constructed time-series visualization model. The system automatically identifies and extracts critical state nodes in the event development process, such as: the moment the autonomous driving function is activated, the system first detects an obstacle, issues a takeover request, the steering wheel speed suddenly increases, AEB is triggered, the airbags deploy, and the vehicle stops—time points with clear semantic significance. Around these nodes, the system recreates the evolution of vehicle state, driver behavior, system decision output, and environmental changes frame by frame, forming a traceable and interactive "digital event recording." For example, in a driver takeover event, the model can clearly show the complete chain of "the system requests takeover due to difficulty in identifying the construction area ahead → the dashboard flashes a warning → the driver turns the steering wheel 3 seconds later → autonomous driving exits." This reconstruction not only presents "what happened" but also reveals "why it happened" and "how it evolved." The generated initial scene reconstruction data provides intuitive and structured input material for subsequent risk assessment and root cause analysis.

[0117] Step S450: Perform integrity verification on the initial scene reconstruction data, supplement missing key data fragments through interpolation algorithm, generate scene reconstruction data in a standardized format, and store it in association with the hazardous scene type and trigger timestamp.

[0118] In step S450, due to potential momentary packet loss, signal delay, or temporary sensor failure in vehicle communication, the initial reconstructed data may contain missing or abrupt data. Therefore, the system needs to perform integrity verification, identify abnormal intervals, and use appropriate interpolation algorithms (such as linear interpolation, spline interpolation, or machine learning-based predictive completion) to reasonably fill in key signals (such as vehicle speed, steering angle, and braking pressure) to ensure the smoothness and physical plausibility of the time series. Subsequently, the repaired data is converted into a unified standardized format (such as JSON Schema, ASAM OpenDRIVE / OSI standard) to facilitate sharing and automated processing between different systems. Finally, this standardized scene reconstruction data is strongly correlated with the corresponding hazard scenario type (such as "emergency system exit"), trigger timestamp, vehicle VIN, and other metadata, and stored in a cloud-based event database, forming a searchable, traceable, and reusable safety event archive. This process not only ensures the reliability of the analysis results but also provides high-quality data assets for subsequent big data analysis, model training, and regulatory compliance audits.

[0119] In some embodiments of this application, the risk assessment algorithm includes the FMEA (Failure Mode and Effects Analysis) algorithm.

[0120] FMEA is a systematic and structured reliability engineering analysis method that is widely used in industries with high safety requirements, such as automotive, aerospace, and medical. It is used to identify potential failure modes in products or processes, analyze their impact, and assess their risk priority.

[0121] First, the introduction of the FMEA algorithm makes the risk assessment of autonomous driving systems in hazardous scenarios highly systematic and comprehensive. By constructing an FMEA table, the system can analyze the cause, potential effect, severity, occurrence, and detectability of each possible failure mode (such as perception misidentification, decision delay, control failure, communication interruption, etc.), and calculate the risk priority number (RPN = S × O × D). This multi-dimensional assessment mechanism can objectively quantify the relative importance of different risks, avoid biases caused by subjective judgment, and provide OEMs and R&D teams with a scientific basis for decision-making.

[0122] Secondly, FMEA helps identify "hidden risks" and "long-tail problems" in autonomous driving systems. In actual operation, some malfunctions may not immediately lead to serious accidents, but if they occur repeatedly or are coupled with other factors, they may evolve into major safety hazards. For example, a brief malfunction of a camera in backlight conditions may not cause a collision, but if it is not detected and repaired in time, it may cause false or missed triggering of AEB in certain scenarios. Through FMEA analysis, even if the severity of such events is not high, if they occur frequently or are difficult for the system to detect, their RPN value may still reach the warning threshold, thus being marked as an issue requiring key attention and optimization, enabling proactive management of potential risks.

[0123] In some embodiments of this application, the root cause analysis algorithm includes, for example, fault tree analysis algorithm, fishbone diagram analysis algorithm, Bayesian network, or knowledge graph-based analysis algorithm.

[0124] First, the application of Fault Tree Analysis (FTA) enables the system to decompose hazardous events layer by layer in a top-down logical structure. Taking "unexpected exit of an autonomous driving system on a highway" as an example, FTA can treat this as the top-level event, decomposing it step by step into intermediate events such as "perception failure," "decision anomaly," "actuator failure," and "communication interruption," and further refining it to specific hardware or software modules, such as basic events like "blurred camera image," "fusion algorithm confidence level below the threshold," and "insufficient computing power of the domain controller." By constructing logic gates (such as AND gates and OR gates) to connect each level, the system can clearly display the causal chain between various factors, identify the critical path and minimum cut set that led to the event, and thus determine whether it is caused by a single fault or multiple coupled faults. This structured analysis method is particularly suitable for studying the failure mechanisms of complex systems, helping to identify weak links in system design and improve the robustness of the overall architecture.

[0125] Secondly, the fishbone diagram analysis algorithm (cause-effect graph analysis) systematically identifies various factors that may influence the occurrence of an event from multiple dimensions, including human, machine, environment, management, and law. In the autonomous driving scenario, "human" refers to the driver's state and takeover behavior; "machine" encompasses system modules such as perception, decision-making, and control; "environment" includes external factors such as weather, lighting, and road conditions; "management" involves OTA update strategies and remote monitoring mechanisms; and "law" refers to system logic rules and ODD boundary definitions. By classifying and grouping these factors, the fishbone diagram helps engineers comprehensively examine the problem background and avoid overlooking potential causes. For example, in an AEB false triggering event, the fishbone diagram may reveal the synergistic effect of multiple factors such as "rainy weather reflection causing camera misidentification" (environment), "the fusion algorithm not enabling rain mode" (machine), and "the driver's inattention and failure to take over in time" (human), thereby promoting cross-module collaborative optimization.

[0126] Third, the introduction of Bayesian networks provides powerful probabilistic reasoning capabilities for root cause analysis. Unlike traditional deterministic analysis, Bayesian networks can handle uncertainty and partial observational information, dynamically updating the probability distribution of each node based on prior knowledge and real-time data. For example, when the system detects a sudden deceleration of a vehicle but fails to trigger AEB (Autonomous Emergency Braking), the Bayesian network can combine multiple pieces of evidence, such as the perception confidence of "obstacle ahead," "brake system pressure change," and "driver's pedal signal," to calculate the probability of hypotheses such as "perception missed," "control delay," or "driver's autonomous braking," and output the most likely explanation. This probabilistic reasoning mechanism is particularly suitable for real-world road scenarios with high sensor noise and incomplete information, significantly improving the accuracy and adaptability of diagnosis.

[0127] Finally, knowledge graph-based analysis algorithms represent an advanced analytical paradigm driven by artificial intelligence. By constructing a knowledge graph encompassing multi-source information such as autonomous driving system architecture, component relationships, historical fault cases, maintenance records, software versions, and environmental conditions, the system can achieve semantic-level associative reasoning. For example, when a certain vehicle model frequently experiences "minimum risk strategy mis-start" under a specific software version, the knowledge graph can automatically link it to the version's changelog, test reports of related modules, operational data of similar vehicles, and even use natural language processing technology to analyze keywords in user feedback, thereby identifying potential common defects. The knowledge graph also supports analogical reasoning, matching new events with historical cases based on similarity, quickly recommending verified solutions, and significantly shortening the problem response cycle.

[0128] In summary, the hazard scene detection method for autonomous driving provided in this application has the following technical effects.

[0129] This method constructs a systematic set of five hazard scenario detection rules (driver takeover, emergency system exit, accident, exit from ODD, and minimum risk strategy) to accurately identify and classify various potential safety risks during autonomous driving operations. The method collects status signals in real time at the vehicle and compares them against the rules. Once the triggering condition is met, it automatically collects multi-dimensional embedded data points from the entire vehicle within a time window of [-15s, 5s], based on the triggering time, and uploads it to the cloud with user authorization. This mechanism overcomes the passivity and limitations of traditional data recording systems, achieving proactive, efficient, and closed-loop data collection and transmission for critical safety events. It significantly improves the monitoring capabilities of hazard scenarios and provides a high-quality data foundation for accident reconstruction, liability determination, and system optimization.

[0130] Furthermore, this method constructs a complete data analysis and feedback loop in the cloud. By performing integrity verification, time alignment, and visual modeling on uploaded data, the system can accurately reconstruct the entire event process. Combining advanced algorithms such as FMEA, fault trees, fishbone diagrams, Bayesian networks, and knowledge graphs, it achieves risk level assessment and root cause localization, and generates targeted software repair solutions. Finally, the repair package is pushed to the target vehicle via OTA, completing remote upgrades and problem closure. This process not only significantly shortens the problem response cycle but also forms a continuous evolution mechanism of "problem discovery—problem analysis—problem resolution—effect verification," effectively supporting autonomous driving systems in addressing long-tail scenario challenges, improving overall safety, traceability, and intelligence levels, and driving intelligent connected vehicles towards a higher level of autonomous evolution.

[0131] Secondly, refer to Figure 2This application provides a hazard scene detection system for autonomous driving, including a rule setting module 510, a signal acquisition module 520, a hazard scene detection module 530, and a safety repair module 540.

[0132] The rule setting module 510 is used to set various hazard scenario detection rules, including driver takeover rules, emergency system exit rules, accident rules, exit operation design domain rules, and minimum risk rules. Each type of rule corresponds to at least one trigger condition and monitoring signal.

[0133] The signal acquisition module 520 is used to continuously acquire vehicle signals during autonomous driving.

[0134] The hazard scene detection module 530 compares the vehicle signals with the trigger conditions of the hazard scene detection rules to determine whether the trigger conditions are met. When the trigger conditions are met, the module collects vehicle embedded data within a time period near the trigger time according to a preset time window and uploads it to the cloud.

[0135] The safety repair module 540 is used to perform scene reconstruction, risk assessment and problem analysis based on vehicle embedded data in the cloud, generate a safety repair solution, and upgrade the vehicle software via OTA.

[0136] Furthermore, embodiments of this application provide an electronic device, including a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the aforementioned steps of the hazard scene detection method for autonomous driving.

[0137] Furthermore, this application provides a vehicle that integrates the aforementioned autonomous driving hazard scene detection system.

[0138] Similarly, the technical effects of the systems, electronic devices, and vehicles provided in the embodiments of this application are the same as those of the method embodiments described above, and will not be repeated here.

[0139] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0140] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0141] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0142] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0144] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or, if necessary, processing in a suitable manner, and then stored in computer memory.

[0145] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0146] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0147] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0148] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for detecting hazardous scenarios in autonomous driving, characterized in that, Includes the following steps: Set up multiple hazard scenario detection rules, with each rule corresponding to at least one trigger condition and monitoring signal; During autonomous driving, the vehicle's signals are continuously collected. The vehicle signal is compared with the triggering conditions of the hazard scenario detection rules to determine whether the triggering conditions are met. When the triggering condition is met, according to the preset time window, the vehicle embedded data within a time period near the triggering time is collected and uploaded to the cloud; In the cloud, based on the vehicle's embedded data, scene reconstruction, risk assessment, and problem analysis are performed to generate a safety repair plan, and the vehicle software is upgraded via OTA.

2. The method for detecting hazardous scenarios in autonomous driving according to claim 1, characterized in that, The hazard scenario detection rules include driver takeover rules, emergency system exit rules, accident rules, exit operation design domain rules, and minimum risk rules; The triggering conditions for the driver takeover rule include, but are not limited to: the autonomous driving function is activated and the steering wheel speed exceeds a preset speed threshold. The triggering conditions for the emergency system exit rule include, but are not limited to: the autonomous driving function being activated and the active safety system being activated; The triggering conditions for the accident rules include, but are not limited to: the autonomous driving function being activated and the passive safety system issuing a collision signal; The triggering conditions for the exit operation design domain rule include, but are not limited to: environmental conditions exceeding the autonomous driving design domain; The triggering conditions for the minimum risk rule include, but are not limited to: the autonomous driving function being activated and the system issuing the highest level alarm.

3. The method for detecting hazardous scenarios in autonomous driving according to claim 1, characterized in that, The vehicle embedded data includes: Basic vehicle information, including vehicle identification number and location; Vehicle status and dynamic information, including ignition information; Driver status and operational information, including driver status monitoring information; Environmental perception-related information, including camera-related video and light intensity information; Autonomous driving system operating information, including the requested steering angle; Instrument and HMI display information, including information related to DTC diagnostic fault codes; Information related to new energy sources includes the SOC value at the start and end of charging.

4. The method for detecting hazardous scenarios in autonomous driving according to claim 1, characterized in that, When the triggering condition is met, the vehicle's embedded data within a time period near the triggering time is collected according to a preset time window and uploaded to the cloud, including the following steps: The timestamp of the triggering moment is taken as the origin and marked as the reference time point for data collection; Based on the aforementioned reference time point, the data collection time window is defined as [-15s, 5s] of the trigger time, covering the whole vehicle embedded data from 15 seconds before the trigger to 5 seconds after the trigger. Real-time storage of the vehicle embedded data within the time window; When the amount of collected vehicle data points accumulates to a preset data volume threshold, the cached vehicle data points are packaged and uploaded to the cloud within the scope authorized by the user. After receiving the vehicle's embedded data, the cloud system associates and stores it with the corresponding hazard scenario type and trigger timestamp, supporting subsequent scenario reconstruction and security analysis.

5. The method for detecting hazardous scenarios in autonomous driving according to claim 1, characterized in that, The process, conducted on the cloud, involves scene reconstruction, risk assessment, and problem analysis based on the vehicle's embedded data, generating a security remediation plan, and upgrading the vehicle software via OTA, including the following steps: After receiving the vehicle embedded data, the cloud verifies the data integrity and timing consistency of the vehicle embedded data, and removes outliers and redundant information from the vehicle embedded data through data cleaning processing. Based on the timestamp, the vehicle embedded data within the time window [-15s, 5s] is aligned to construct a time-series visualization model of the hazardous scenario, reconstruct the vehicle state change process, and generate scenario reconstruction data; By invoking a preset risk assessment algorithm and combining the scene reconstruction data, the risk level is calculated, potential safety hazards are identified, and a risk assessment result is generated. Based on the risk assessment results, the problem module is located using a root cause analysis algorithm; For the identified problematic modules, generate corresponding software security remediation solutions; The security repair solution is packaged into an OTA upgrade package, pushed to the target vehicle, and the upgrade progress and success rate of the corresponding software are monitored.

6. The method for detecting hazardous scenarios in autonomous driving according to claim 5, characterized in that, The process of aligning the vehicle's embedded data within the time window [-15s, 5s] based on timestamps to construct a time-series visualization model of the hazardous scenario, reconstructing the vehicle's state change process, and generating scenario reconstruction data includes the following steps: Using the timestamp of the triggering time as a reference point, the time coordinate system of the vehicle embedded data is unified; Based on the time coordinate system, align the vehicle embedded data within the time window [-15s, 5s]; The integrated and aligned vehicle embedded data is used to define the temporal association rules between the data and construct the temporal visualization model containing a three-dimensional spatiotemporal coordinate system. Using the time-series visualization model, key state nodes before and after the triggering time are extracted, the change process of the whole vehicle embedded data is reproduced, the interaction process between the vehicle and the environment under the hazardous scenario is restored, and the initial scenario restoration data is generated. The initial scene reconstruction data is subjected to integrity verification. Missing key data fragments are supplemented by interpolation algorithm to generate the scene reconstruction data in a standardized format, and the hazardous scene type and trigger timestamp are associated and stored.

7. The method for detecting hazardous scenarios in autonomous driving according to claim 5, characterized in that, The risk assessment algorithm includes the FMEA (Failure Mode and Effects Analysis) algorithm; the root cause analysis algorithm includes algorithms such as fault tree analysis, fishbone diagram analysis, Bayesian networks, or knowledge graph-based analysis algorithms.

8. A hazard detection system for autonomous driving, characterized in that, It includes a rule setting module, a signal acquisition module, a hazard scenario detection module, and a security repair module; The rule setting module is used to set various hazard scenario detection rules, and each type of rule corresponds to at least one trigger condition and monitoring signal; The signal acquisition module is used to continuously acquire vehicle signals during autonomous driving. The hazard scene detection module is used to compare the vehicle signal with the trigger conditions of the hazard scene detection rules to determine whether the trigger conditions are met; when the trigger conditions are met, the module collects vehicle embedded data within a time period near the trigger time according to a preset time window and uploads it to the cloud. The security repair module is used to perform scene reconstruction, risk assessment and problem analysis based on the vehicle's embedded data in the cloud, generate a security repair solution, and upgrade the vehicle software via OTA.

9. An electronic device, characterized in that, It includes a processor and a memory; the memory stores a computer program, and the processor executes the computer program to implement the steps of the hazard scene detection method for autonomous driving as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle integrates an autonomous driving hazard detection system as described in claim 8.

Images