A smart connected vehicle safety brain system based on vehicle-cloud interaction

Abstract

This invention relates to the field of autonomous driving technology, and particularly to an intelligent connected vehicle safety brain system based on vehicle-cloud interaction. The system includes a vehicle-side safety brain system located in the vehicle and a cloud-based safety brain system located in the cloud. The vehicle-side safety brain system acquires vehicle safety information in real time from the vehicle network system to identify anticipated functional safety risks and functional safety risks. Based on the identification results, it determines whether emergency response is required. If so, it sends the generated emergency response information to the vehicle's execution system for emergency control of the autonomous vehicle. The cloud-based safety brain system performs single-vehicle and multi-vehicle safety analysis on the autonomous vehicle, further conducting comprehensive safety monitoring. In this solution, the vehicle-side safety brain system is responsible for real-time processing and emergency response, while the cloud-based safety brain system is responsible for big data analysis and safety monitoring. The two systems complement each other, providing comprehensive safety assurance for autonomous vehicles.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an intelligent connected vehicle safety brain system based on vehicle-cloud interaction. Background Technology

[0002] The widespread application of artificial intelligence (AI) technology has promoted the rapid development of vehicle intelligence, with autonomous driving being a typical example. However, the long-tail problem, edge cases, and lack of interpretability of AI technology make the safety of autonomous vehicles a critical issue that urgently needs to be addressed. Furthermore, with the development of communication technology, vehicles have become a crucial link in communication networks, making traffic scenarios more complex. Vehicles not only interact with the physical environment but also exchange a large amount of data with the outside world, making automotive information security particularly urgent.

[0003] Against this backdrop, traditional safety measures may no longer be sufficient to address the growing safety challenges, necessitating a systematic and comprehensive approach to protecting the safety of autonomous vehicles. The rapid development of vehicle-to-everything (V2X) technology has provided automobiles with the ability to communicate with their external environment in real time, while the application of cloud computing and big data analytics offers new tools for predicting and preventing potential safety risks.

[0004] Therefore, there is an urgent need for a smart connected vehicle safety brain system based on vehicle-cloud interaction. Summary of the Invention

[0005] To address the problem that traditional safety protection methods are insufficient for comprehensive safety protection of autonomous vehicles, this invention provides an intelligent connected vehicle safety brain system based on vehicle-cloud interaction.

[0006] In a first aspect, embodiments of the present invention provide a vehicle-cloud-based intelligent connected vehicle safety brain system, comprising: a vehicle-side safety brain system installed on the vehicle and a cloud-based safety brain system installed in the cloud, wherein the vehicle-side safety brain system and the cloud-based safety brain system are respectively connected to the original intelligent connected vehicle system, and the vehicle-side safety brain system and the cloud-based safety brain system are interconnected; the original intelligent connected vehicle system includes a vehicle networking system, a vehicle autonomous driving system, and a vehicle execution system;

[0007] The vehicle-side safety brain system is used to obtain vehicle safety information from the vehicle network system in real time to identify expected functional safety risks and functional safety risks, and to determine whether to take emergency measures based on the identification results. If so, the generated emergency measures information is sent to the vehicle execution system to perform emergency control of the autonomous vehicle.

[0008] The cloud-based security brain system is used to obtain fault information, autonomous driving performance data, and driver operation behavior data of autonomous vehicles from the vehicle network system and the vehicle-side security brain system, so as to perform single-vehicle safety analysis of autonomous vehicles and multi-vehicle safety analysis of the same model of autonomous vehicles, and further perform comprehensive safety monitoring of autonomous vehicles.

[0009] Secondly, embodiments of the present invention also provide a method for safety monitoring of intelligent connected vehicles, comprising:

[0010] The vehicle-side safety brain system uses real-time vehicle safety information from the vehicle network system to identify anticipated functional safety risks and functional safety risks. Based on the identification results, it determines whether to take emergency measures. If so, the generated emergency measures information is sent to the vehicle execution system for emergency control of the autonomous vehicle.

[0011] The cloud-based security brain system acquires fault information, autonomous driving performance data, and driver operation behavior data of autonomous vehicles from the vehicle network system and the vehicle-side security brain system to perform single-vehicle safety analysis and multi-vehicle safety analysis of the same model of autonomous vehicles, thereby further enabling comprehensive safety monitoring of autonomous vehicles.

[0012] This invention provides an intelligent connected vehicle safety brain system based on vehicle-cloud interaction. The vehicle-side safety brain system identifies anticipated functional safety risks and functional safety risks in real time. When it is determined that emergency response is required, the vehicle-side safety brain system takes priority over the control of the vehicle's autonomous driving system and uses the emergency response information generated by the vehicle-side safety brain system to perform emergency control on the vehicle. The vehicle-side safety brain system of this solution can simultaneously consider functional safety issues and anticipated functional safety issues, and provide real-time comprehensive safety protection for autonomous vehicles.

[0013] The cloud-based security brain system performs single-vehicle safety analysis on autonomous vehicles and multi-vehicle safety analysis on a massive number of autonomous vehicles. Leveraging the advantages of big data, it comprehensively monitors the operating status of intelligent connected vehicles, identifies risks and determines defects, thereby improving the safety level of autonomous vehicles.

[0014] Therefore, this solution combines the rapid real-time response capability of the vehicle-side safety brain system with the powerful computing capability of the cloud-based safety brain system, forming a complementary advantage of vehicle-cloud collaboration. This effectively resolves the contradiction between real-time response and big data analysis, thereby providing more comprehensive safety protection for autonomous vehicles. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the composition of an intelligent connected vehicle safety brain system based on vehicle-cloud interaction, provided in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the composition of a vehicle-side safety brain system provided in an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of the composition of a cloud-based security brain system provided in an embodiment of the present invention;

[0019] Figure 4 This is a flowchart of a safety monitoring method for intelligent connected vehicles provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] Please refer to Figure 1 This invention provides a vehicle-cloud-based intelligent connected vehicle safety brain system, comprising: a vehicle-side safety brain system located on the vehicle and a cloud-based safety brain system located in the cloud. The vehicle-side safety brain system and the cloud-based safety brain system are respectively connected to the original intelligent connected vehicle system, and the vehicle-side safety brain system and the cloud-based safety brain system are interconnected. The original intelligent connected vehicle system includes a vehicle networking system, a vehicle autonomous driving system, and a vehicle execution system.

[0022] The vehicle-side safety brain system is used to obtain vehicle safety information from the vehicle network system in real time to identify expected functional safety risks and functional safety risks, and to determine whether to take emergency measures based on the identification results. If so, the generated emergency measures information is sent to the vehicle execution system to carry out emergency control of the autonomous vehicle.

[0023] The cloud-based safety brain system is used to obtain fault information, autonomous driving performance data, and driver operation behavior data of autonomous vehicles from the vehicle network system and the vehicle-side safety brain system, so as to perform single-vehicle safety analysis of autonomous vehicles and multi-vehicle safety analysis of the same model of autonomous vehicles, and further conduct comprehensive safety monitoring of autonomous vehicles.

[0024] In this embodiment of the invention, the vehicle-side safety brain system identifies anticipated functional safety risks and functional safety risks in real time. When it is determined that emergency response is required, the vehicle-side safety brain system takes priority over the control of the vehicle's autonomous driving system and uses the emergency response information generated by the vehicle-side safety brain system to perform emergency control on the vehicle. The vehicle-side safety brain system of this solution can simultaneously consider functional safety issues and anticipated functional safety issues, and provide real-time comprehensive safety protection for autonomous vehicles.

[0025] The cloud-based security brain system performs single-vehicle safety analysis on autonomous vehicles and multi-vehicle safety analysis on a massive number of autonomous vehicles. Leveraging the advantages of big data, it comprehensively monitors the operating status of intelligent connected vehicles, identifies risks and determines defects, thereby improving the safety level of autonomous vehicles.

[0026] Therefore, this solution combines the rapid real-time response capability of the vehicle-side safety brain system with the powerful computing capability of the cloud-based safety brain system, forming a complementary advantage of vehicle-cloud collaboration. This effectively resolves the contradiction between real-time response and big data analysis, thereby providing more comprehensive safety protection for autonomous vehicles.

[0027] In this embodiment of the invention, the vehicle-side safety brain system further includes a data encryption module, which is used to encrypt the data required by the cloud-based safety brain system and then transmit it to the cloud-based safety brain system.

[0028] The cloud-based security brain system also includes a data parsing module, which is used to parse the encrypted data transmitted by the vehicle-side security brain system.

[0029] In this embodiment, the data encryption module in the vehicle-side security brain system can prevent potential security threats and information tampering during the data transmission and storage lifecycle, providing a higher level of security for the entire system. The data parsing module in the cloud-based security brain system uses data decryption technologies such as symmetric decryption algorithms and asymmetric decryption algorithms to parse the encrypted data transmitted in the vehicle-side security brain system, providing data for other modules.

[0030] Next, we will explain the vehicle-side security brain system and the cloud-based security brain system respectively.

[0031] (1) Vehicle-side safety brain system

[0032] First, please refer to Figure 1 and Figure 2 The vehicle-side safety brain system includes: a risk identification module, a safety and defect analysis module, and an emergency response module;

[0033] The risk identification module connects to the original vehicle system to obtain vehicle safety information from the original vehicle system for the purpose of identifying expected functional safety risks and functional safety risks. The vehicle safety information includes the vehicle's state vector and the state vectors of surrounding traffic participants for scenario risk assessment, operational data for monitoring the driver's human-machine interaction behavior, and the state signals of the vehicle's autonomous driving system's sensors, communications, and controllers.

[0034] The safety and defect analysis module is used to perform fault analysis and source tracing based on the real-time identification results of the risk identification module, and to calculate and classify the comprehensive risk value. Based on the classification results, it is determined whether emergency response should be carried out. If so, an emergency response strategy is formulated based on vehicle safety information and classification results.

[0035] The emergency response module is used to generate emergency response information based on the emergency response strategy formulated by the safety and defect analysis module and send it to the vehicle execution system for emergency control of autonomous vehicles.

[0036] In this embodiment, anticipated functional safety issues refer to the hazards and risks caused by the inadequacy of the autonomous driving system's perception, decision-making, and control functions in complex traffic scenarios, as well as the hazards and risks caused by reasonably foreseeable human misuse; functional safety issues refer to problems caused by system malfunctions. Since there is currently a lack of comprehensive safety protection systems that can simultaneously consider both types of safety issues, the vehicle-side safety brain system is equipped with a risk identification module. This module performs real-time identification of anticipated functional safety risks and functional safety risks, including identifying complex and dangerous traffic scenarios, driver misoperation, and malfunctions of the vehicle's autonomous driving system. Then, the safety and defect analysis module assesses the comprehensive risk value and risk level. When the risk level is high, an emergency response strategy is formulated based on vehicle safety information and the classification results. The emergency response module generates emergency response information for the vehicle's execution system, prioritizing control over the autonomous driving system, and performs emergency control of the vehicle. Therefore, this solution can simultaneously consider both functional safety issues and anticipated functional safety issues to provide comprehensive safety protection for autonomous vehicles, offering a solid guarantee for their safe operation.

[0037] In some implementations, reference Figure 2 The risk identification module includes a vehicle driving risk identification module, a human-machine interaction monitoring module, and a vehicle functional safety risk identification module.

[0038] The vehicle driving risk identification module is used to assess the driving risk value in the current traffic scenario based on the risk model, the vehicle's state vector, and the state vectors of surrounding traffic participants.

[0039] The human-machine interaction monitoring module is used to monitor the operation data of the driver's human-machine interaction behavior in order to identify the driver's misuse or improper operation behavior and determine the factors of human misuse.

[0040] The vehicle functional safety risk identification module is used to detect the status signals of sensors, communications, and controllers in the vehicle's autonomous driving system, and to determine the fault results and functional safety risk values.

[0041] In this embodiment, the vehicle driving risk identification module analyzes the vehicle state vector and the state vectors of surrounding traffic participants collected by the original vehicle system for scenario risk assessment. The vehicle state vector includes information such as position, heading angle, speed, and acceleration. At the same time, it collects similar information of surrounding traffic participants to form multiple state vectors. Then, based on the risk model, it assesses the driving risk value of factors such as the relative distance, speed, and potential collision points between vehicles in the current traffic scenario, identifies potential dangers or abnormal behaviors, and immediately issues a warning message to the safety and defect analysis module once the driving risk value is detected to exceed the safety threshold.

[0042] In addition, this module also considers environmental factors such as weather conditions, road type, traffic signs, and traffic lights, making risk identification more comprehensive. The human-machine interaction monitoring module monitors the actions of drivers and passengers in operating vehicle controls, including the timing, force, and duration of pressing the accelerator and brake pedals; the timing, torque, and direction of steering wheel operation; and actions in operating the vehicle's infotainment system, including button force, button frequency, and operation duration, to identify driver misuse or improper operation behaviors and determine the factors contributing to human misuse.

[0043] The Vehicle Functional Safety Risk Identification Module is responsible for diagnosing sensor, communication, and controller faults in the autonomous driving system and transmitting the information in real time to the Safety and Defect Analysis Module to ensure system stability and reliability. Sensor fault detection involves reading and analyzing fault signals emitted by sensors, including monitoring for communication errors, state failures, state blockages, and over-temperature conditions. For communication and controller fault detection, the Vehicle Functional Safety Risk Identification Module uses advanced communication technology to monitor CAN signals in real time. When a CAN fault occurs, the system can quickly identify the error state in the CAN network and accurately locate the fault using LabVIEW signal processing. Simultaneously, the LabVIEW program uses Virtual Instruments (VIs) to further detect and diagnose errors in the CAN signals, ensuring the stability and reliability of the communication network. These three modules work closely together to perform comprehensive risk identification for both anticipated and functional safety.

[0044] In some implementations, the driving risk value is assessed in the following manner:

[0045] Determine the Time-of-Collision (TTC) based risk indicator value (ITTC):

[0046]

[0047] In the formula, D x D is the lateral distance between the vehicle and other vehicles. y L is the longitudinal distance between the vehicle and other vehicles. w For the width of the vehicle, L l For the length of the vehicle, V rx V is the lateral relative velocity between the vehicle and other vehicles. ry Let A be the longitudinal relative velocity between the vehicle and another vehicle. rx Let A be the lateral relative acceleration between the vehicle and the other vehicle. ry Let be the longitudinal relative acceleration of the vehicle and the other vehicle, and 'a' be the influence coefficient of the relative acceleration of the vehicle and the other vehicle.

[0048] The ratio of the vehicle's speed to the road speed limit is used as the correlation coefficient λ in the calculation of driving risk value. The calculation formula is as follows:

[0049]

[0050] In the formula, v is the speed of the vehicle. l To limit speed on roads, such as a maximum speed of 120 km / h on highways, then v l The value is 120. The larger the value of λ, the faster the vehicle's speed is relative to the road speed limit, and the greater the impact on the driving risk value.

[0051] The driving risk value r under the current traffic scenario is:

[0052] r = exp(-λ·ITTC).

[0053] In some implementations, the human misuse factor is determined as follows: if the human-machine interaction monitoring module identifies that the driver has misused or improper operation behavior in the current detection cycle, the human misuse factor is determined to be 2; if no human misuse occurs, the human misuse factor is 1.

[0054] In some implementations, the functional safety risk value is determined by referring to the ASIL vehicle safety integrity level and assessing the functional safety risk value based on three factors: severity, exposure, and controllability of the fault determined by the vehicle functional safety risk identification module in the current inspection cycle. The functional safety risk value can be rated as 1, 2, 3, or 4 from low to high.

[0055] In some implementations, the vehicle functional safety risk identification module is also used to adjust the fault detection priority based on the high-risk traffic scenario when the vehicle driving risk identification module assesses a high-risk traffic scenario, so as to more quickly pinpoint the source of the fault and prioritize the determination of the operating status of key safety components in the current traffic scenario.

[0056] In some implementations, the security and defect analysis module includes at least a fault tracing analysis module, a defect analysis module, a security calculation and classification module, and an emergency response strategy module;

[0057] The input end of the fault tracing analysis module is connected to the vehicle functional safety risk identification module. It is used to establish a system fault propagation model by using a hybrid algorithm based on physical models and data-driven approaches. It uses artificial intelligence technology based on long short-term memory networks (LSTM) to infer the causal relationship between time series, locate the fault location and type, and complete the fault tracing.

[0058] The input end of the defect analysis module is connected to the vehicle functional safety risk identification module. It is used to collect vehicle defect clues through multiple channels and use artificial intelligence technology based on supervised learning to perform defect correlation analysis. It classifies the same or similar faults that occur in different safety events of the vehicle. Then, it uses correlation analysis to mine and summarize the patterns with common attributes in multi-source data to determine the root cause of the fault.

[0059] The safety calculation and classification module is used to determine the comprehensive risk value based on the driving risk value output by the vehicle driving risk identification module, the human misuse factor output by the human-machine interaction monitoring module, and the functional safety risk value output by the vehicle functional safety risk identification module, and to complete the safety classification based on fuzzy theory and clustering algorithm.

[0060] The emergency response strategy module is used to formulate driving routes based on vehicle safety information and determine the braking urgency of the driving routes based on the classification results.

[0061] In this embodiment, when the vehicle functional safety risk identification module identifies a fault in the vehicle's autonomous driving system, the fault tracing analysis module and the defect analysis module trace and analyze the fault. If no fault is identified, the fault tracing analysis module and the defect analysis module do not operate.

[0062] The safety calculation and classification module determines the overall risk value R in the following way:

[0063] R = β × γ × r

[0064] In the formula, β is the functional safety risk value, γ is the personnel misuse factor, and r is the driving risk value. The larger the comprehensive risk value R, the higher the vehicle risk level.

[0065] Furthermore, the emergency response strategy module not only considers the current risk level and fault status but also predicts the risk development trend within a certain forecast time window, thus making a more predictive safety analysis. Based on the analysis and prediction, it determines a safe driving route, as well as measures such as deceleration, lane changing, stopping, or initiating emergency avoidance procedures. In special circumstances, if the system detects a serious fault or an irreversible risk, the emergency response strategy module may also decide to safely guide the vehicle to a stop and initiate emergency assistance procedures.

[0066] In some implementations, the security and defect analysis module also includes a security log generation module;

[0067] The input end of the safety log generation module is connected to the risk identification module, fault tracing and analysis module, defect analysis module, safety calculation and classification module, and emergency response strategy module, while the output end is connected to the safety brain system located in the cloud. The safety log generation module is used to record the vehicle operating status, driver operation data, and vehicle autonomous driving system faults when a safety incident occurs. After being encrypted by the data encryption module, the data is reported to the cloud-based safety brain system for big data analysis.

[0068] In some implementations, the emergency response module includes at least: a lateral response module and a longitudinal response module;

[0069] The lateral response module is used to generate emergency response information for the vehicle steering system based on emergency response strategies in order to control the direction of the vehicle.

[0070] The longitudinal response module is used to generate emergency response information on vehicle speed and acceleration based on emergency response strategies, in order to control the vehicle's power output and braking force output.

[0071] In this embodiment, the lateral response module is responsible for the vehicle's direction and path following, including steering, lane changing, and lane keeping. By controlling the vehicle's steering system, it ensures that the vehicle accurately follows the safe driving path defined by the emergency response strategy module, thereby maintaining vehicle stability and good handling performance even in emergency avoidance or complex road conditions. The longitudinal response module mainly adjusts the vehicle's speed and acceleration, including acceleration, deceleration, and emergency braking. When the emergency response strategy module determines that a speed change or stopping is necessary, the longitudinal control intervenes and controls the vehicle's power output and braking force output.

[0072] In some implementations, the emergency response module also includes a safety warning module; the safety warning module obtains risk information from the safety and defect analysis module and provides safety warnings and feedback to the driver in emergency situations through head-up display (HUD) information prompts, steering wheel and seat vibrations, and in-vehicle voice prompts.

[0073] In some implementations, the emergency response module is also used to: generate emergency response information using the vehicle safety information at the last moment before the failure, and control the vehicle when a fault causes communication data between systems to fail.

[0074] In this embodiment, the emergency response module serves as the last line of defense in the safety protection system. Besides executing the conventional functions of the emergency response strategy output from the safety and defect analysis module, it exhibits strong fault tolerance and robustness in extreme situations. Even without critical data from the safety and defect analysis module, it can achieve a safe vehicle response solely based on the vehicle's own status information. When a malfunction in the autonomous driving system or an internal fault in the system of this embodiment causes communication data between systems to fail, the vehicle safety information at the last moment before the malfunction occurs is used to predict the surrounding environment based on a kinematic model and control the vehicle.

[0075] In this embodiment of the invention, when a fault causes the communication data between the systems to fail, the longitudinal response module is used to calculate the safe driving distance in real time using the deceleration model and decelerate based on the sliding mode control algorithm to stop within the safe distance; the lateral response module uses the last effective expected path issued by the vehicle's autonomous driving system to control the vehicle's steering system, thereby ensuring basic safety response functions.

[0076] Therefore, the vehicle-side safety brain system for autonomous vehicles in this embodiment of the invention covers three key aspects: risk identification, safety and defect analysis, and emergency response. In the risk identification section, this system implements a comprehensive risk identification strategy, capable of accurately and in real-time detecting and assessing safety risks in traffic scenarios, monitoring human-machine interaction behavior, and diagnosing faults in various functions of the autonomous driving system. The safety and defect analysis module comprehensively analyzes traffic scenario data, human-machine interaction behavior data, and vehicle fault conditions obtained from the risk identification module. Based on this information, it performs comprehensive analysis, calculates the comprehensive risk value and safety level, intelligently selects the most appropriate safety strategy, and generates a safety log to address the current traffic environment and system status. Furthermore, the safety and defect analysis module can also trace the source of faults by factors such as fault location and type, uses artificial intelligence technology for correlation analysis, accurately identifies the factors associated with the occurrence of defects, and the generated safety log records vehicle operation data, event information, and occupant behavior, providing a reliable data source for accident reconstruction. In terms of emergency response, in addition to its standard functions, the system can maintain effective control of the vehicle even when the autonomous driving system and safety and defect analysis module fail. This is thanks to the reference model embedded in the emergency response module, which enables safe control even in extreme situations. Furthermore, a safety warning module is included to issue safety warnings in emergencies, prompting occupants to take necessary actions and protective measures to ensure driving safety.

[0077] (2) Cloud Security Brain System

[0078] Please refer to Figure 1 and Figure 3 The cloud-based security brain system includes: a digital twin model, a vehicle safety monitoring module, and a security situation awareness and defect determination module;

[0079] Digital twin models are used to build physical models of various types of autonomous vehicles, and to construct digital twin virtual scenes using collected data on the surrounding environment, vehicle status and driving behavior, so as to conduct simulation analysis on various types of autonomous vehicles.

[0080] The vehicle safety monitoring module is used to perform single-vehicle safety analysis of autonomous vehicles based on digital twin models, fault information and state performance data of various vehicle subsystems, as well as multi-vehicle safety analysis of autonomous vehicles of the same model based on fault information, autonomous driving performance data and driver operation behavior data of various autonomous vehicles of the same model.

[0081] The safety situation awareness and defect judgment module is used to assess and classify the operational safety risks of each autonomous vehicle based on the safety analysis results of a single vehicle and the safety analysis results of multiple vehicles.

[0082] It is understandable that the cloud-based security brain system can simultaneously connect to the original vehicle systems of several autonomous vehicles, and there can be multiple models of autonomous vehicles, with multiple vehicles of each model.

[0083] In this embodiment of the invention, analyzing only the data of a single vehicle is insufficient to achieve comprehensive safety protection; it is also necessary to utilize big data to perform multi-vehicle analysis of vehicles of the same model. Therefore, this solution conducts single-vehicle safety analysis on autonomous vehicles, as well as multi-vehicle safety analysis using massive amounts of fault information, autonomous driving performance data, and driver operation behavior data from autonomous vehicles. This allows for comprehensive monitoring of the operating status of intelligent connected vehicles, identification of risks, and assessment of defects, thereby improving the safety level of autonomous vehicles.

[0084] In some implementations, reference Figure 1 and Figure 3 The cloud-based security monitoring system also includes an application service module; the application service module includes a defect recall module, a sandbox monitoring module, and a quality and safety management module.

[0085] When the risk level assessed by the security situation awareness and defect determination module exceeds the first threshold but is less than the second threshold, the defect recall module is used to trigger the defect recall process and notify the vehicle manufacturer and vehicle owner to carry out defect repair.

[0086] When the risk level assessed by the security situation awareness and defect judgment module exceeds the second threshold, the quality and safety management module is used to issue an alarm to notify the vehicle manufacturer and vehicle owner to carry out quality and safety management.

[0087] The Sandbox Oversight module is used to create sandbox environments to simulate and test the running state of new features.

[0088] In this embodiment, the risk level in the safety situation awareness and defect judgment module is evaluated in real time. When the risk level exceeds the first threshold but is less than the second threshold, it indicates that the autonomous vehicle has a defect and needs to be repaired, and the defect recall module is activated. When the risk level exceeds the second threshold, it indicates that the safety problem of the autonomous vehicle is more serious. For example, if the vehicle model has been involved in an accident, it indicates that the autonomous driving function of the vehicle model may have safety hazards and requires urgent quality and safety management.

[0089] In this embodiment, the quality and safety management module strictly adheres to safety standards and continuously monitors the vehicle's software and hardware components to regularly check their compliance, ensuring that the vehicle system always meets relevant safety regulations. Once the system detects any non-compliance, the application service immediately issues an alert and takes necessary corrective measures. Simultaneously, the service enhances vehicle owners' understanding of vehicle safety features and proper operating methods by regularly updating safety information and providing safety guidance. This not only helps maintain the vehicle's good condition but also effectively improves the overall safety level.

[0090] In addition, the Sandbox monitoring module creates a "sandbox" environment in which new features and updates are simulated and tested. This monitors the operational status of new features without negatively impacting the normal operation of the vehicle, thereby improving the security and reliability of new in-vehicle software and updates before their official deployment.

[0091] In some implementations, reference Figure 1 and Figure 3 The cloud-based security monitoring system also includes a security knowledge base;

[0092] The safety knowledge base is used to extract safety features from collected fault data, defect data, and accident data, and to clean, classify, and grade the extracted key features in order to establish a typical fault database, a typical defect database, and a typical accident database.

[0093] In this embodiment, the safety knowledge base extracts key safety features from fault data, defect data, and accident data collected from the vehicle-side safety brain system, vehicle safety monitoring module, and original vehicle system. These features include the time, location, and type of the fault, defect, and accident. Based on these key features, the data is cleaned, classified, and graded, and imported and transformed to establish typical fault, typical defect, and typical accident databases within the safety knowledge base. Simultaneously, the effective storage and management of safety knowledge data is designed and implemented, and the safety knowledge base is regularly tested, maintained, and optimized. The typical fault database systematically collects and stores information on various common faults that may occur during vehicle operation, recording in detail the characteristics, causes, manifestations, detection methods, and repair measures of the faults. This provides a rapid diagnostic and fault repair information base, which can help manufacturers prevent these problems in future designs. The typical defect database records and analyzes typical defect information that occurs during vehicle design, manufacturing, or material use, including defect descriptions, impact ranges, detection methods, and corrective measures, serving the setting of industry safety and quality standards. The typical accident database stores vehicle accident cases, including the causes of the accidents, specific circumstances, vehicle models and behaviors involved, accident consequences, and post-accident analysis. By analyzing the commonalities and patterns of accident occurrences, it can provide valuable data resources for government regulatory agencies and automobile manufacturers to enhance the pertinence of safety strategies and measures.

[0094] In some implementations, reference Figure 1 and Figure 3 The cloud-based security monitoring system also includes a model optimization module; the model optimization module includes a vehicle-side security brain model optimization module and a cloud-based security brain model optimization module.

[0095] The vehicle-side safety brain model optimization module is connected to the vehicle-side safety brain system and the safety knowledge base. It is used to conduct intensive testing on the models of each module in the vehicle-side safety system based on the data collected from the vehicle-side safety brain system and the safety knowledge base, so as to continuously update and optimize the model parameters.

[0096] The cloud-based security brain model optimization module connects with the vehicle safety monitoring module, digital twin model, and security knowledge base. It is used to conduct intensive testing on each model in the vehicle safety monitoring module based on data collected from the vehicle safety monitoring module and security knowledge base, so as to continuously update and optimize the model parameters.

[0097] In this embodiment, the model optimization module collects data from the vehicle-side safety brain system, the vehicle safety monitoring module, and the safety knowledge base. It conducts intensive testing on each model in the vehicle-side safety brain system and the vehicle safety monitoring module to identify performance limitations. Through parameter updates and optimization, it adjusts the internal parameter weights and settings of the model. It utilizes data augmentation to expand and improve the training dataset of the artificial intelligence model. By employing model fusion to combine the advantages of multiple models, it continuously improves the robustness and generalization ability of the model, achieving higher prediction accuracy and efficiency.

[0098] In some implementations, reference Figure 3 The vehicle safety monitoring module includes the following modules for single-vehicle safety analysis: vehicle cumulative fault analysis module, vehicle functional safety early warning module, and vehicle quality status inspection module.

[0099] The vehicle cumulative fault analysis module is used to store fault information of autonomous vehicles in each time period, and to perform statistical analysis on the stored and generated cumulative fault data to determine the high-frequency faults of the autonomous vehicle.

[0100] The vehicle functional safety early warning module is used to determine the correlation between each fault and vehicle functional safety based on the cumulative fault data and digital twin model of the autonomous vehicle, so as to provide early warning of vehicle functional safety problems that the current fault may cause when real-time fault data is obtained.

[0101] The vehicle quality status assessment module uses artificial intelligence technology based on recurrent neural network models, along with the mileage, wear data, and status performance data of each subsystem of the autonomous vehicle, to predict the component quality status of the autonomous vehicle.

[0102] In this embodiment, the vehicle cumulative fault analysis module collects and stores fault information from historical time periods, including fault occurrence time, fault duration, fault location, and fault cause. It then performs statistical analysis on the stored and generated cumulative fault data to identify high-frequency faults in each autonomous vehicle. The vehicle functional safety early warning module, based on the cumulative fault data and digital twin model of the autonomous vehicle, determines the correlation between each fault and vehicle functional safety. When real-time fault data is acquired, it determines whether the current fault has a strong correlation with vehicle functional safety, i.e., the probability that the fault only affects vehicle performance or may cause functional safety problems. If a fault that may cause functional safety problems is involved, corresponding early warning measures need to be taken to improve vehicle safety and reliability. The vehicle quality status inspection module comprehensively collects data such as driving time, mileage, tire wear, tire pressure, and the status and performance data of the vehicle's engine system, transmission system, electronic system, and braking system from historical time periods. It uses artificial intelligence technology based on recurrent neural networks (RNN) to predict the status of the collected sequence data, inferring potential quality problems caused by component aging over time, and scoring the status of each system.

[0103] In some implementations, reference Figure 3 The vehicle safety monitoring module includes the following modules for multi-vehicle safety analysis: batch defect identification module, autonomous driving performance evaluation module, and driving behavior analysis module.

[0104] The batch defect identification module is used to identify batch defects of vehicles based on the fault information of each autonomous vehicle of the same model and the production time of each autonomous vehicle of the same model.

[0105] The autonomous driving performance evaluation module is used to evaluate the autonomous driving performance of a vehicle of the same model based on the autonomous driving performance data and scoring model of various autonomous vehicles.

[0106] The driving behavior analysis module is used to determine the abnormal state and unsafe behavior of drivers based on the operational behavior data of various driving groups in different autonomous vehicles of the same model.

[0107] In this embodiment, in addition to functional safety issues caused by system malfunctions, anticipated functional safety issues are also very important for autonomous vehicles. Therefore, in order to provide comprehensive safety protection for both functional safety issues and anticipated functional safety issues, a batch defect identification module, an autonomous driving performance evaluation module, and a driving behavior analysis module are designed to provide comprehensive safety protection for functional safety issues and anticipated functional safety issues respectively.

[0108] The batch defect identification module uses deep learning-based artificial intelligence to deeply mine batch failure vehicle data, including failure type, failure time, failure cause, failure frequency, and environmental factors, to identify batch defects and determine their batch characteristics, thereby preventing potential risks in advance. For example, a certain batch of a certain model may have emission issues. The autonomous driving performance evaluation module considers the perception, decision-making, planning, and control performance of different autonomous driving vehicles of the same model under different driving scenarios. It uses a scoring model to evaluate the autonomous driving performance of different vehicle groups, ensuring their reliability and safety during use. The driving behavior analysis module focuses on the driving behavior of groups. Using unsupervised learning-based artificial intelligence, it analyzes multi-dimensional driving behavior data, including the driver's attention state, hand and foot movements during driving, and multi-directional interactions between the driver and the vehicle. It performs driving behavior paradigm analysis on different driving groups, such as different age groups, driving experience levels, or driving styles, to better understand the driving behavior of different groups, promptly detect abnormal driver states or unsafe behaviors, and reduce traffic accidents caused by human factors.

[0109] In some implementations, when the autonomous driving performance evaluation module evaluates the autonomous driving performance of a vehicle model based on autonomous driving performance data and scoring models from various autonomous vehicles of the same model, it is specifically used for:

[0110] Step S1, for each research period, is to: acquire autonomous driving performance data for that research period and divide the autonomous driving performance data for that research period into several research cycles; wherein, the autonomous driving performance data includes the lateral and longitudinal acceleration of autonomous driving, the duration of driver takeover, and the number of dangerous events, where dangerous events refer to the failure of the autonomous driving system or a collision of the vehicle.

[0111] In this embodiment, the research period can be set to 3 days, 5 days, one week, or after the vehicle uses the autonomous driving function for the first time. Therefore, the autonomous driving performance data of each vehicle of the same model can be obtained during this research period. It can be understood that autonomous driving performance data can only be collected when the driver uses the autonomous driving function.

[0112] Step S2: Determine the performance score for each study cycle based on the lateral and longitudinal accelerations and the duration of driver takeover within each study cycle.

[0113] In some implementations, the performance score for each study period is evaluated in the following manner:

[0114]

[0115] In the formula, s iLet t be the performance score for the i-th study period, t be the driver takeover time during the study period, T be the duration of the study period, and m be the number of times the lateral and longitudinal accelerations exceeded the discrimination threshold.

[0116] In this embodiment, the criteria for determining lateral and longitudinal acceleration are shown in the table below:

[0117]

[0118] The lateral and longitudinal stability of autonomous vehicles during operation can be evaluated using the methods described above.

[0119] Step S3: Calculate the comprehensive autonomous driving performance score for each research period based on the performance score for each research cycle and the number of dangerous events that occurred during that research period.

[0120] In some implementations, the overall score for autonomous driving performance during this study period is evaluated in the following manner:

[0121]

[0122] In the formula, S represents the comprehensive score of autonomous driving performance during the study period, and s i Let n be the performance score for the i-th study period, n be the number of study periods within that study period, and p be the number of dangerous events that occurred within that study period.

[0123] It is evident that a higher S indicates better autonomous driving performance. When S is less than 0, it means that the autonomous vehicle does not meet the requirements for normal operation and requires a single-vehicle defect recall or quality and safety management.

[0124] It is understandable that by using the above-mentioned comprehensive scoring formula for autonomous driving performance, the number of dangerous events is discussed in different cases, which greatly increases the impact of dangerous events on the scoring results, so that the occurrence of dangerous events has a veto function. As long as S is less than 0, the risk level assessed by the safety situation awareness and defect judgment module can trigger defect recall or quality and safety management services.

[0125] Therefore, the intelligent connected vehicle safety brain system based on vehicle-cloud interaction provided in this embodiment of the invention has a vehicle-side safety brain system responsible for real-time processing and emergency response, while the cloud-side safety brain system is responsible for big data analysis and safety monitoring. The two complement each other and can provide comprehensive safety protection for autonomous vehicles.

[0126] like Figure 4 As shown, this embodiment of the invention also provides a method for safety monitoring of intelligent connected vehicles based on a system according to any embodiment of the specification, the method comprising:

[0127] Step 400: The vehicle-side safety brain system is used to obtain vehicle safety information from the vehicle network system in real time to identify expected functional safety risks and functional safety risks. Based on the identification results, it is determined whether to take emergency measures. If so, the generated emergency measures information is sent to the vehicle execution system to carry out emergency control of the autonomous vehicle.

[0128] Step 402: Use the cloud-based security brain system to obtain fault information, autonomous driving performance data and driver operation behavior data of autonomous vehicles from the vehicle network system and the vehicle-side security brain system, so as to conduct single-vehicle safety analysis of autonomous vehicles and multi-vehicle safety analysis of the same model of autonomous vehicles, and further conduct comprehensive safety monitoring of autonomous vehicles.

[0129] Since the above method is based on the same concept as the system embodiment of the present invention, the specific details can be found in the description of the system embodiment of the present invention, and will not be repeated here.

[0130] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle-cloud interaction-based intelligent networked vehicle safety brain system, characterized in that, The application relates to a vehicle-end safety brain system and a cloud-end safety brain system, wherein the vehicle-end safety brain system and the cloud-end safety brain system are connected with an intelligent networked vehicle original system, and the vehicle-end safety brain system and the cloud-end safety brain system are connected with each other; the intelligent networked vehicle original system comprises a vehicle networking system, a vehicle automatic driving system and a vehicle execution system; the vehicle-end safety brain system is used for acquiring vehicle safety information from the vehicle networking system in real time, identifying expected functional safety risks and functional safety risks, and determining whether to perform emergency disposal according to the identification results; if yes, emergency disposal information is sent to the vehicle execution system to perform emergency control on the automatic driving vehicle; the expected functional safety problem is a hazard and risk caused by insufficient functions of automatic driving system perception, decision and control in a complex traffic scene, and a hazard and risk caused by reasonable human misuse; the functional safety problem is a problem caused by system failure; the cloud-end safety brain system is used for acquiring fault information, automatic driving performance data and driver operation behavior data of the automatic driving vehicle from the vehicle networking system and the vehicle-end safety brain system, performing single-vehicle safety analysis on the automatic driving vehicle, performing multi-vehicle safety analysis on the same type of automatic driving vehicle, and further performing comprehensive safety monitoring on the automatic driving vehicle; the vehicle-end safety brain system comprises a risk identification module, a safety and defect analysis module and an emergency disposal module; the risk identification module is connected with the original vehicle system and is used for acquiring vehicle safety information from the original vehicle system to identify expected functional safety risks and functional safety risks; the vehicle safety information comprises a self-vehicle state vector and a surrounding traffic participant state vector for scene risk assessment, operation data for monitoring driver human-computer interaction behavior, and state signals of sensors, communication and controllers of the vehicle automatic driving system; the risk identification module comprises a vehicle driving risk identification module, a human-computer interaction monitoring module and a vehicle functional safety risk identification module; the safety and defect analysis module is used for performing fault analysis and tracing according to the real-time identification results of the risk identification module, performing comprehensive risk value calculation and grading, and determining whether to perform emergency disposal according to the grading results; if yes, an emergency disposal strategy is formulated based on the vehicle safety information and the grading results; the emergency disposal module is used for generating emergency disposal information according to the emergency disposal strategy formulated by the safety and defect analysis module, sending the emergency disposal information to the vehicle execution system, and performing emergency control on the automatic driving vehicle; the cloud-end safety brain system comprises a digital twin model, a vehicle safety monitoring module and a safety situation awareness and defect determination module; the digital twin model is used for building physical models of various types of automatic driving vehicles, constructing digital twin virtual scenes by using collected surrounding environment, vehicle state and driving behavior data, and performing simulation analysis on various types of automatic driving vehicles; ​ ​ ​ ​ ​ ​ ​ ​ 2. The system of claim 1, wherein, ​ ​ The vehicle safety monitoring module is configured to perform single-vehicle safety analysis on the autonomous vehicle based on the digital twin model, the fault information, and state performance data of each subsystem of the vehicle, and perform multi-vehicle safety analysis on the autonomous vehicles of the same model based on fault information, autonomous driving performance data, and driving behavior data of the driver of each autonomous vehicle of the same model. The safety situation awareness and defect determination module is configured to evaluate and grade the operation safety risk of each autonomous vehicle based on the single-vehicle safety analysis result and the multi-vehicle safety analysis result.

3. The system of claim 1, wherein, The vehicle-end safety brain system further comprises a data encryption module configured to encrypt data required by the cloud-end safety brain system and transmit the encrypted data to the cloud-end safety brain system. The cloud-end safety brain system further comprises a data analysis module configured to analyze the encrypted data transmitted by the vehicle-end safety brain system.

4. The system of claim 1, wherein, The vehicle driving risk identification module is configured to evaluate a driving risk value in the current traffic scenario based on a risk model, a state vector of the ego vehicle, and state vectors of surrounding traffic participants. The human-machine interaction monitoring module is configured to monitor operation data of the driver's human-machine interaction behavior to identify misuse or improper operation behavior of the driver and determine a personnel misuse factor. The vehicle functional safety risk identification module is configured to detect state signals of sensors, communication, and controllers of the vehicle autonomous driving system to determine a fault result and a functional safety risk value.

5. The system of claim 4, wherein, The driving risk value is evaluated in the following manner: A risk indicator value ITTC based on a collision time is determined as follows: wherein, is the lateral distance between the ego vehicle and the other vehicle, is the longitudinal distance between the ego vehicle and the other vehicle, is the width of the ego vehicle, is the length of the ego vehicle, is the relative speed of the ego vehicle and the other vehicle in the lateral direction, is the relative speed of the ego vehicle and the other vehicle in the longitudinal direction, is the relative acceleration of the ego vehicle and the other vehicle in the lateral direction, is the relative acceleration of the ego vehicle and the other vehicle in the longitudinal direction, is the influence coefficient of the relative acceleration of the ego vehicle and the other vehicle. A correlation coefficient of a ratio of a speed at which a vehicle travels and a road speed limit as a travel risk value , and a calculation formula is In the formula, is a vehicle speed of the own vehicle, is a road speed limit; A driving risk value r in the current traffic scenario is determined as follows: 。 6. The system of claim 2, wherein, The module for multi-vehicle safety analysis in the vehicle safety monitoring module comprises a batch defect identification module, an autonomous driving performance evaluation module, and a driving behavior analysis module. The batch defect identification module is configured to identify vehicle batch defects based on fault information of each autonomous vehicle of the same model and production times of each autonomous vehicle of the same model. The autonomous driving performance evaluation module is configured to evaluate the autonomous driving performance of the vehicle of the same model based on autonomous driving performance data of each autonomous vehicle of the same model and a scoring model. The driving behavior analysis module is configured to determine abnormal states and unsafe behaviors of the driver based on operation behavior data of each driving group of each autonomous vehicle of the same model.

7. The system of claim 6, wherein, When performing the evaluation of the autonomous driving performance of the vehicle of the same model based on the autonomous driving performance data of each autonomous vehicle of the same model and the scoring model, the autonomous driving performance evaluation module is specifically configured to: For each research period, the following is performed: Obtain autonomous driving performance data in the research period and divide the autonomous driving performance data in the research period into a plurality of research periods; wherein the autonomous driving performance data includes lateral and longitudinal acceleration of autonomous driving, driving personnel takeover duration, and number of dangerous events, and the dangerous event refers to a fault of the autonomous driving system or a collision of the vehicle; Determine a performance score of each research period based on the lateral and longitudinal acceleration and the driving personnel takeover duration in each research period. Based on the performance score of each research cycle and the number of dangerous events occurring in the research period, the comprehensive score of the autonomous driving performance in the research period is calculated.

8. The system of claim 7, wherein, The comprehensive score of the autonomous driving performance in the research period is evaluated by the following way: wherein is the overall automated driving performance score for the study period, is the performance score for the i-th study cycle, is the number of study cycles within the study period, is the number of hazardous events occurring within the study period. 9.A method for intelligent vehicle safety monitoring based on the system of any one of claims 1-8, characterized in that, It comprises: The vehicle-end safety brain system obtains vehicle safety information from the Internet of Vehicles system in real time to identify expected functional safety risks and functional safety risks, and determines whether to perform emergency disposal according to the identification result, if yes, the generated emergency disposal information is sent to the vehicle execution system to perform emergency control of the autonomous vehicle; The cloud-end safety brain system obtains fault information, autonomous driving performance data and driver operation behavior data of the autonomous vehicle from the Internet of Vehicles system and the vehicle-end safety brain system to perform single-vehicle safety analysis on the autonomous vehicle, multi-vehicle safety analysis on the same type of autonomous vehicle, and further comprehensive safety monitoring on the autonomous vehicle.

Images