Vehicle intelligent safety management and control system

Abstract

The invention relates to the field of vehicle safety management and control, in particular to a vehicle intelligent safety management and control system which comprises a multi-mode perception fusion subsystem for collecting driver and vehicle running state information, a digital twinborn prediction subsystem for constructing a vehicle core component digital twinborn body based on the vehicle running state information and predicting faults, and a vehicle safety management and control subsystem for monitoring the vehicle core component digital twinborn body. An edge computing unit of the edge cloud cooperative computing subsystem executes real-time data processing and local model reasoning, a cloud service cluster executes global model training and data analysis, and the hierarchical active safety control subsystem implements hierarchical risk response according to driver state information and a fault prediction result. The full-link security protection subsystem ensures data security; the multi-modal sensing fusion subsystem is provided with a plurality of sensors and a fusion processing module, the digital twin prediction subsystem is provided with a geometric modeling module and the like, the side cloud cooperative computing subsystem implements a dynamic task unloading strategy, and the hierarchical active safety control subsystem is provided with four risk levels and adopts encrypted communication. The technical effects of comprehensively and accurately performing intelligent safety management and control on the vehicle, ensuring data safety and effectively reducing the vehicle operation risk are achieved.

Description

Technical Field

[0001] This application relates to the field of vehicle safety management, and in particular to an intelligent vehicle safety management system. Background Technology

[0002] With the rapid increase in the number of commercial vehicles and the continuous improvement in operational intensity, vehicle safety issues have become increasingly prominent. Vehicle accidents have become a core bottleneck seriously restricting the high-quality development of the transportation industry. This not only poses a huge threat to people's lives and property but also affects the operational efficiency and sustainable development of the entire industry. Therefore, the development of vehicle safety management technology is crucial to ensuring the safety and smooth operation of transportation.

[0003] In existing technologies, various methods have been employed to address vehicle safety management issues. For driver status monitoring, some systems rely on a single visible light camera to capture facial features to determine the driver's condition; others use millimeter-wave radar as a standalone application, monitoring single vital signs such as heart rate to aid in assessment. Regarding fault warnings, traditional OBD diagnostic systems trigger alarms based on thresholds, while some predictive maintenance solutions rely on centralized cloud computing. For safety response, existing systems mostly employ a passive "alarm notification" mode, using voice or light alerts to indicate risks; even some solutions with control capabilities lack tiered response mechanisms. In terms of computing architecture, complex AI models struggle to run on in-vehicle edge devices, while fully cloud-based architectures are significantly affected by network bandwidth fluctuations, and existing edge-cloud collaborative solutions lack dynamic task offloading strategies.

[0004] However, these existing technologies have significant drawbacks. Multi-source sensing lacks accuracy and environmental adaptability; single-modal systems have a high average false alarm rate, failing to meet the high reliability requirements of commercial scenarios. Fault warnings are delayed and lack predictive capabilities; traditional solutions struggle to identify latent faults, and the advance warning time is insufficient to support planned maintenance. Control responses are passive, lacking a safety closed loop; traditional systems mostly respond passively, lacking linkage and hierarchical response mechanisms with the vehicle control bus, easily leading to secondary accidents. The computing architecture is rigid and resource adaptability is poor; traditional architectures struggle to balance real-time performance and computing power, making the system prone to failure during network outages, unable to meet the continuous operation requirements of commercial vehicles. Summary of the Invention

[0005] The purpose of this application is to overcome the above-mentioned technical problems and provide a vehicle intelligent safety management and control system. A vehicle intelligent safety management and control system includes: The multimodal perception fusion subsystem is configured to collect driver status information and vehicle operating status information; A digital twin prediction subsystem, coupled to the multimodal perception fusion subsystem, is used to construct digital twins of core vehicle components based on the vehicle operating status information and predict their faults. The edge-cloud collaborative computing subsystem includes an edge computing unit and a cloud service cluster. The edge computing unit performs real-time data processing and local model inference, while the cloud service cluster performs global model training and data analysis. A tiered active safety control subsystem, coupled to the multimodal perception fusion subsystem and the edge-cloud collaborative computing subsystem, implements tiered risk responses based on the driver status information and fault prediction results; And a full-link security protection subsystem to ensure the security of data collection, transmission, storage and use.

[0006] By adopting the above technical solutions, the multimodal perception fusion subsystem can collect driver and vehicle status information, solving the problem of poor environmental adaptability of single sensors and significantly improving perception accuracy; the digital twin prediction subsystem can construct digital twins of core vehicle components and predict faults, achieving a breakthrough in forward-looking fault prediction; the edge-cloud collaborative computing subsystem implements a dynamic task offloading strategy through the division of labor between edge computing units and cloud service clusters, ensuring that the response latency at the edge meets the standard, satisfying the real-time requirements of vehicle control, and can also operate independently locally when the network is interrupted; the hierarchical active safety control subsystem can implement hierarchical risk response based on status information and fault prediction results, forming a safety protection closed loop and reducing the accident rate; and the full-link safety protection subsystem ensures the security of data throughout the entire process, meeting automotive-grade information security requirements.

[0007] Preferably, the multimodal sensing fusion subsystem includes: At least one infrared camera is deployed on top of the vehicle's windshield to capture key points on the driver's face; At least one millimeter-wave radar is installed inside the steering column to monitor micro-movement signals in the chest cavity; At least one triaxial accelerometer is installed at the steering shaft connection to measure steering wheel angle and vibration; An oil spectrum sensor, installed in the oil circulation line, is used to detect the concentration of metal wear particles; It also includes a tire pressure and temperature sensor, embedded in the tire valve, for real-time collection of tire pressure and temperature data.

[0008] By adopting the above technical solution, the multimodal perception fusion subsystem uses multiple sensors to collect data from different dimensions: an infrared camera captures key points on the driver's face, millimeter-wave radar monitors micro-movement signals in the chest cavity, a three-axis accelerometer measures steering wheel angle and vibration, an oil spectrum sensor detects the concentration of metal abrasive particles, and a tire pressure and temperature sensor collects tire pressure and temperature data. This enables accurate perception of the driver's state and the vehicle's operating state, solves the problem of poor environmental adaptability of single sensors, significantly improves perception accuracy, significantly increases the accuracy of driver state monitoring, and significantly reduces the false alarm rate in low-light environments at night.

[0009] Preferably, the multimodal perception fusion subsystem further includes a fusion processing module, which is configured to: Extract eyelid closure degree, heart rate coefficient of variation, and steering wheel angle entropy value; The driver state index is generated by summing the features using dynamically adjusted weighting coefficients. The weighting coefficients are adjusted based on environmental parameters, including light intensity and driving speed.

[0010] By adopting the above technical solution, the fusion processing module of the multimodal perception fusion subsystem can extract eyelid closure degree, heart rate variability coefficient and steering wheel angle entropy value. By dynamically adjusting the weighting coefficients, these features are weighted and summed to generate a driver state index. The weighting coefficients can be adjusted according to environmental parameters such as light intensity and driving speed, which can realize accurate perception of driver state and vehicle operating state, solve the problem of poor environmental adaptability of single sensors, greatly improve the accuracy of driver state monitoring, and significantly reduce the false alarm rate in complex lighting and other environments.

[0011] Preferably, the dynamic adjustment includes: When the driving speed is detected to exceed the threshold, the weight of the steering wheel angle entropy value is increased; When the light intensity is below the threshold, the weight of the heart rate variability coefficient is increased.

[0012] By adopting the above technical solutions, when the driving speed exceeds the threshold, the weight of the steering wheel angle entropy value is increased, which can more accurately reflect the impact of the driver's operation behavior on his / her state when driving at high speed, making the calculation of the driver's state index more in line with the actual driving situation; when the light intensity is below the threshold, the weight of the heart rate variability coefficient is increased, which can improve the attention to the driver's vital signs in low light environment, make up for the problem that infrared cameras may not be able to extract facial features in low light, thereby improving the accuracy and reliability of driver state monitoring and reducing the false alarm rate.

[0013] Preferably, the digital twin prediction subsystem includes: The geometric modeling module is used to create 3D models of the vehicle's core components. The physical degradation model module uses dedicated models to correlate sensor data for different components. It also includes a prediction module that integrates physical models and data-driven models to output remaining lifetime and failure probability.

[0014] By adopting the above technical solution, a three-dimensional model of the vehicle's core components is constructed. Dedicated models are used to associate sensor data with different components. The physical model and data-driven model are integrated to output the remaining life and failure probability. This enables early warning of latent faults and prediction of remaining life, solving the problem of insufficient predictability of traditional solutions. The engine fault warning advance is 56±8 days, and the tire abnormality warning advance is 32±5 days, which is 5-8 times higher than the traditional OBD system. The bearing early fault identification rate is ≥92%.

[0015] Preferably, the physical degradation model module includes: For bearings, a wavelet packet-1DCNN hybrid model is used to extract vibration signal features; For tires, an improved Arrhenius equation is used in conjunction with tire pressure, tire temperature, and mileage to predict aging.

[0016] By adopting the above technical solutions, the wavelet packet-1DCNN hybrid model is used to extract vibration signal features for bearings, which can improve the feature extraction capability and make the bearing fault identification accuracy ≥96.8% and the early fault (wear amount <0.1mm) identification rate ≥92%. For tires, the improved Arrhenius equation is used in combination with tire pressure, tire temperature and mileage for aging prediction, which can achieve more accurate tire aging prediction and achieve a tire abnormality warning advance of 32±5 days.

[0017] Preferably, the edge-cloud collaborative computing subsystem implements a dynamic task offloading strategy, and the edge computing unit determines whether the task is processed locally or executed in the cloud based on the current resource load and network quality.

[0018] By adopting the above technical solutions, the edge-cloud collaborative computing subsystem implements a dynamic task offloading strategy. The edge computing unit determines the execution location of the task based on the current resource load and network quality, which can balance real-time performance and computing power, solve the rigidity problem of traditional architecture, and make the edge response latency ≤50ms, meeting the real-time requirements of vehicle control. It also has the ability to run independently locally for ≥72 hours when the network is interrupted, making it suitable for continuous operation scenarios of commercial vehicles.

[0019] Preferably, the cloud service cluster collaborates with edge nodes through a federated learning mechanism, transmitting only gradient data and not the original data to update model parameters.

[0020] By adopting the above technical solutions, data privacy can be protected, bandwidth usage can be reduced, and global model optimization in the cloud can be achieved.

[0021] Preferably, the graded active safety control subsystem has four risk levels, corresponding to different response actions, including speed limit, seat vibration, audible and visual alarm, and emergency braking command, and the control command is sent to the vehicle ECU via encrypted communication.

[0022] By adopting the above technical solution, four risk levels are set and corresponding to different response actions, hierarchical active control is realized. Appropriate measures can be taken according to different risk levels, thereby reducing the accident rate and improving the collision avoidance rate in emergency scenarios. Control commands are sent to the vehicle ECU through encrypted communication, which ensures the security of control command transmission, prevents commands from being tampered with or interfered with, and improves the reliability and security of the system.

[0023] Preferably, the encrypted communication uses the AES-GCM algorithm and adds a timestamp to prevent replay attacks. By adopting the above technical solution, in the hierarchical active safety control subsystem of the vehicle intelligent safety management and control system, the encrypted communication of control commands using the AES-GCM algorithm and adding a timestamp can effectively prevent replay attacks, ensure the security of control command transmission, thereby improving the security and reliability of the entire system and better realizing the safety management and control of vehicles.

[0024] In summary, this application includes at least one of the following beneficial technical effects: The multimodal perception fusion subsystem uses multiple sensors to collect information and generates a driver state index by fusing features through dynamic weighting coefficients, which greatly improves the accuracy of driver state monitoring and solves the problems of poor environmental adaptability of single sensors and insufficient accuracy of multi-source perception. The digital twin prediction subsystem constructs digital twins of core vehicle components, integrates physical models and data-driven models to predict faults, and significantly increases the advance warning of faults in core components such as engines, bearings, and tires, solving the problems of delayed fault warnings and lack of predictive capabilities in traditional solutions. The edge-cloud collaborative computing subsystem implements a dynamic task offloading strategy, resulting in low response latency for edge computing units. It can run independently locally when the network is interrupted, balancing real-time performance and computing power, and solving the problems of rigidity and poor resource adaptability in traditional computing architectures. The hierarchical active safety control subsystem sets four risk levels and corresponding response actions. It sends control commands through encrypted communication, forming a safety protection closed loop, which solves the problems of passive control response and lack of safety closed loop in traditional solutions. The end-to-end security protection subsystem ensures the security of data collection, transmission, storage, and use, meets automotive-grade information security requirements, and resolves the risks of data leakage and tampering. Attached Figure Description

[0025] Figure 1 : Workflow diagram of the digital twin prediction subsystem.

[0026] Figure 2 : Architecture diagram of edge-cloud collaborative computing subsystem.

[0027] Figure 3 : Schematic diagram of the hierarchical structure of the end-to-end security protection subsystem.

[0028] Figure 4 : Schematic diagram of the deployment locations of multimodal sensors on vehicles. Detailed Implementation

[0029] The technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments are only possible technical implementations of the present invention, but are not limited thereto. Other embodiments obtained by those skilled in the art in conjunction with the embodiments of the present invention without creative effort are also within the protection scope of the present invention.

[0030] This application mainly adopts a multi-module collaborative approach to achieve a vehicle safety full-scenario management and control solution, which improves the perception accuracy, fault prediction capability, real-time response capability and safety protection level of vehicle safety management and control. The following is a further detailed description of this application.

[0031] Example 1: The vehicle intelligent safety management and control system provided in this application includes a multimodal perception fusion subsystem, a digital twin prediction subsystem, an edge-cloud collaborative computing subsystem, a hierarchical active safety control subsystem, and a full-link safety protection subsystem. The multimodal perception fusion subsystem collects driver status information and vehicle operating status information. The digital twin prediction subsystem constructs digital twins of core vehicle components based on vehicle operating status information and predicts their faults. The edge computing unit of the edge-cloud collaborative computing subsystem performs real-time data processing and local model inference, while the cloud service cluster performs global model training and data analysis. The hierarchical active safety control subsystem implements hierarchical risk responses based on driver status information and fault prediction results. The full-link safety protection subsystem ensures the security of data collection, transmission, storage, and use, achieving the effect of realizing full-scenario vehicle safety management and control and improving the performance of various aspects of vehicle safety management and control. This is because the various subsystems work together: the multimodal perception fusion subsystem provides accurate data foundation, the digital twin prediction subsystem detects potential faults in advance, the edge-cloud collaborative computing subsystem balances computing resources, the hierarchical active safety control subsystem responds to risks in a timely manner, and the full-link security protection subsystem ensures data security. Together, they make vehicle safety management more comprehensive, efficient, and reliable.

[0032] Specifically, the multimodal perception fusion subsystem includes at least one infrared camera, at least one millimeter-wave radar, at least one triaxial accelerometer, an oil spectral sensor, and a tire pressure and temperature sensor. The infrared camera can be a product like the FLIR Blackfly S BFS-U3-16S2C-CS, deployed at the top center of the vehicle's windshield. The lens angle is typically tilted downwards at a certain angle, such as 15°, to ensure coverage of the driver's face. The infrared camera connects to the edge computing unit via a USB 3.0 interface, typically powered by 12V, with relatively low power consumption, usually ≤5W. It features 1920×1080@30fps resolution, an 850nm wavelength, and an F1.8 aperture, primarily used to capture key facial points of the driver. These key points can be further used to extract relevant information, achieving 68 facial key point detection, eyelid closure recognition, and a nighttime recognition accuracy of ≥98.2%. Of course, other cameras with similar functions can also be used, as long as they can accurately capture facial key points.

[0033] The millimeter-wave radar TIAWR1843BOOST can be selected and installed inside the steering column, with the radar antenna facing the driver's chest cavity and a certain horizontal offset requirement, generally ≤±3°. It communicates with the edge computing unit via an SPI interface, operates at a frequency of 77GHz, has a bandwidth of 1.5GHz, a ranging accuracy of ±0.1m, a sampling rate of 60Hz, and a detection range between 0.5-5m. It is used to monitor chest cavity micro-motion signals and extract respiratory / heart rate, with a heart rate measurement error of ±0.5 beats / minute. Other millimeter-wave radars with the ability to accurately monitor chest cavity micro-motion signals can also be used as alternatives.

[0034] A triaxial accelerometer, such as the ADI ADXL355, is mounted at the steering shaft connection and connects to the edge computing unit via an I2C interface. It has a sampling rate of 10kHz, a range of ±200g, and a resolution of 16 bits. It is used to measure steering wheel angle and jitter, with an angle measurement error of ±0.1°. Triaxial accelerometers from different brands but with similar performance can also play the same role in this system.

[0035] The HORIBA MEXA-ONE oil spectrometer sensor can be selected and installed at the oil circulation line outlet. It outputs the concentration of metal abrasive particles (iron, copper, aluminum) via a CANFD interface. It features 16-channel UV-VIS, a detection range of 200-800nm, and a detection resolution of 1ppm, used for detecting the concentration of metal abrasive particles. Other sensors capable of accurately detecting the concentration of metal abrasive particles in engine oil can also be used as alternatives.

[0036] Tire pressure and temperature sensors, such as the Schrader 33500, are embedded inside the tire valve stem and communicate with the vehicle's gateway via Bluetooth 5.0. They have a sampling rate of 1Hz, a pressure accuracy of ±0.02 Bar, a pressure measurement range of 0-8 Bar, and a temperature measurement range of -40℃ to 125℃. They can collect tire pressure and temperature data in real time, with a pressure measurement error of ≤1.5%. Any sensor that can stably and accurately collect tire pressure and temperature data can be used as a replacement.

[0037] These sensors, combined, can collect driver and vehicle operating status information from multiple dimensions, providing comprehensive and accurate data for subsequent analysis and decision-making. Different types of sensors target different monitoring objects, acquiring the necessary information through their unique working principles, complementing each other to improve the accuracy and reliability of perception. For example, infrared cameras capture the driver's facial state from a visual perspective, millimeter-wave radar monitors the driver's vital signs from a physiological signal perspective, a three-axis accelerometer obtains steering wheel motion information from an operational behavior perspective, and oil pressure and temperature sensors monitor the vehicle's oil and tire conditions, respectively.

[0038] Specifically, the multimodal perception fusion subsystem further includes a fusion processing module.

[0039] The fusion processing module is configured to extract eyelid closure degree, heart rate coefficient of variation, and steering wheel angle entropy value. It processes facial key point data captured by the infrared camera, calculates the ratio of vertical to horizontal distance of the eyelids to obtain the eyelid closure degree; performs IQ signal demodulation, distance FFT, and constant false alarm rate detection on the chest cavity micro-motion signals monitored by millimeter-wave radar, and extracts the heart rate coefficient of variation; and performs histogram statistics on the steering angle data collected by the triaxial accelerometer, calculating the steering wheel angle entropy value using the information entropy formula.

[0040] Then, the fusion processing module sums these features using dynamically adjusted weighting coefficients to generate a driver state index. The weighting coefficients are adjusted based on environmental parameters, including light intensity and driving speed. Initial values ​​are 0.46, 0.31, and 0.23. When the driving speed exceeds a threshold, such as 80 km / h, the weight of the steering wheel angle entropy is increased to 0.35, because driver input is more critical at high speeds, and this input more accurately reflects driving status. When light intensity is below a threshold, such as 500 lux, the weight of the heart rate variability coefficient is increased, because the accuracy of infrared cameras may be affected in low-light conditions, while the heart rate variability coefficient monitored by millimeter-wave radar more reliably reflects the driver's physiological state. This dynamic adjustment of weighting coefficients allows the generated driver state index to more accurately reflect the driver's true state in different environments.

[0041] Specifically, the digital twin prediction subsystem includes a geometric modeling module, a physical degradation model module, and a prediction module.

[0042] The geometric modeling module can create 3D models of core vehicle components using software such as SolidWorks. High-precision 3D models of core components like engine blocks, bearing inner and outer rings, and tire bodies can be created with an accuracy of up to 0.01mm. These models are then imported into Unity3D to build a visual twin scene. This method allows for a direct and intuitive display of the structure and state of the vehicle's core components.

[0043] The physical degradation model module employs dedicated models to correlate sensor data for different components. For bearings, a wavelet packet-1DCNN hybrid model is used to extract vibration signal features. This model introduces a custom wavelet pooling layer, which better preserves frequency domain features and improves feature extraction capabilities. The model structure includes a custom wavelet pooling layer, a 1DCNN feature extraction part, and a classification output layer. Model training uses full lifecycle data from 200 bearings (each bearing includes 18 months of continuous monitoring data), achieving a fault identification accuracy of ≥96.8% and an early fault (wear <0.1mm) identification rate of ≥92%. For tires, an improved Arrhenius equation is used in conjunction with tire pressure, tire temperature, and mileage for aging prediction. The formula is: dtdH=A⋅e−Ea / (R(T+273.15))⋅(1−σ / σmax)n, where H is the rate of change of rubber hardness (indirectly calculated through tire pressure), and A is the pre-exponential factor (measured value 5.2×10⁻). 4 Ea is the activation energy (measured value of 38.6 kJ / mol for rubber materials), R is the gas constant (8.314 J / (mol・K)), T is the tire temperature (°C), σ is the shear stress (MPa, calculated from tire pressure and load), and n is the material degradation coefficient (measured value of 3.2 ± 0.15, dynamically corrected with mileage). This equation introduces a shear stress correction term, which can more accurately reflect the aging of the tire.

[0044] The prediction module integrates the remaining lifespan and failure probability outputs of the physical model and the data-driven model. It achieves multi-source information fusion through DS evidence theory, combining the physical model output with the data-driven model prediction results, comprehensively considering various factors to more accurately predict the remaining lifespan and failure probability of core vehicle components. Furthermore, it calibrates the physical model parameters using real-time sensor data (vibration, temperature, pressure) and employs a Kalman filter algorithm to correct model prediction biases, with a calibration cycle of 10 minutes.

[0045] Specifically, the edge-cloud collaborative computing subsystem includes edge computing units and cloud service clusters.

[0046] The edge computing unit can utilize the NVIDIA Jetson AGX Xavier development kit, installed in the passenger-side toolbox in the driver's cab. It connects to the vehicle's ECU via a CAN bus interface (KvaserLeafProv2), powered by a DC-DC converter (input 12-24V, output 5V / 10A), and equipped with an active cooling fan (operating temperature -40℃~105℃). Its software stack includes TensorRT 8.6, ROS2 Foxy, and CANoe, used for real-time data processing, local model inference, and emergency control decisions, with response latency controlled to ≤50ms. The edge computing unit determines whether tasks are processed locally or executed in the cloud based on current resource load and network quality, implementing a dynamic task offloading strategy. When local resources are sufficient and network quality is poor, local processing is chosen; when local resources are scarce and network quality is good, tasks are offloaded to the cloud. Edge nodes dynamically allocate computing tasks based on real-time resource load and network quality, using a multi-factor weighted decision algorithm to determine task scheduling. If the local processing score is higher than the cloud processing score, local processing is performed. This involves checking if the model is loaded; if not, it is loaded dynamically. When memory is insufficient, low-priority models are unloaded, and a TensorRT optimized model (FP16 accuracy, 2.3x faster inference speed) is loaded and warmed up. If the cloud processing score is higher, the data is encrypted and compressed before being transmitted to the cloud. The cloud service cluster uses a Kubernetes cluster with 8 nodes. Each node is configured with an Intel Xeon Gold 6348 CPU (28 cores), 256GB RAM, and an NVIDIA V100 GPU (32GB VRAM), and 10TB SSDs (RAID5) for storage. Its software stack includes PyTorch 2.0, Kafka, and InfluxDB for global model training and data analysis, with batch processing latency ≤10s. The cloud service cluster collaborates with edge nodes through a federated learning mechanism, using a federated averaging algorithm to achieve global model optimization in the cloud. Edge nodes only upload model gradients, not raw data, protecting data privacy. The process involves initializing the global model in the cloud and distributing it to each edge node. The edge nodes train the model based on local data, calculate the gradient, and add differential privacy noise (ε=1.5). The cloud collects the gradients of each node, aggregates them by weight according to the amount of data in each node, updates the global model in the cloud, and incrementally distributes the model parameters to the edge nodes (only transmitting the changed weights, reducing bandwidth usage by 70%).

[0047] Specifically, the graded active safety control subsystem is set up with four risk levels, each corresponding to a different response action.

[0048] When the system is in a safe state (driver status index between 0 and 0.6) with no risk of malfunction, it performs normal monitoring and data recording. When in a warning state (DSI value between 0.6 and 0.7) with low risk, the instrument panel red light flashes, the HUD displays "Please drive carefully," and the data is uploaded to the cloud. When in a high-risk state (DSI value between 0.7 and 0.85) with medium risk, the seat vibrates (200Hz for 2 seconds), a voice prompt says "Please concentrate," and the maximum speed is limited to 80km / h. When in an emergency state (DSI value between 0.85 and 1.0) with high risk, the electronic throttle is limited (maximum opening 50%), the air conditioning activates high-speed mode (18℃, maximum fan speed), a 120dB audible and visual alarm is issued, and emergency contacts are automatically dialed. Control commands are sent to the vehicle ECU via encrypted communication using the AES-GCM algorithm, with timestamps added to prevent replay attacks.

[0049] The secure transmission of control commands is achieved by using an "AES-GCM encryption + identity authentication" mechanism. The global key (burned into the device at the factory and updated periodically via the cloud) is encrypted by concatenating the system timestamp with the control command and sending the CAN frame after attaching an authentication tag.

[0050] Specifically, the end-to-end security protection subsystem is used to ensure the security of data collection, transmission, storage, and use.

[0051] At the data acquisition layer, facial images captured by infrared cameras are stored locally after being encrypted with lightweight AES-128, and the key is managed by the Hardware Security Module (HSM). At the transport layer, edge nodes communicate with the cloud using the TLS 1.3 protocol, with key negotiation using the P-384 elliptic curve algorithm, and LZ4 compression (compression ratio ≥ 3:1) performed before data transmission. At the storage layer, cloud-based time-series data (sensor data) is stored using transparent encryption, while unstructured data (images, videos) is stored using fragmented encryption and blockchain notarization, with notarization information including data hash, acquisition time, and device ID. At the usage layer, model inference results use a digital signature mechanism to prevent tampering, with the signature algorithm being RSA-3072. Meanwhile, the system also has device identity authentication and access control mechanisms, including factory authentication (each device has a unique built-in device certificate (based on the X.509 standard), which is signed by a CA institution before leaving the factory and the certificate validity is verified at startup), access authentication (when edge nodes access the cloud, they use "certificate + dynamic token" two-factor authentication, the token is valid for 1 hour and is updated hourly through a secure channel), and access control hierarchy (using the RBAC (role-based access control) model, divided into three levels: administrator (full permissions), maintenance personnel (device management permissions), and monitoring personnel (data viewing permissions), and permission changes require two-factor authentication), as well as firmware security and self-healing mechanisms, such as using differential upgrade + signature verification mechanism for firmware upgrade (upgrade packages are signed with RSA-3072, and support rollback to the previous version in case of interruption during the upgrade process), automatically switching to redundancy schemes when core sensors or modules fail (such as enhancing the vital sign monitoring weight of millimeter-wave radar when infrared cameras fail), and triggering maintenance alarms at the same time.

[0052] The implementation principle of this embodiment is as follows: The vehicle intelligent safety management and control system of this embodiment collects driver and vehicle status information from multiple dimensions through a multimodal perception fusion subsystem, improving the accuracy of perception and environmental adaptability. The digital twin prediction subsystem can detect hidden faults in core vehicle components in advance, providing support for planned maintenance. The edge-cloud collaborative computing subsystem balances real-time performance and computing power through dynamic task offloading and federated learning mechanisms, adapting to different network and resource conditions. The hierarchical active safety control subsystem takes corresponding measures in a timely manner according to the risk level, forming a safety closed loop. The full-link security protection subsystem ensures data security and prevents data leakage and tampering. The overall system solves the defects of existing vehicle safety management and control technologies in terms of multi-source perception, fault early warning, control response, and computing architecture, improves the overall performance of vehicle safety management and control, meets the high reliability requirements of commercial vehicles, and has significant practicality and innovation, representing a significant improvement and enhancement compared to existing technologies.

[0053] Example 2 differs from the above examples in that the sensor components in the multimodal perception fusion subsystem employ a different combination. The multimodal perception fusion subsystem in this partial alternative includes at least one high-definition visible light camera, at least one ultrasonic radar, at least one angle sensor, an oil and gas analysis sensor, and a tire wear monitoring sensor.

[0054] High-definition visible light cameras are deployed in suitable locations inside the vehicle to clearly capture the driver's facial expressions and movements, aiding in the assessment of the driver's condition. They connect to the edge computing unit via a video interface, offering high resolution and frame rate. Ultrasonic radar is installed in key areas of the vehicle, such as near the driver, to detect subtle body movements, acquiring relevant data through the principle of ultrasonic wave reflection. Angle sensors are mounted on the steering mechanism to accurately measure the steering wheel's rotation angle and speed. Oil and gas analysis sensors are installed in the oil and gas system to analyze the composition and quality of the oil and gas, detecting potential faults. Tire wear monitoring sensors are embedded inside the tires to monitor tire wear in real time.

[0055] The implementation principle of this embodiment is as follows: By employing different sensor combinations, this embodiment can also achieve the collection of driver status information and vehicle operating status information. Different types of sensors have their own unique advantages. High-definition visible light cameras can provide rich visual information, ultrasonic radar has good effects in close-range micro-motion monitoring, angle sensors can more accurately measure the steering wheel angle, and oil and gas analysis sensors and tire wear monitoring sensors can monitor specific parts of the vehicle. This alternative solution may have better adaptability in certain specific scenarios, providing more options for vehicle safety management, further improving the system's flexibility and reliability, and providing a beneficial supplement and expansion to existing vehicle safety management solutions.

[0056] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A vehicle intelligent safety management and control system, characterized in that, include: The multimodal perception fusion subsystem is configured to collect driver status information and vehicle operating status information; A digital twin prediction subsystem, coupled to the multimodal perception fusion subsystem, is used to construct digital twins of core vehicle components based on the vehicle operating status information and predict their faults. The edge-cloud collaborative computing subsystem includes an edge computing unit and a cloud service cluster. The edge computing unit performs real-time data processing and local model inference, while the cloud service cluster performs global model training and data analysis. A tiered active safety control subsystem, coupled to the multimodal perception fusion subsystem and the edge-cloud collaborative computing subsystem, implements tiered risk responses based on the driver status information and fault prediction results; And a full-link security protection subsystem to ensure the security of data collection, transmission, storage and use.

2. The vehicle intelligent safety management and control system according to claim 1, characterized in that, The multimodal perception fusion subsystem includes: at least one infrared camera deployed on the top of the vehicle's windshield to capture key points on the driver's face; at least one millimeter-wave radar installed inside the steering column to monitor micro-movement signals in the chest cavity; at least one triaxial accelerometer installed at the steering shaft connection to measure steering wheel angle and vibration; an oil spectral sensor installed in the oil circulation line to detect the concentration of metal abrasive particles; and a tire pressure and temperature sensor embedded in the tire valve to collect tire pressure and temperature data in real time.

3. The vehicle intelligent safety management and control system according to claim 2, characterized in that, The multimodal perception fusion subsystem further includes a fusion processing module, which is configured to: extract eyelid closure degree, heart rate variability coefficient and steering wheel angle entropy value; and generate a driver state index by weighting and summing the features through dynamically adjusted weighting coefficients; wherein the weighting coefficients are adjusted according to environmental parameters including light intensity and driving speed.

4. The vehicle intelligent safety management and control system according to claim 3, characterized in that, The dynamic adjustment includes: increasing the weight of the steering wheel angle entropy value when the driving speed exceeds the threshold; and increasing the weight of the heart rate variability coefficient when the light intensity is below the threshold.

5. The vehicle intelligent safety management and control system according to claim 1, characterized in that, The digital twin prediction subsystem includes: a geometric modeling module, which establishes a 3D model of the vehicle's core components; a physical degradation model module, which uses a dedicated model to associate sensor data with different components; and a prediction module, which integrates the physical model and the data-driven model to output the remaining lifespan and failure probability.

6. The vehicle intelligent safety management and control system according to claim 5, characterized in that, The physical degradation model module includes: for bearings, a wavelet packet-1DCNN hybrid model is used to extract vibration signal features; for tires, an improved Arrhenius equation is used in combination with tire pressure, tire temperature and mileage to predict aging.

7. The vehicle intelligent safety management and control system according to claim 1, characterized in that, The edge-cloud collaborative computing subsystem implements a dynamic task offloading strategy, and the edge computing unit determines whether the task is processed locally or executed in the cloud based on the current resource load and network quality.

8. The vehicle intelligent safety management and control system according to claim 7, characterized in that, The cloud service cluster collaborates with edge nodes through a federated learning mechanism, transmitting only gradient data without transmitting the original data, to update model parameters.

9. The vehicle intelligent safety management and control system according to claim 1, characterized in that, The graded active safety control subsystem is set with four risk levels, corresponding to different response actions, including speed limit, seat vibration, audible and visual alarms, and emergency braking commands. The control commands are sent to the vehicle ECU via encrypted communication.

10. The vehicle intelligent safety management and control system according to claim 9, characterized in that, The encrypted communication uses the AES-GCM algorithm and adds a timestamp to prevent replay attacks.

Images