Man-machine co-driving safety defense method and system based on digital twin intrusion detection

Abstract

The invention discloses a man-machine co-driving safety defense method and system based on digital twinning intrusion detection, and relates to the technical field of intelligent driving safety, and the method comprises the following steps: obtaining real driving data, mapping the driving data generated by a pre-established vehicle digital twinning model to a digital space, performing real-time mapping and comparison on the driving data and real driving data to judge whether the driving data and the real driving data conform to the standard or not, and marking the data which do not conform to the standard as known abnormal data; the driving data comprises a speed, a steering wheel angle and a brake signal; the method comprises the steps of automatically detecting abnormal points distributed in real driving data based on an Isolation Forest unsupervised learning method to obtain unknown abnormal data, fusing known abnormal data and the unknown abnormal data to generate an abnormal level label, and triggering defense measures according to the abnormal level label, the exception level comprises a light exception, a medium exception and a serious exception.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving safety, specifically to a human-machine co-driving safety defense method and system based on digital twin intrusion detection. Background Art

[0002] With the development of autonomous driving technology, the human-machine co-driving mode has been gradually popularized. However, the existing safety defense means mainly rely on traditional sensor data and lack the effective detection and defense capabilities against abnormal behaviors such as human misoperations, malicious attacks, and remote hijackings. For example, human misoperations, such as accidentally stepping on the accelerator or making a sharp turn, may lead to traffic accidents; malicious intrusions, as reported: more than 1.4 million Skoda Superb III models were found to have 12 security vulnerabilities, which may lead to unauthorized access; in addition, it may also be subject to sensor spoofing attacks, such as GPS spoofing and camera interference, which may lead to incorrect system decisions.

[0003] The reasons are as follows. On the one hand, traditional intrusion detection systems (IDS) are mainly used for network security and their applications in the intelligent driving environment are still relatively limited. Moreover, traditional intrusion detection systems (IDS) are based on static rule matching or abnormal traffic detection, but have weak recognition capabilities for new types of attacks and are difficult to adapt to the constantly changing driving environment. On the other hand, intelligent connected vehicles rely on sensors such as cameras, radars, and LiDAR to perceive the surrounding environment. However, attackers can use sensor spoofing technology to make the vehicle perceive incorrect environmental information, thus threatening the safety of autonomous driving. Summary of the Invention

[0004] To solve the deficiencies mentioned in the above background art, the purpose of the present invention is to provide a human-machine co-driving safety defense method and system based on digital twin intrusion detection.

[0005] In the first aspect, the purpose of the present invention can be achieved through the following technical solutions: A human-machine co-driving safety defense method based on digital twin intrusion detection, the method comprising the following steps:

[0006] Obtain real driving data, map the driving data generated by a pre-established vehicle digital twin model to the digital space, perform real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard, and mark the data that does not meet the standard as known abnormal data; the driving data includes speed, steering wheel angle, and braking signal;

[0007] The unsupervised learning method based on Isolation Forest automatically detects the outliers distributed in the real driving data to obtain unknown abnormal data, fuses the known abnormal data and the unknown abnormal data to generate abnormal level labels, and triggers defense measures according to the abnormal level labels. Among them, the abnormal levels include mild abnormality, moderate abnormality, and severe abnormality.

[0008] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the pre-established vehicle digital twin model is constructed based on driver behavior data, vehicle state data, and environmental perception data;

[0009] The driver behavior data includes: steering wheel angle, accelerator, brake, pedal pressure, driver eye movement trajectory, and fatigue state. The vehicle state data includes: speed, acceleration, yaw rate, and lane deviation. The environmental perception data includes GPS position, camera video, and LIDAR point cloud.

[0010] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the collection of the driver behavior data, vehicle state data, and environmental perception data is performed by deploying a sensor group at the vehicle end, including a CAN bus, an IMU inertial unit, a binocular camera, and a 64-line lidar;

[0011] The driver eye movement trajectory is obtained by tracking the pupil coordinates based on the binocular camera, and the features of the steering wheel angle change rate and the accelerator pedal gradient are extracted by using a sliding window algorithm, so as to obtain the steering wheel angle and the pedal pressure.

[0012] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the determination process of performing real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard:

[0013] Based on the difference calculation between the driving data and the real driving data, if the difference exceeds the preset detection threshold, it is marked as known abnormal data, otherwise it is normal data.

[0014] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the calculation process of the difference calculation based on the driving data and the real driving data is as follows:

[0015] Speed deviation Deviation judgment formula:

[0016] Deviation = |V real -V twin | > threshold

[0017] Steering wheel angle deviation Steering_diff judgment formula:

[0018] Steering_diff = |θ real - θ twin | > threshold

[0019] Wherein, V represents the vehicle speed, θ represents the steering wheel angle, V real and θ real respectively represent the true values of the speed and the steering wheel angle obtained from the vehicle sensor, V twin and θ twin respectively represent the simulated values of the speed and the steering wheel angle predicted by the digital twin model, and threshold represents the deviation tolerance threshold.

[0020] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the process of automatically detecting the outliers distributed in the real driving data by the Isolation Forest unsupervised learning method:

[0021] Construct a feature vector from the speed v and the steering wheel angle θ in the collected real vehicle driving data:

[0022] X = [v, θ]

[0023] Construct an Isolation Forest model, for each sample point X i Generate multiple random binary trees, and calculate its outlier score S(X i ) according to the average path length h(X i ) in the tree. The formula is as follows:

[0024]

[0025] Wherein, h(X i ) represents the average path length of the sample X i in multiple Isolation Trees;

[0026] c(n) is the expected path length when the dataset size is n, which is:

[0027]

[0028] Wherein, H(i) = ln(i) + 0.5772

[0029] According to the set outlier threshold S0, if S(X i ) > S0, then the sample is determined to be an abnormal driving behavior and enters the subsequent defense strategy.

[0030] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: for the minor anomaly: triggering a warning prompt and recording the anomaly information in a log; for the moderate anomaly: automatically limiting the speed or correcting the direction to adjust the vehicle running state; for the severe anomaly: performing an emergency brake or switching to the autonomous driving mode to prevent human misoperation or remote hijacking.

[0031] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the expression of the pre-established vehicle digital twin model is as follows:

[0032]

[0033] where represents the vehicle state vector predicted by the twin model, including the speed, acceleration, and direction angle at the prediction moment;

[0034] s(t) is the vehicle state vector at the current moment;

[0035] u(t) is the current control input vector, including the throttle, brake, and steering wheel angle;

[0036] f() is the established state transition function, implemented by a physical modeling or data-driven model, and the data-driven model includes LSTM and Transformer;

[0037] ∈ is the system modeling error or noise term.

[0038] In a second aspect, to achieve the above object, the present invention discloses a human-machine co-driving safety defense system based on digital twin intrusion detection, including:

[0039] An anomaly detection module, configured to obtain real driving data, map the driving data generated by the pre-established vehicle digital twin model to the digital space, perform real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard, and mark the data that does not meet the standard as known anomaly data; the driving data includes speed, steering wheel angle, and braking signal;

[0040] A safety defense module, configured to automatically detect the anomaly points distributed in the real driving data based on the Isolation Forest unsupervised learning method to obtain unknown anomaly data, fuse the known anomaly data and the unknown anomaly data to generate an anomaly level label, and trigger a defense measure according to the anomaly level label, where the anomaly level includes minor anomaly, moderate anomaly, and severe anomaly.

[0041] In another aspect of the present invention, in order to achieve the above object, a terminal device is disclosed, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor. When the processor loads and executes the computer program, the above-mentioned human-machine co-driving safety defense method based on digital twin intrusion detection is adopted.

[0042] Advantages of the present invention:

[0043] Through digital twin modeling, the present invention creates a virtual driving environment and synchronizes physical vehicle data in real time to form physical-virtual dual detection, improving the detection accuracy. In addition, the method of this invention patent is based on a machine learning training model to detect unknown attacks, supports the detection of known and unknown attacks, and enhances the adaptive ability. Description of the drawings

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings;

[0045] Figure 1 is a schematic diagram of the method flow of the present invention;

[0046] Figure 2 is a framework diagram of the human-machine co-driving safety defense method based on digital twin intrusion detection of the present invention;

[0047] Figure 3 is a schematic diagram of the change of driving data of the present invention;

[0048] Figure 4 is a schematic diagram of the intrusion detection effect of the present invention;

[0049] Figure 5 is a schematic diagram of the system structure of the present invention. Detailed implementation manners

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0051] Embodiment 1:

[0052] As Figure 1As shown, a human-machine co-driving safety defense method based on digital twin intrusion detection, the method comprising the following steps:

[0053] S101: Obtain real driving data, map the driving data generated by a pre-established vehicle digital twin model to the digital space, perform real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard, and mark the data that does not meet the standard as known abnormal data; the driving data includes speed, steering wheel angle, and braking signal;

[0054] Collect driver behavior data and vehicle state data through in-vehicle sensors (CAN bus, IMU, GPS, camera, etc.) and synchronize them to the digital twin system. Specifically, the collected data includes: driver behavior data (steering wheel angle, throttle, brake, fatigue state); vehicle operation data (speed, acceleration, yaw rate, lane deviation); sensor data (GPS position, camera video, LIDAR point cloud). The data is transmitted with low latency through Kafka or MQTT and stored in the ELK (Elasticsearch, Logstash, Kibana) platform for analysis. Next, according to the collected data, create vehicle and driver digital twin models, which include three small models: First, the driver behavior model: Use LSTM / Transformer to train the normal driving mode and predict future behavior; Second, the vehicle physical model: Use dynamic simulation (such as CARLA, SUMO) to simulate the vehicle motion state; Third, the environment perception model: Integrate LIDAR / camera data to provide high-precision environment modeling. In this way, real-time data updates are realized, and synchronization with the physical vehicle behavior is ensured, thereby realizing dynamic adjustment and virtual-real comparison.

[0055] The pre-established vehicle digital twin model is constructed based on driver behavior data, vehicle state data, and environment perception data;

[0056] The driver behavior data includes: steering wheel angle, throttle, brake, pedal pressure, driver eye movement trajectory, and fatigue state, the vehicle state data includes: speed, acceleration, yaw rate, and lane deviation, and the environment perception data includes GPS position, camera video, and LIDAR point cloud.

[0057] The expression of the pre-established vehicle digital twin model is as follows:

[0058]

[0059] Wherein, represents the vehicle state vector predicted by the twin model, including speed, acceleration, direction angle, etc. at the prediction moment;

[0060] s(t) is the vehicle state vector at the current moment;

[0061] u(t) is the current control input vector, including throttle, brake, steering wheel angle, etc.;

[0062] f() is the established state transition function, which can be implemented by physical modeling or data-driven models (such as LSTM, Transformer);

[0063] ∈ is the system modeling error or noise term.

[0064] Specifically, the solution of the present invention will be further elaborated through the following embodiments: Multi-source data collection and data preprocessing

[0065] Deploy a sensor group at the vehicle end, including CAN bus, IMU inertial unit, binocular camera and 64-line lidar, and continuously collect the following three types of data: Driver behavior data includes steering wheel angle with an accuracy of ±0.5°, the range of pedal pressure is 0 - 1000 kPa, and the driver's eye movement trajectory is tracked based on the pupil coordinates tracked by the infrared camera. Vehicle state data includes vehicle speed, yaw rate and battery temperature. Environmental perception data includes lidar point cloud and camera image. The sliding window algorithm is used to extract 12-dimensional features such as the change rate of the steering wheel angle and the throttle pedal gradient. Filter out outliers, such as discarding the frame data when the steering wheel angle mutation exceeds 200 degrees / second.

[0066] Twin model construction

[0067] Use the Transformer-XL neural network to process the behavior data, predict the driving behavior in the next 3 seconds, and establish a driver behavior model. Build an 18-degree-of-freedom dynamics model based on the CARLA simulation engine to calculate the vehicle motion state in real time. Use a multi-modal Transformer network to fuse the lidar point cloud and camera image to generate a 3D environmental grid map with an accuracy of 5 cm.

[0068] Real-time synchronization mechanism

[0069] Use the CAN FD bus to transmit control data with a delay of less than 5 ms; the perception data is transmitted through the 5G uRLLC channel with a delay of less than 15 ms. When the yaw rate deviation between the physical vehicle and the digital twin exceeds 0.5 rad / s and lasts for 200 ms, trigger the federated learning mechanism to dynamically update the twin model parameters.

[0070] Digital twin-driven intrusion detection (DT-IDS)

[0071] Based on digital twin comparison, a two-layer detection mechanism is adopted, namely rule-based and machine learning-based, to analyze whether there are abnormal or attack behaviors. The first layer: Rule-based detection. Threshold detection is set: speed magnitude and steering wheel conversion angle thresholds are set; Twin data comparison is used: if the real vehicle behavior is not equal to the predicted twin behavior, it is marked as abnormal; CAN data monitoring is combined: such as detecting abnormal ECU instructions and mismatches between brake signals and sensor data. The second layer: Machine learning IDS detection: Isolation Forest is used to detect unknown attacks; XGBoost / Random Forest is used to detect known attack patterns; LSTM / Transformer is combined to predict driving behaviors and detect abnormal situations deviating from the normal trajectory. Next, abnormal behaviors are classified, which are divided into three categories: mild, moderate, and severe abnormalities. The specific classification criteria are as follows: driver's misoperation (accidentally stepping on the accelerator, sudden braking) → mild abnormality; sensor spoofing attack (GPS interference, camera distortion) → moderate abnormality; CAN bus attack or remote hijacking (malicious instruction control) → severe abnormality.

[0072] The calculation process of calculating the difference between driving data and real driving data is as follows:

[0073] Speed deviation Deviation judgment formula:

[0074] Deviation = |V real -V twin | > threshold

[0075] Steering wheel angle deviation Steering_diff judgment formula:

[0076] Steering_diff = |θ real -θ twin | > threshold

[0077] Among them, V represents the vehicle speed, θ represents the steering wheel angle, V real and θ real respectively represent the real values of speed and steering wheel angle obtained from vehicle sensors, V twin and θ twin respectively represent the simulated values of speed and steering wheel angle predicted by the digital twin model, and threshold represents the deviation tolerance threshold.

[0078] Specifically, the solution of the present invention will be further elaborated through embodiments below:

[0079] Rule Detection and Digital Twin Comparison: The detection threshold is set using vehicle kinematics principles and sensor data, and behavior prediction and comparison are performed through the digital twin model. In terms of speed monitoring, if the vehicle speed exceeds 120 km / h or is lower than 5 km / h, it is marked as abnormal; in the analysis of steering wheel operations, if the instantaneous steering angle exceeds ±45°, it is determined to be an abnormal sharp turn. At the same time, combined with the time series matching algorithm, the difference degree between the actual driving operation and the digital twin predicted trajectory is calculated. If the continuous deviation exceeds three standard deviations, an alarm is triggered.

[0080] Machine Learning-Based Anomaly Detection: For known attacks, XGBoost or Random Forest training models are used to identify common attack patterns, such as malicious acceleration control, hard braking, or abnormal gear shifting; for unknown attacks, the Isolation Forest unsupervised learning method is adopted to automatically detect abnormal points in the data distribution and identify potential security threats. To further improve the prediction ability, deep learning models such as LSTM or Transformer are used to analyze driving behavior based on time series. By comparing the deviation between the predicted value and the actual driving data, abnormal situations are detected.

[0081] The rationality of sensor data is ensured through physical rule verification. During vehicle driving, its motion state is affected by various physical factors, such as road adhesion coefficient, load status, and slope. To accurately identify anomalies, the detection threshold needs to be dynamically adjusted according to the current road conditions. Considering that a heavier vehicle load will affect the reasonable range of the steering angle due to inertia changes, the system also adjusts the abnormal detection standard for the steering wheel angle according to the load status.

[0082] A cascaded detection model is adopted to improve the detection accuracy. To quickly screen suspicious data, first, an efficient anomaly detection method is used to preliminarily filter the data. For example, if the speed and steering angle show sudden changes within a certain time period, and the change amplitude far exceeds the normal historical data range, then this data point is marked as suspicious data and sent to a more complex deep learning model for further analysis. In the second stage, the system uses a deep neural network for temporal context verification, that is, not only considering whether a single data point is abnormal but also analyzing the change trend of the data over a period of time. If a certain abnormal situation persists, or the change pattern of the data in the time series significantly deviates from the previous normal driving behavior, the system will determine that this data point belongs to a true anomaly, thereby reducing the false alarm rate and improving the overall detection accuracy.

[0083] To enhance the defense ability against adversarial attacks, a noise injection module is added. In some attack scenarios, attackers may use specific means to forge seemingly normal data to bypass the detection system. For example, in a CAN bus attack, an attacker may forge the data of the speed sensor so that it still appears to be within the normal range, while in fact, the true speed of the vehicle has become abnormal. To solve this problem, a certain amount of random noise is introduced during the model training stage to simulate the forged data that the attacker may impose, and the model is made to learn how to distinguish normal data from maliciously forged data through reinforcement learning.

[0084] S102: Automatically detect the outliers distributed in the real driving data based on the Isolation Forest unsupervised learning method to obtain unknown abnormal data, fuse the known abnormal data and the unknown abnormal data to generate an anomaly level label, and trigger a defense measure according to the anomaly level label, where the anomaly level includes mild anomaly, moderate anomaly, and severe anomaly.

[0085] The process of automatically detecting the outliers distributed in the real driving data based on the Isolation Forest unsupervised learning method:

[0086] (1) Construct a feature vector X = [v, θ] from the speed v and the steering wheel angle θ in the collected real vehicle driving data

[0087] (2) Construct an Isolation Forest model and for each sample point X i Generate multiple random binary trees, and calculate its anomaly score S(X i ) according to the average path length h(X i ) of the sample in the tree. The formula is as follows:

[0088]

[0089] where h(X i ) represents the average path length of the sample X i in multiple Isolation Trees;

[0090] c(n) is the expected path length when the dataset size is n, approximately:

[0091]

[0092] where H(i) = ln(i) + 0.5772

[0093] (3) According to the set anomaly threshold S0, if S(X i ) > S0, then determine that the sample is an abnormal driving behavior and enter the subsequent defense strategy module.

[0094] Digital Twin-Driven Security Defense Strategy

[0095] Based on the intrusion detection results, this system adopts an intelligent defense strategy to adjust the vehicle control logic according to the anomaly level and prevent security threats. For minor anomalies: trigger voice or visual warnings, such as reminding the driver to adjust operations, and record the anomaly log; for moderate anomalies: intelligent assisted driving adjustment, reducing the vehicle speed or correcting the steering wheel angle, and initiate driver takeover detection, such as requiring the driver to perform interactive verification, simulate the defense strategy in the Digital Twin system, and recommend the optimal driving mode; for severe anomalies: immediately switch to the autonomous driving mode, restrict the driver's input, perform emergency braking to prevent the vehicle from getting out of control, and initiate remote security monitoring, send the security event log to the cloud, and notify the management center.

[0096] Specifically, the solution of the present invention will be further elaborated through embodiments as follows: when a minor anomaly is detected, the system will trigger a voice or visual warning to remind the driver to pay attention to the driving state and recommend adjusting the operation mode. At the same time, the system will automatically record the anomaly log, storing in detail the relevant sensor data, anomaly category, and timestamp for subsequent analysis and optimization of the defense strategy.

[0097] In the case of detecting a moderate anomaly, the system will activate the intelligent assisted driving function to appropriately adjust the vehicle control parameters, such as reducing the vehicle speed or slightly adjusting the steering wheel angle to improve driving safety. In addition, the system will initiate a driver takeover detection mechanism to evaluate the driver's state and response ability through interactive verification methods (such as touch screen confirmation, voice response, or slight steering wheel rotation). At the same time, the system will simulate the defense strategy in real time in the Digital Twin environment and recommend the optimal driving mode in combination with historical data and machine learning algorithms to minimize potential safety hazards.

[0098] For severe anomaly events, the system will immediately switch to the autonomous driving mode, restrict the driver's input authority, and perform emergency braking operations to prevent the vehicle from getting out of control. At the same time, the system will initiate a remote security monitoring mechanism to send the security event log to the cloud in real time and notify the management center for further security response and decision-making. In addition, the system can also combine vehicle networking technology to send warning messages to surrounding vehicles and road infrastructure to ensure the safety of the overall traffic environment.

[0099] Starting from data collection in the physical environment, a digital twin model is constructed through sensors and predictive data and updated in real time. The system establishes a multi-dimensional model based on driver behavior, vehicle physical characteristics, and environmental perception, and conducts normal monitoring in combination with preset rules. When encountering a cyber attack or generating abnormal data, multi-level analysis is performed through comparative analysis, machine learning classification (ML tagging), rule detection, and intrusion detection (ML_IDS), and finally abnormal information is identified and defense measures are deployed. The overall process integrates digital twin and machine learning technologies to achieve a security closed-loop from data monitoring to active defense.

[0100] To verify the effectiveness of the present patent invention, we used Isolation Forest to detect abnormal behaviors according to the trends of speed and steering wheel angle changing over time, such as Figure 3 shown. The blue line represents the normal speed change, and the red dots represent the detected abnormal behaviors, such as sudden acceleration and sudden deceleration; the green line represents the normal steering, and the red crosses represent abnormal steering, such as sharp turns or abnormal instructions. Specifically analyzed, the speed and steering wheel angle are represented as a feature vector as:

[0101] X = {(Speed, Steering_Angle)}

[0102] The Isolation Forest model is used to train an anomaly detector, randomly partition the data, and generate anomaly scores. The data is judged according to the anomaly scores and the set threshold (0.65), and the points with scores exceeding the threshold are marked as abnormal behaviors. Specifically, Isolation Forest judges through the following features: 1. Sudden braking: The speed suddenly drops significantly; 2. Sudden acceleration: The speed rises sharply in a short time; 3. Sharp turn: The steering wheel angle changes sharply in a short time. When the anomaly score exceeds the threshold of 0.65, it is marked as abnormal. Finally, the detected abnormal behaviors are visualized, and the abnormal behaviors are classified into three categories: mild, moderate, and severe according to the change ranges of speed and steering wheel angle.

[0103] As Figure 4 shown, normal driving (blue): These points are mainly concentrated in the positions where the vehicle speed is relatively low (20 - 80 km / h) and the steering wheel angle is close to 0. Most of the points of normal driving behaviors are in this area, indicating that when the vehicle speed is low, the change of the steering wheel angle is small.

[0104] Mild anomaly (green): These points are widely scattered, the vehicle speed can range from 20 km / h to 120 km / h, and the steering wheel angle also shows a large change. The green points may be misoperations by the driver, such as slight sudden braking, acceleration, or steering.

[0105] Moderate anomaly (red): These points are fewer and mostly distributed at higher vehicle speeds (above 90 km / h), and the steering wheel angle also shows significant fluctuations. The red points may represent situations of sensor or input errors, such as GPS signal errors or sensor-misleading behaviors.

[0106] Severe anomaly (purple): These points are very few and usually appear when both the vehicle speed and the steering wheel angle are in extreme positions, indicating possible attack behaviors, such as CAN bus attacks or remote control of the vehicle.

[0107] Embodiment 2: As Figure 5 shown, to achieve the above object, the present invention discloses a human-machine co-driving safety defense system based on digital twin intrusion detection, including:

[0108] Anomaly detection module 11, which is used to obtain real driving data, map the driving data generated by a pre-established vehicle digital twin model to the digital space, perform real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard, and mark the data that does not meet the standard as known anomaly data; the driving data includes speed, steering wheel angle, and braking signal;

[0109] Safety defense module 12, which is used to automatically detect the abnormal points distributed in the real driving data based on the Isolation Forest unsupervised learning method to obtain unknown anomaly data, fuse the known anomaly data and the unknown anomaly data to generate an anomaly level label, and trigger a defense measure according to the anomaly level label, where the anomaly level includes mild anomaly, moderate anomaly, and severe anomaly.

[0110] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is used to implement one or more instructions, specifically used to load and execute one or more instructions in the computer storage medium to implement the above method.

[0111] It should be further noted that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the above method. The storage medium may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electro-magnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0112] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0113] The above shows and describes the basic principles, main features, and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure will have various changes and improvements, and all these changes and improvements fall within the scope of the present disclosure claimed.

Claims

1. A human-machine co-driving safety defense method based on digital twin intrusion detection, characterized in that, The method includes the following steps: Obtain real driving data, map the driving data generated by a pre-established vehicle digital twin model to the digital space, perform real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard, and mark the data that does not meet the standard as known abnormal data; the driving data includes speed, steering wheel angle, and braking signal; Automatically detect abnormal points distributed in the real driving data based on the Isolation Forest unsupervised learning method to obtain unknown abnormal data, fuse the known abnormal data and the unknown abnormal data to generate an abnormal level label, and trigger a defense measure according to the abnormal level label, where the abnormal level includes mild abnormality, moderate abnormality, and severe abnormality.

2. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 1, wherein, The pre-established vehicle digital twin model is constructed based on driver behavior data, vehicle state data, and environmental perception data; The driver behavior data includes: steering wheel angle, throttle, brake, pedal pressure, driver eye movement trajectory, and fatigue state, and the vehicle state data includes: speed, acceleration, yaw rate, and lane deviation, and the environmental perception data includes GPS position, camera video, and LIDAR point cloud.

3. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 2, wherein, The acquisition of the driver behavior data, vehicle state data, and environmental perception data is carried out by deploying a sensor group at the vehicle end, including a CAN bus, an IMU inertial unit, a binocular camera, and a 64-line lidar; Obtain the driver eye movement trajectory based on tracking the pupil coordinates by the binocular camera, and use a sliding window algorithm to extract the features of the steering wheel angle change rate and the throttle pedal gradient, so as to obtain the steering wheel angle and pedal pressure.

4. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 1, wherein, The determination process of performing real-time mapping and comparison between the driving data and the real driving data to determine whether it meets the standard: Perform a difference calculation based on the driving data and the real driving data. If the difference exceeds a preset detection threshold, it is marked as known abnormal data, otherwise it is normal data.

5. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 4, characterized in that, The calculation process of performing a difference calculation based on the driving data and the real driving data is as follows: Speed deviation Deviation judgment formula: Deviation=|V real -V twin |>threshold Steering wheel angle deviation Steering_diff judgment formula: Steering_diff = |θ real -θ twin | > threshold Among them, V represents the vehicle speed, θ represents the steering wheel angle, V real and θ real respectively represent the true values of the speed and steering wheel angle obtained from the vehicle sensors, V twin and θ twin respectively represent the simulated values of the speed and steering wheel angle predicted by the digital twin model, and threshold represents the deviation tolerance threshold.

6. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 1, wherein, The process of automatically detecting abnormal points distributed in the real driving data based on the Isolation Forest unsupervised learning method: Construct a feature vector with speed v and steering wheel rotation angle θ in the collected real vehicle driving data: X = [v, θ Construct an Isolation Forest model for each sample point X i Generate multiple random binary trees and calculate its anomaly score S(X i ) according to the average path length h(X i ) in the tree. The formula is as follows: Among them, h(X i ) represents the average path length of the sample X i in multiple Isolation Trees; c(n) is the expected path length when the dataset size is n, and it is: where H(i) = ln(i) + 0.5772 According to the set anomaly threshold S0, if S(X i ) > S0, the sample is determined to be an abnormal driving behavior and enters the subsequent defense strategy.

7. The method for human-machine co-driving safety defense based on digital twin intrusion detection according to claim 1, wherein For the mild abnormality: trigger a warning prompt and record the abnormal information in the log; for the moderate abnormality: automatically limit the speed or correct the direction to adjust the vehicle running state; for the severe abnormality: perform emergency braking or switch to the autonomous driving mode to prevent human misoperation or remote hijacking.

8. The human-machine co-driving safety defense method based on digital twin intrusion detection according to claim 2, wherein, The expression of the pre-established vehicle digital twin model is as follows: Among them, represents the vehicle state vector predicted by the twin model, including the speed, acceleration, and direction angle at the prediction moment; s(t) is the vehicle state vector at the current moment; u(t) is the current control input vector, including throttle, brake, and steering wheel angle; f() is the established state transition function, implemented by a physical modeling or data-driven model, and the data-driven model includes LSTM and Transformer; ∈ is the system modeling error or noise term.

9. A human-machine co-driving safety defense system based on digital twin intrusion detection, characterized in that, It includes: Anomaly detection module, which is used to obtain real driving data, map the driving data generated by the pre-established vehicle digital twin model to the digital space, compare the driving data with the real driving data in real time to determine whether it meets the standard, and mark the data that does not meet the standard as known abnormal data; the driving data includes speed, steering wheel angle, and braking signal; Safety defense module, which is used to automatically detect the abnormal points distributed in the real driving data based on the Isolation Forest unsupervised learning method to obtain unknown abnormal data, fuse the known abnormal data and the unknown abnormal data to generate an anomaly level label, and trigger a defense measure according to the anomaly level label, where the anomaly level includes mild anomaly, moderate anomaly, and severe anomaly.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it adopts the human-machine co-driving safety defense method based on digital twin intrusion detection described in any one of claims 1 to 8.

Images