Vehicle abnormal driving state recognition method and system fusing car-road cooperation information

Abstract

This invention relates to the field of automotive driving safety technology, specifically to a method and system for identifying abnormal vehicle driving states by integrating vehicle-road cooperative information. The system can collect vehicle dynamic parameters and onboard perception information in real time, receive roadside cooperative information, construct a scene, and generate a local dynamic map with the vehicle as the origin. It extracts three sets of feature vectors and concatenates them with basic vehicle state information to generate a comprehensive feature vector. The comprehensive feature vector and roadside cooperative information are input into a trained deep learning model to obtain an anomaly probability score. Based on the anomaly probability score within a sliding time window, a judgment threshold is dynamically calculated, thereby identifying the vehicle's driving state. The technical solution presented in this invention integrates vehicle information and roadside information, enabling the detection of abnormal behavior and a better understanding of the external environmental motivations behind the anomalies. This allows for the differentiation between reasonable risk avoidance and dangerous driving, significantly reducing the false alarm rate. Furthermore, the dynamic threshold can adapt to different driving styles and complex road scenarios.

Description

Technical Field

[0001] This invention relates to the field of automotive driving safety technology, specifically to a method and system for identifying abnormal vehicle driving states by integrating vehicle-road cooperative information. Background Technology

[0002] Currently, vehicle abnormal driving status identification mainly relies on onboard sensors and vehicle-based algorithms. The mainstream technical solutions can be broadly divided into two categories: First, rule-based models monitor dynamic parameters such as vehicle lateral and longitudinal acceleration, yaw rate, and steering wheel angle using preset thresholds. An alarm is triggered when a parameter exceeds a safety threshold. This method is simple and direct, but rigid rules cannot cope with complex and ever-changing real-world traffic scenarios, and are prone to generating numerous false alarms due to drivers' personalized driving styles or reasonably aggressive evasive maneuvers. Second, statistical machine learning models learn normal driving patterns from historical driving data and classify deviations from these patterns as abnormal. While this method is an improvement over rule-based models, its feature extraction capabilities are limited, and it heavily relies on the vehicle's own perception information. Model performance is poor in extreme cases not covered by training data, and its generalization ability has bottlenecks.

[0003] Existing vehicle-road cooperative technologies are mostly applied to macro-level scenarios such as forward collision warning and intersection traffic assistance. The global roadside perception information they provide has not been deeply mined and used in the diagnosis and judgment of the vehicle's own micro-level abnormal states. The information from the vehicle and the roadside is still simply superimposed rather than deeply integrated. Existing technologies mainly have the following deep-seated technical problems:

[0004] The identification decision is based solely on single-vehicle sensor data, lacking roadside global information from a bird's-eye view as the decision context. This makes the system unable to distinguish "why it is abnormal." For example, it cannot distinguish whether a vehicle's emergency lane change is to avoid an obstacle (a reasonable and necessary safety behavior) or due to driver fatigue and distraction (a real danger and abnormality), thus causing the system to misjudge.

[0005] The judgment thresholds are mostly statically set and cannot be adaptively adjusted according to real-time traffic scenarios or drivers' personalized habits. They lack flexibility, and once deployed, their recognition capabilities are fixed. They cannot use the new data constantly generated in actual operation to evolve and continuously optimize themselves, making it difficult to cope with new abnormal driving scenarios that will continue to emerge in the future.

[0006] Based on the above problems, there is a need for a method and system for identifying abnormal vehicle driving states that integrates vehicle-road cooperative information, in order to solve the aforementioned problems. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a method and system for identifying abnormal vehicle driving status by integrating vehicle-road cooperative information, so as to solve to some extent the problems of single data source and lack of flexibility in judgment threshold in the prior art.

[0008] According to a first aspect of the present invention, a method for identifying abnormal vehicle driving states by fusing vehicle-road cooperative information is provided, comprising:

[0009] The system collects vehicle dynamic parameters and onboard perception information in real time; receives roadside cooperative information through a communication protocol; and builds a scene based on the vehicle dynamic parameters, onboard perception information, and roadside cooperative information, and integrates them to generate a local dynamic map with the vehicle as the origin.

[0010] Three sets of feature vectors are extracted from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; the three sets of feature vectors are concatenated with the vehicle's basic state information to generate a comprehensive feature vector.

[0011] A deep learning model is constructed, and supervised training is performed on the deep learning model using historical normal and abnormal driving data. The integrated feature vector and roadside cooperative information are input into the trained deep learning model to obtain the abnormality probability score.

[0012] The judgment threshold is dynamically calculated based on the anomaly probability score in the sliding time window; the driving status of the vehicle is identified based on the anomaly probability score and the judgment threshold.

[0013] Preferably, the vehicle dynamic parameters are collected in real time, including: collecting the vehicle dynamic parameters in real time through the vehicle-mounted sensing unit, the vehicle dynamic parameters including: steering wheel angle, throttle opening, brake pedal travel, yaw rate, longitudinal acceleration and lateral acceleration.

[0014] Receiving roadside cooperative information via a communication protocol includes: receiving roadside cooperative information via an onboard communication unit; the roadside cooperative information is generated by roadside intelligent infrastructure deployed at key road nodes; the roadside intelligent infrastructure senses structured environmental data within a preset radius centered on the vehicle and generates roadside cooperative information.

[0015] Preferably, a scene is constructed based on the vehicle dynamic parameters and the roadside cooperative information, and a local dynamic map with the vehicle as the origin is generated, including:

[0016] The vehicle dynamic parameters, on-board perception information and roadside coordination information are spatiotemporally synchronized and stored in a cache queue;

[0017] Based on the data in the cache queue, the short-term state of the vehicle and traffic participants is predicted using extended Kalman filtering, and the roadside sensing targets and vehicle-mounted sensing targets are matched and associated using the nearest neighbor data association algorithm.

[0018] The data in the roadside coordinate system is transformed to the vehicle coordinate system with the vehicle's center of gravity as the origin using a coordinate transformation matrix.

[0019] Based on all data in the vehicle coordinate system with the vehicle's center of gravity as the origin, vectorized information of all traffic participants is obtained;

[0020] By fusing vehicle dynamic parameters with vectorized information from all traffic participants, a local dynamic map is generated.

[0021] Preferably, three sets of feature vectors are extracted from the local dynamic map, including:

[0022] Based on the vehicle dynamic parameters in the local dynamic map, the steering wheel angle entropy, accelerator / brake pedal interlock relationship, and deviation between longitudinal acceleration and powertrain output power are calculated, and the above calculation results are integrated into the vehicle's own handling feature vector.

[0023] The collected steering wheel angle and vehicle speed are input into a pre-established three-degree-of-freedom vehicle dynamics model to obtain theoretical yaw rate and lateral acceleration, which are used as theoretical values. The actual measured yaw rate and lateral acceleration in the local dynamic map are used as actual values. Based on the actual values ​​and theoretical values, the yaw rate deviation, lateral acceleration deviation, and estimated values ​​of the center of gravity sideslip angle are calculated. The above calculation results are integrated into a vehicle dynamic response feature vector.

[0024] Based on the local dynamic map, the ratio of the actual headway between the vehicle and surrounding vehicles to their expected headway, the lateral offset of the vehicle's trajectory from the lane centerline and its rate of change, and the TTC index of whether the vehicle can legally pass the stop line within the current traffic light cycle are calculated. The above calculation results are integrated into a vehicle-road cooperative interaction feature vector.

[0025] Preferably, constructing a deep learning model includes:

[0026] A two-layer attention mechanism deep learning model is constructed, with the first layer being a spatiotemporal self-attention encoding layer and the second layer being a vehicle-road cooperative cross-attention decoding layer;

[0027] The first layer is used to receive the comprehensive feature vector arranged in time series, calculate the dependency between features at different time steps within the sequence through a self-attention mechanism, capture transient and short-term anomaly patterns, and output the vehicle state feature sequence.

[0028] The second layer fuses the vehicle's state feature sequence with roadside cooperative information to obtain an interpretive feature representation.

[0029] Preferably, the judgment threshold is dynamically calculated based on the anomaly probability score within the sliding time window, including:

[0030] The sliding time window stores the latest time series of the vehicle's anomaly probability score;

[0031] Calculate the mean μ and standard deviation σ of the anomaly probability scores in the time series, and then calculate the judgment threshold θ for the current time step using the following formula:

[0032] θ=μ+k×σ

[0033] Where k is the adjustment coefficient.

[0034] Preferably, identifying the vehicle's driving state based on the anomaly probability score and the determination threshold further includes:

[0035] Set the initial anomaly integral;

[0036] Calculate the difference between the instantaneous anomaly probability score and the dynamic threshold. If the anomaly probability score is greater than the dynamic threshold, add the difference to the anomaly integral. If the anomaly probability score is less than or equal to the dynamic threshold, reduce the value of the anomaly integral according to a preset attenuation factor.

[0037] The vehicle's driving status at the current moment is determined by the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the value of the anomaly integral at that moment.

[0038] Preferably, the method further includes:

[0039] The vehicle's driving status at the current moment is classified based on the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the anomaly integral value at that moment: normal driving, Level 1 warning, Level 2 alarm, and Level 3 danger.

[0040] According to a second aspect of the present invention, a vehicle abnormal driving state identification system integrating vehicle-road cooperative information is provided, comprising:

[0041] The vehicle-mounted multi-source data acquisition module is used to collect the vehicle's dynamic parameters and vehicle-mounted perception information in real time; and to receive roadside cooperative information through a communication protocol.

[0042] The spatiotemporal synchronization and map building module is used to build a scene based on the vehicle dynamic parameters, vehicle perception information and roadside cooperative information, and to fuse and generate a local dynamic map with the vehicle as the origin.

[0043] The multi-dimensional feature extraction and fusion module is used to extract three sets of feature vectors from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; and to concatenate the three sets of feature vectors with the vehicle's basic state information to generate a comprehensive feature vector.

[0044] A dual-layer attention anomaly detection module is used to construct a deep learning model and conduct supervised training on the deep learning model using historical normal and abnormal driving data; the integrated feature vector and roadside cooperative information are input into the trained deep learning model to obtain an anomaly probability score;

[0045] The dynamic threshold classification decision module is used to dynamically calculate the judgment threshold based on the anomaly probability score in the sliding time window; and to identify the driving status of the vehicle based on the anomaly probability score and the judgment threshold.

[0046] Preferably, the system further includes:

[0047] The multi-level collaborative execution feedback module is used to execute different response measures according to the vehicle's driving status, including: if the vehicle's driving status is a level one warning, a voice prompt is issued; if the vehicle's driving status is a level one alarm, the vehicle control system is triggered to access; if the vehicle's driving status is a level three danger, a safe path is planned based on roadside collaborative information, triggering the emergency takeover of the autonomous driving system.

[0048] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0049] It is understood that the technical solution presented in this invention can collect vehicle dynamic parameters and onboard perception information in real time, receive roadside cooperative information, build a scene, fuse and generate a local dynamic map with the vehicle as the origin, extract three sets of feature vectors and concatenate them with the vehicle's basic state information to generate a comprehensive feature vector; input the comprehensive feature vector and roadside cooperative information into a trained deep learning model to obtain an anomaly probability score; dynamically calculate the judgment threshold based on the anomaly probability score in the sliding time window; and thus identify the vehicle's driving state. The technical solution presented in this invention, by fusing vehicle information and roadside information, can detect abnormal behavior and better understand the external environmental motivations that cause anomalies, thereby distinguishing between reasonable risk avoidance and dangerous driving, and significantly reducing the false alarm rate; at the same time, the dynamic threshold can adapt to different driving styles and complex road scenarios.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0052] Figure 1 This is a schematic diagram illustrating the steps of a method for identifying abnormal vehicle driving states by fusing vehicle-road cooperative information, according to an exemplary embodiment.

[0053] Figure 2 This is a schematic block diagram illustrating a vehicle abnormal driving state recognition system that integrates vehicle-road cooperative information according to an exemplary embodiment. Detailed Implementation

[0054] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0055] In one embodiment, Figure 1 This is a schematic diagram illustrating the steps of a method for identifying abnormal vehicle driving states by fusing vehicle-road cooperative information, according to an exemplary embodiment. See also... Figure 1 A method for identifying abnormal vehicle driving states by integrating vehicle-road cooperative information is provided, including:

[0056] Step S11: Collect vehicle dynamic parameters and onboard perception information in real time; receive roadside cooperative information through communication protocol; build a scene based on the vehicle dynamic parameters, onboard perception information and roadside cooperative information, and fuse them to generate a local dynamic map with the vehicle as the origin.

[0057] Vehicle-mounted perception information refers to the sensor information on the vehicle itself, such as millimeter-wave radar and vision sensors.

[0058] It should be noted that the real-time collection of vehicle dynamic parameters includes: real-time collection of the vehicle's dynamic parameters through the on-board sensing unit, including: steering wheel angle, throttle opening, brake pedal travel, yaw rate, longitudinal acceleration, and lateral acceleration.

[0059] Receiving roadside cooperative information via a communication protocol includes: receiving roadside cooperative information via an onboard communication unit; the roadside cooperative information is generated by roadside intelligent infrastructure deployed at key road nodes; the roadside intelligent infrastructure senses structured environmental data within a preset radius (e.g., 300 meters) centered on the vehicle and generates roadside cooperative information. The communication protocol is, for example, the C-V2X communication protocol, and the roadside intelligent infrastructure (RSU) broadcasts the roadside cooperative information during travel.

[0060] Structured environmental data should include at least the precise trajectories, speeds, and accelerations of surrounding vehicles, pedestrians, and non-motorized vehicles, as well as traffic light status, road geometry, and estimated road surface adhesion coefficients.

[0061] It should be noted that in step S11, the scene is constructed based on the vehicle dynamic parameters and the roadside cooperative information, and a local dynamic map with the vehicle as the origin is generated, including:

[0062] Step S111: Perform spatiotemporal synchronization of the vehicle dynamic parameters, vehicle-mounted perception information, and roadside cooperative information, and store them in a cache queue.

[0063] In practice, high-precision spatiotemporal stamps are applied to the asynchronously collected data streams from the vehicle and roadside for spatiotemporal synchronization, and the data is then stored in a cache queue.

[0064] Step S112: Based on the data in the cache queue, use extended Kalman filter to predict the short-term state of the vehicle and traffic participants, and use the nearest neighbor data association algorithm to perform target association matching between roadside sensing targets and vehicle-mounted sensing targets.

[0065] The extended Kalman filter algorithm can predict the short-term state (such as the position and speed in the next 1-2 seconds) of the vehicle and surrounding traffic participants (such as surrounding vehicles and pedestrians) based on historical data, making up for the slight delay in data transmission / collection and ensuring data real-time performance.

[0066] Since roadside equipment and vehicle-mounted sensors may simultaneously detect the same target, target association matching is required. By using the nearest neighbor data association algorithm, the targets sensed by the roadside and the targets sensed by the vehicle are matched to clarify the "correspondence of the same target", avoid data duplication or conflict, and lay the foundation for subsequent fusion.

[0067] Step S113: Transform the data in the roadside coordinate system to the vehicle coordinate system with the vehicle's center of gravity as the origin using a coordinate transformation matrix.

[0068] Step S114: Based on all data in the vehicle coordinate system with the vehicle's center of gravity as the origin, obtain the vectorized information of all traffic participants. Vectorized information includes, for example, the position vector, velocity vector, and acceleration vector of a vehicle in the aforementioned vehicle coordinate system with the vehicle's center of gravity as the origin.

[0069] Step S115: Fuse the vehicle dynamic parameters with the vectorized information of all traffic participants to generate a local dynamic map. Preferably, the local dynamic map includes dynamic objects (pedestrians, vehicles) and their credibility scores.

[0070] Step S12: Extract three sets of feature vectors from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; concatenate the three sets of feature vectors with the vehicle's basic state information to generate a comprehensive feature vector. This comprehensive feature vector is used to characterize the human-vehicle-road-environment interaction state.

[0071] It should be noted that three sets of feature vectors are extracted from the local dynamic map, including:

[0072] Based on the vehicle dynamic parameters in the local dynamic map, the steering wheel angle entropy, accelerator / brake pedal interlock relationship, and deviation between longitudinal acceleration and powertrain output power are calculated, and the above calculation results are integrated into the vehicle's own handling feature vector.

[0073] The collected steering wheel angle and vehicle speed are input into a pre-established three-degree-of-freedom vehicle dynamics model to obtain the theoretical yaw rate and lateral acceleration, which are used as theoretical values. The actual measured yaw rate and lateral acceleration in the local dynamic map are used as actual values. Based on the actual values ​​and theoretical values, the yaw rate deviation, lateral acceleration deviation, and estimated values ​​of the center of gravity sideslip angle are calculated. The above calculation results are integrated into the vehicle dynamic response feature vector.

[0074] Vehicle dynamic response feature vectors are used to determine whether the actual vehicle dynamics are consistent with the driver's expected operation. For example, when the driver turns the steering wheel, the vehicle should theoretically generate a corresponding yaw rate. If the actual yaw rate deviates too much from the theoretical value, it may mean that the vehicle control system is abnormal (such as steering failure) or that there is external environmental interference (such as road slippage).

[0075] The three-degree-of-freedom vehicle dynamics model is specifically designed to simulate the normal dynamic response of a vehicle in the three dimensions of "longitudinal, lateral, and yaw".

[0076] Yaw rate deviation: Actual yaw rate - Theoretical yaw rate. The larger the deviation, the more significant the difference between the vehicle's steering response and the expected response.

[0077] Lateral acceleration deviation: Actual lateral acceleration - theoretical lateral acceleration. The larger the deviation, the more significant the difference between the vehicle's lateral motion response and the expected response.

[0078] Estimated value of center of gravity sideslip angle: Based on the deviation of yaw rate and lateral acceleration, combined with the dynamic formula of the three-degree-of-freedom model, the vehicle's center of gravity sideslip angle (the angle between the direction of the vehicle's center of gravity velocity and the longitudinal axis of the vehicle body, reflecting whether the vehicle has a tendency to sideslip, is the core indicator for judging the dynamic stability of the vehicle) is estimated.

[0079] Based on the local dynamic map, the ratio of the actual headway between the vehicle and surrounding vehicles to their expected headway, the lateral offset of the vehicle's trajectory from the lane centerline and its rate of change, and the TTC index of whether the vehicle can legally pass the stop line within the current traffic light cycle are calculated. The above calculation results are integrated into a vehicle-road cooperative interaction feature vector.

[0080] The actual headway is calculated based on the precise trajectory of the nearest neighboring vehicles and the real-time trajectory of the vehicle itself, to determine the "time distance" between the two vehicles. The expected headway is calculated by combining the "road geometry features" (such as highways and urban roads) in the roadside cooperative information with the vehicle's current speed, and pre-setting an "expected safe headway" that meets road safety standards.

[0081] Taking the car in front as an example, if the ratio is less than 1, it means that the following distance is too close (interaction abnormality); if the ratio is approximately 1, it means that the following distance is reasonable; if the ratio is too large, there may be interaction abnormalities such as low-speed lane occupation.

[0082] Preferably, when concatenating the three sets of feature vectors with the vehicle's basic state information to generate a comprehensive feature vector, the three sets of feature vectors and the vehicle's basic state information are first standardized and preprocessed. Min-Max scaling or Z-score standardization methods are used to eliminate differences in different physical dimensions and magnitudes, ensuring that each feature is in the same numerical range and avoiding certain dimensions from dominating model training due to excessively large values.

[0083] The preprocessed vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector are horizontally concatenated with the standardized vehicle basic state to form a higher-dimensional comprehensive feature vector. The comprehensive feature vector is structured into a tensor form that can be directly input into the model, which fully encapsulates the driver's maneuvering intention, the vehicle's actual dynamic response, and its interactive coupling relationship with the external environment.

[0084] Step S13: Construct a deep learning model and perform supervised training on the deep learning model using historical normal and abnormal driving data; input the integrated feature vector and roadside cooperative information into the trained deep learning model to obtain an anomaly probability score. For example, the anomaly probability score can be a score between 0 and 1.

[0085] It should be noted that building a deep learning model includes:

[0086] A dual-layer attention mechanism deep learning model is constructed, with the first layer being a spatiotemporal self-attention encoding layer and the second layer being a vehicle-road cooperative cross-attention decoding layer. This dual-layer attention mechanism deep learning model is specifically designed for anomaly identification in time-series sequences.

[0087] The first layer is used to receive the comprehensive feature vector arranged in time series, calculate the dependency between features at different time steps within the sequence through a self-attention mechanism, capture transient and short-term anomaly patterns, and output the vehicle state feature sequence.

[0088] The second layer fuses the vehicle's state feature sequence with roadside collaborative information to obtain an interpretive feature representation. This layer performs cross-comparison and weighted fusion of information. The vehicle's state feature sequence output from the self-attention encoding layer is used as the query, while the surrounding traffic environment encoding information provided by the roadside units is used as the key and value. By calculating the similarity between the key and value, the model assigns corresponding attention weights to different roadside environment information. These weights are then used to perform a weighted summation of the values, thereby generating an interpretive feature representation that deeply integrates internal state and external motivation.

[0089] Step S14: Dynamically calculate the judgment threshold based on the anomaly probability score in the sliding time window; identify the vehicle's driving status based on the anomaly probability score and the judgment threshold.

[0090] This step designs a dynamic adaptive threshold mechanism to replace the fixed threshold for a two-layer attention mechanism network deep learning model. The judgment threshold is dynamically calculated based on the distribution of the vehicle's abnormal probability score within the historical window period.

[0091] It should be noted that the judgment threshold is dynamically calculated based on the anomaly probability score within the sliding time window, including:

[0092] The sliding time window stores the latest time series of the vehicle's anomaly probability score. The deep learning model maintains a fixed-length sliding time window in real time.

[0093] Calculate the mean μ and standard deviation σ of the anomaly probability scores in the time series, and then calculate the judgment threshold θ for the current time step using the following formula:

[0094] θ=μ+k×σ

[0095] Where k is the adjustment coefficient.

[0096] This threshold effectively adapts to different driving styles and constantly changing external road conditions, ensuring that the threshold remains at a reasonable level.

[0097] Preferably, in another embodiment, an integrator with a decay factor is introduced to achieve abnormal energy accumulation. Identifying the vehicle's driving state based on the abnormality probability score and the determination threshold further includes:

[0098] Set an initial anomaly integral; calculate the difference between the instantaneous anomaly probability score and the dynamic threshold. If the anomaly probability score is greater than the dynamic threshold, add the difference to the anomaly integral; if the anomaly probability score is less than or equal to the dynamic threshold, reduce the value of the anomaly integral according to a preset attenuation factor; based on the difference between the instantaneous anomaly probability score and the dynamic threshold, and the value of the anomaly integral at that moment, determine the vehicle's driving state at the current moment.

[0099] In practice, an integrator with a decay factor is set up. When the instantaneous abnormal score exceeds the dynamic threshold, the integrator begins to accumulate the excess. If the score is below the threshold, the accumulated value decays slowly, capturing low-intensity abnormal behaviors that are not violent but persistent, thus achieving early, sensitive and stable warnings of potential risks.

[0100] This embodiment introduces the concept of abnormal energy accumulation, taking into account whether the instantaneous score exceeds the threshold, to capture continuous low-intensity abnormal behavior, and to identify the abnormal driving state of the vehicle based on the instantaneous exceedance magnitude and energy accumulation value.

[0101] It should be noted that the method also includes:

[0102] The vehicle's driving status at the current moment is classified based on the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the anomaly integral value at that moment: normal driving, Level 1 warning, Level 2 alarm, and Level 3 danger.

[0103] In practice, when an abnormal driving condition is detected, for a Level 1 warning, a mild prompt is provided to the driver via the in-vehicle HUD or voice system; for a Level 2 alarm, in addition to enhancing the intensity of the audio and visual warnings, slight intervention can be made in the vehicle control system; for a Level 3 hazard, while issuing a strong warning, the emergency takeover function of the autonomous driving system is triggered with the highest authority based on the safe path planning provided by roadside cooperative information, and the type, time, location, and environmental snapshot of the abnormal event are uploaded to the cloud traffic management platform via the V2X network, forming a regional warning information broadcast to other surrounding vehicles to achieve group collaborative safety.

[0104] In another embodiment, after determining a Level 3 anomaly, the vehicle system immediately packages the event type, precise timestamp, high-precision positioning coordinates, and key information such as the environmental snapshot captured by the roadside sensing unit at the time of the anomaly into a structured safety event data packet. The data packet is then transmitted to the cloud traffic management platform via the V2I communication link, relayed through the roadside unit, or directly via the cellular network C-V2X.

[0105] After receiving the data packet, the cloud platform parses, verifies, and geographically matches it to quickly generate a standardized collaborative perception message. This message is then broadcast to all networked roadside units within a specific geographical area surrounding the incident location via a downlink broadcast channel. The networked roadside units then broadcast the message to vehicles traveling within the area via V2I communication, enabling beyond-line-of-sight perception of dangerous situations ahead and forming a collaborative safety protection network.

[0106] This technical solution constructs a cloud-based collaborative optimization closed loop, which uses digital twin technology to conduct in-depth retrospective analysis of abnormal events and adopts OTA (Over-The-Air) updates to achieve incremental model updates, enabling the system to continuously evolve and self-optimize. At the same time, based on the V2X communication-based group security protection mechanism, abnormal events of individual vehicles are quickly transformed into regional early warning information, realizing a leap from individual vehicle intelligence to group collaborative security. This end-to-cloud collaborative architecture not only improves the driving safety level of individual vehicles, but also builds a regional proactive safety protection network through information sharing, significantly improving the overall road traffic safety coefficient.

[0107] The technical solution presented in this invention, by deeply integrating vehicle-road cooperative information and vehicle perception data, and adopting a two-layer attention mechanism network model, can use roadside global environmental information as a key decision-making basis. This enables the model not only to detect abnormal behavior, but also to understand the external environmental motivations that cause the abnormality, thereby significantly distinguishing between reasonable risk avoidance and dangerous driving, greatly reducing the false alarm rate. The introduction of dynamic adaptive threshold and abnormal energy accumulation mechanism enables the system to adapt to different driving styles and complex road scenarios, achieving early and accurate warnings for continuous low-intensity abnormal behavior, thus improving the practicality and reliability of the system.

[0108] In another embodiment, see Figure 2 A vehicle abnormal driving state recognition system integrating vehicle-road cooperative information is provided, comprising:

[0109] The vehicle-mounted multi-source data acquisition module is used to collect the vehicle's dynamic parameters and vehicle-mounted perception information in real time; and to receive roadside cooperative information through a communication protocol.

[0110] In practice, this module has the function of collecting vehicle steering wheel angle, throttle opening, brake pedal travel, yaw rate and acceleration parameters in real time through the vehicle-mounted sensing unit. At the same time, it receives structured environmental data broadcast by the roadside unit through the C-V2X communication protocol, which includes the trajectory of surrounding traffic participants, traffic light status and road features.

[0111] The spatiotemporal synchronization and map building module is used to build a scene based on the vehicle dynamic parameters, vehicle perception information and roadside cooperative information, and to generate a local dynamic map with the vehicle as the origin.

[0112] In practice, this module has the functions of timestamping multi-source asynchronous data, performing target matching through extended Kalman filtering and nearest neighbor data association algorithms, and generating a local high-precision map with the vehicle as the origin and containing dynamic target vectorization information and uncertainty measurement through coordinate system transformation.

[0113] The multi-dimensional feature extraction and fusion module is used to extract three sets of feature vectors from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; the three sets of feature vectors are concatenated with the vehicle's basic state information to generate a comprehensive feature vector.

[0114] In practice, this module has the function of calculating steering wheel angle entropy, pedal interlock relationship and other handling characteristics, yaw rate deviation and other dynamic response characteristics, and front-end distance ratio and lateral offset and other interactive characteristics from the map, and splicing the standardized features with the vehicle's basic state to form a comprehensive feature vector.

[0115] A dual-layer attention anomaly recognition module is used to construct a deep learning model. The deep learning model is trained in a supervised manner using historical normal and abnormal driving data. The integrated feature vector and roadside cooperative information are input into the trained deep learning model to obtain an anomaly probability score.

[0116] In practice, this module has the function of capturing temporal anomaly patterns through a spatiotemporal self-attention coding layer, fusing roadside environmental information through a vehicle-road cooperative cross-attention decoding layer, and finally outputting anomaly probability scores.

[0117] The dynamic threshold classification decision module is used to dynamically calculate the judgment threshold based on the anomaly probability score in the sliding time window; and to identify the driving status of the vehicle based on the anomaly probability score and the judgment threshold.

[0118] In practice, this module has the functions of dynamically calculating the judgment threshold based on historical window data, capturing continuous low-intensity anomalies through an abnormal energy accumulation mechanism, and classifying three levels of anomalies according to the magnitude of exceeding the standard and the accumulated value.

[0119] It should be noted that the system also includes:

[0120] The multi-level collaborative execution feedback module is used to execute different response measures according to the vehicle's driving status, including: if the vehicle's driving status is a level one warning, a voice prompt is issued; if the vehicle's driving status is a level one alarm, the vehicle control system is triggered to access; if the vehicle's driving status is a level three danger, a safe path is planned based on roadside collaborative information, triggering the emergency takeover of the autonomous driving system.

[0121] In practice, this module has the functions of executing HUD / voice prompts according to the level of abnormality, slightly intervening in the vehicle control system, taking over emergency autonomous driving, and uploading event data packets to the cloud platform via V2X.

[0122] Preferably, the multi-dimensional feature extraction and fusion module includes:

[0123] The vehicle handling feature extraction unit can quantify the abruptness of steering control by calculating the steering wheel angle entropy value, analyze the interlock relationship between the accelerator and brake pedals to determine power request conflicts, and calculate the deviation between longitudinal acceleration and powertrain output power to identify drive system anomalies.

[0124] The dynamic response feature extraction unit can calculate the theoretical yaw rate and lateral acceleration based on the three-degree-of-freedom vehicle dynamics model, using the actual steering wheel angle and vehicle speed as inputs. It then generates a dynamic mismatch vector composed of yaw rate deviation, lateral acceleration deviation, and estimated center of gravity sideslip angle by comparing the calculated values ​​with the IMU values.

[0125] The vehicle-road cooperative feature extraction unit can use global traffic flow data provided by the roadside to calculate the ratio of the actual headway between the vehicle and the adjacent vehicle to the expected value, monitor the lateral offset of the vehicle's trajectory relative to the lane centerline and its differential rate of change in real time, and calculate the remaining time to safely pass through the intersection (TTC) index based on the traffic light sequence and the current vehicle speed.

[0126] The feature standardization and fusion unit can use the Z-score standardization method to eliminate the differences in the dimensions of each feature vector, and then horizontally concatenate the three sets of feature vectors with the vehicle's basic state information to generate a structured comprehensive feature tensor.

[0127] Compared with existing technologies, the vehicle abnormal driving state identification method and system that integrates vehicle-road cooperative information provided by the present invention has the following beneficial effects:

[0128] 1. By deeply integrating vehicle-road cooperative information and vehicle perception data, and adopting a two-layer attention mechanism network model, the system effectively solves the information silo problem in traditional anomaly recognition. The system can use roadside global environmental information as a key decision-making basis, enabling the model not only to detect abnormal behavior, but also to understand the external environmental motivations that generate anomalies. This significantly distinguishes between reasonable risk avoidance and dangerous driving, greatly reducing the false alarm rate. The introduction of dynamic adaptive thresholds and anomaly energy accumulation mechanisms enables the system to adapt to different driving styles and complex road scenarios, achieving early and accurate warnings for persistent low-intensity abnormal behavior, thus improving the system's practicality and reliability.

[0129] 2. The constructed cloud-based collaborative optimization closed loop uses digital twin technology to conduct in-depth retrospective analysis of abnormal events and adopts OTA (Over-The-Air) updates to achieve incremental model updates, enabling the system to continuously evolve and self-optimize. At the same time, the group security protection mechanism based on V2X communication quickly transforms single-vehicle abnormal events into regional early warning information, realizing a leap from single-vehicle intelligence to group collaborative security. This end-to-cloud collaborative architecture not only improves the driving safety level of individual vehicles, but also builds a regional proactive safety protection network through information sharing, significantly improving the overall road traffic safety coefficient.

[0130] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0131] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0132] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0133] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0134] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0135] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0136] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0137] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0138] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for identifying abnormal vehicle driving states by integrating vehicle-road cooperative information, characterized in that, include: Real-time acquisition of vehicle dynamic parameters and onboard perception information; Receive roadside coordination information via communication protocols; Based on the vehicle dynamic parameters, vehicle perception information and roadside cooperative information, a scene is constructed and a local dynamic map with the vehicle as the origin is generated. Three sets of feature vectors are extracted from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; the three sets of feature vectors are concatenated with the vehicle's basic state information to generate a comprehensive feature vector. The extraction of three sets of feature vectors from the local dynamic map includes: calculating the steering wheel angle entropy, accelerator / brake pedal interlock relationship, and deviation between longitudinal acceleration and powertrain output power based on vehicle dynamic parameters in the local dynamic map, and integrating the above calculation results into a vehicle self-management feature vector; collecting steering wheel angle and vehicle speed, inputting them into a pre-established three-degree-of-freedom vehicle dynamics model to obtain theoretical yaw rate and lateral acceleration as theoretical values; using the actual measured yaw rate and lateral acceleration in the local dynamic map as actual values, calculating the yaw rate deviation, lateral acceleration deviation, and estimated center of gravity sideslip angle based on the actual values ​​and theoretical values, and integrating the above calculation results into a vehicle dynamic response feature vector; and calculating the ratio of the actual headway between the vehicle and surrounding vehicles to their expected headway, the lateral offset of the vehicle's trajectory from the lane centerline and its rate of change, and the TTC index of whether the vehicle can legally pass the stop line within the current traffic light cycle based on the local dynamic map, and integrating the above calculation results into a vehicle-road cooperative interaction feature vector. A deep learning model is constructed, and supervised training is performed on the deep learning model using historical normal and abnormal driving data. The integrated feature vector and roadside cooperative information are input into the trained deep learning model to obtain the abnormality probability score. The judgment threshold is dynamically calculated based on the anomaly probability score in the sliding time window; the driving status of the vehicle is identified based on the anomaly probability score and the judgment threshold. The step of dynamically calculating the judgment threshold based on the anomaly probability score in the sliding time window includes: The sliding time window stores the latest time series of the vehicle's anomaly probability score; Calculate the mean μ and standard deviation σ of the anomaly probability scores in the time series, and then calculate the judgment threshold θ for the current time step using the following formula: θ = μ + k × σ Where k is the adjustment coefficient; The step of identifying the vehicle's driving status based on the anomaly probability score and the determination threshold further includes: Set the initial anomaly integral; Calculate the difference between the instantaneous anomaly probability score and the dynamic threshold. If the anomaly probability score is greater than the dynamic threshold, add the difference to the anomaly integral. If the anomaly probability score is less than or equal to the dynamic threshold, reduce the value of the anomaly integral according to a preset attenuation factor. The vehicle's driving status at the current moment is determined by the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the value of the anomaly integral at that moment.

2. The method according to claim 1, characterized in that, Real-time acquisition of vehicle dynamic parameters, including: real-time acquisition of vehicle dynamic parameters of the vehicle through the on-board sensing unit, the vehicle dynamic parameters including: vehicle steering wheel angle, throttle opening, brake pedal travel, yaw rate, longitudinal acceleration and lateral acceleration; Receiving roadside cooperative information via a communication protocol includes: receiving roadside cooperative information via an onboard communication unit; the roadside cooperative information is generated by roadside intelligent infrastructure deployed at key road nodes; the roadside intelligent infrastructure senses structured environmental data within a preset radius centered on the vehicle and generates roadside cooperative information.

3. The method according to claim 2, characterized in that, Based on the vehicle dynamic parameters and the roadside cooperative information, a scene is constructed and a local dynamic map with the vehicle as the origin is generated, including: The vehicle dynamic parameters, on-board perception information and roadside coordination information are spatiotemporally synchronized and stored in a cache queue; Based on the data in the cache queue, the short-term state of the vehicle and traffic participants is predicted using extended Kalman filtering, and the roadside sensing targets and vehicle-mounted sensing targets are matched and associated using the nearest neighbor data association algorithm. The data in the roadside coordinate system is transformed to the vehicle coordinate system with the vehicle's center of gravity as the origin using a coordinate transformation matrix. Based on all data in the vehicle coordinate system with the vehicle's center of gravity as the origin, vectorized information of all traffic participants is obtained; By fusing vehicle dynamic parameters with vectorized information from all traffic participants, a local dynamic map is generated.

4. The method according to claim 1, characterized in that, Building deep learning models includes: A two-layer attention mechanism deep learning model is constructed, with the first layer being a spatiotemporal self-attention encoding layer and the second layer being a vehicle-road cooperative cross-attention decoding layer; The first layer is used to receive the comprehensive feature vector arranged in time series, calculate the dependency between features at different time steps within the sequence through a self-attention mechanism, capture transient and short-term anomaly patterns, and output the vehicle state feature sequence. The second layer fuses the vehicle's state feature sequence with roadside cooperative information to obtain an interpretive feature representation.

5. The method according to claim 1, characterized in that, Also includes: The vehicle's driving status at the current moment is classified based on the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the anomaly integral value at that moment: normal driving, Level 1 warning, Level 2 alarm, and Level 3 danger.

6. A vehicle abnormal driving state recognition system integrating vehicle-road cooperative information, characterized in that, include: The vehicle-mounted multi-source data acquisition module is used to collect the vehicle's dynamic parameters and vehicle-mounted perception information in real time. Receive roadside coordination information via communication protocols; The spatiotemporal synchronization and map building module is used to build a scene based on the vehicle dynamic parameters, vehicle perception information and roadside cooperative information, and to fuse and generate a local dynamic map with the vehicle as the origin. The multi-dimensional feature extraction and fusion module is used to extract three sets of feature vectors from the local dynamic map: vehicle self-management feature vector, vehicle dynamic response feature vector, and vehicle-road cooperative interaction feature vector; and to concatenate the three sets of feature vectors with the vehicle's basic state information to generate a comprehensive feature vector. The extraction of three sets of feature vectors from the local dynamic map includes: calculating the steering wheel angle entropy, accelerator / brake pedal interlock relationship, and deviation between longitudinal acceleration and powertrain output power based on vehicle dynamic parameters in the local dynamic map, and integrating the above calculation results into a vehicle self-management feature vector; collecting steering wheel angle and vehicle speed, inputting them into a pre-established three-degree-of-freedom vehicle dynamics model to obtain theoretical yaw rate and lateral acceleration as theoretical values; using the actual measured yaw rate and lateral acceleration in the local dynamic map as actual values, calculating the yaw rate deviation, lateral acceleration deviation, and estimated center of gravity sideslip angle based on the actual values ​​and theoretical values, and integrating the above calculation results into a vehicle dynamic response feature vector; and calculating the ratio of the actual headway between the vehicle and surrounding vehicles to its expected headway, the lateral offset of the vehicle's trajectory from the lane centerline and its rate of change, and the TTC index of whether the vehicle can legally pass the stop line within the current traffic light cycle based on the local dynamic map, and integrating the above calculation results into a vehicle-road cooperative interaction feature vector. A dual-layer attention anomaly detection module is used to construct a deep learning model and conduct supervised training on the deep learning model using historical normal and abnormal driving data; the integrated feature vector and roadside cooperative information are input into the trained deep learning model to obtain an anomaly probability score; The dynamic threshold hierarchical decision module is used to dynamically calculate the judgment threshold based on the anomaly probability score in the sliding time window; and to identify the driving status of the vehicle based on the anomaly probability score and the judgment threshold. The step of dynamically calculating the judgment threshold based on the anomaly probability score in the sliding time window includes: The sliding time window stores the latest time series of the vehicle's anomaly probability score; Calculate the mean μ and standard deviation σ of the anomaly probability scores in the time series, and then calculate the judgment threshold θ for the current time step using the following formula: θ = μ + k × σ Where k is the adjustment coefficient; The step of identifying the vehicle's driving status based on the anomaly probability score and the determination threshold further includes: Set the initial anomaly integral; Calculate the difference between the instantaneous anomaly probability score and the dynamic threshold. If the anomaly probability score is greater than the dynamic threshold, add the difference to the anomaly integral. If the anomaly probability score is less than or equal to the dynamic threshold, reduce the value of the anomaly integral according to a preset attenuation factor. The vehicle's driving status at the current moment is determined by the difference between the instantaneous anomaly probability score and the dynamic threshold, as well as the value of the anomaly integral at that moment.

7. The system according to claim 6, characterized in that, Also includes: The multi-level collaborative execution feedback module is used to execute different response measures according to the vehicle's driving status, including: if the vehicle's driving status is a level one warning, a voice prompt is issued; if the vehicle's driving status is a level one alarm, the vehicle control system is triggered to access; if the vehicle's driving status is a level three danger, a safe path is planned based on roadside collaborative information, triggering the emergency takeover of the autonomous driving system.

Images