Abnormal driving behavior recognition method and system based on multi-modal data fusion

Abstract

The application discloses an abnormal driving behavior recognition method and system based on multi-modal data fusion, and particularly relates to the technical field of multi-vehicle cooperative driving safety control, and is used for solving the problem that the existing cooperative driving system cannot convert the abnormal behavior risk of a driver into a cooperative control instruction at the vehicle fleet level; the abnormal state of the driver is recognized by collecting and fusing multi-modal sensor data of the vehicle; when the vehicle is a lead vehicle of a vehicle fleet, the comprehensive risk of the abnormal state to the safety of the vehicle fleet is evaluated based on the abnormal state through probabilistic evolution simulation; the expected effect of different vehicle fleet control strategies is simulated according to the risk evaluation result, and the best strategy is selected; vehicle fleet risk information containing the strategy is generated and sent to all following vehicles through vehicle-to-vehicle communication; the whole process from single-vehicle driver state monitoring to vehicle fleet level cooperative risk prevention and control is realized, and the overall safety of the vehicle fleet system is improved.

Description

Technical Field

[0001] This invention relates to the field of multi-vehicle cooperative driving safety control technology, and more specifically, to a method and system for identifying abnormal driving behavior based on multimodal data fusion. Background Technology

[0002] In the field of multi-vehicle cooperative driving, especially in commercial vehicle platooning, achieving coordinated control of speed and spacing through inter-vehicle wireless communication is a crucial technological direction for improving road throughput and fuel economy. Existing cooperative driving systems primarily rely on the interaction and processing of vehicle dynamics information. For example, they acquire acceleration and speed information of the vehicle ahead through vehicle-to-vehicle communication and generate control commands for the vehicle itself based on a predetermined car-following model to maintain stable platooning. Simultaneously, to ensure safety, individual vehicles are generally equipped with driver status monitoring functions based on in-vehicle sensor data to identify and warn of abnormal driving behaviors such as fatigue and distraction at the vehicle level.

[0003] However, current cooperative driving control logic only considers the matching and optimization of vehicle motion states, failing to incorporate the key risk factor of driver behavior into the fleet-level cooperative safety assessment and decision-making system. When the driver of the lead vehicle in the convoy exhibits abnormal behavior, its single-vehicle monitoring system can only issue a localized warning. However, this warning information and the risk level it represents cannot be effectively transformed into a cooperative signal that the fleet control layer can understand and respond to. As a result, following vehicles cannot proactively perceive the non-dynamic risks originating from the lead vehicle driver and still passively follow and control based on normal vehicle motion data. Once the lead vehicle exhibits unexpected operations due to driver abnormalities, the entire convoy will face a chain reaction safety threat caused by the break in the risk information transmission chain. That is, the driver risk of a single vehicle may quickly evolve into a systemic safety crisis for the entire fleet. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides an abnormal driving behavior recognition method and system based on multimodal data fusion to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Abnormal driving behavior identification methods based on multimodal data fusion include:

[0007] S1. Collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data;

[0008] S2. Determine whether this vehicle is the lead vehicle in the cooperative driving fleet;

[0009] S3. When the vehicle is the lead vehicle and an abnormal state is detected, based on the pattern and intensity of the abnormal state, the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state is simulated through probability evolution. Based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet is obtained.

[0010] S4. Based on the comprehensive risk assessment results, simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet.

[0011] S5. From the simulated preset fleet control strategies, select the preset fleet control strategy with the best expected improvement effect as the recommended control strategy, and generate fleet risk information containing the recommended control strategy.

[0012] S6. Send fleet risk information to following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

[0013] Furthermore, S1 includes:

[0014] Collect data from the vehicle's in-vehicle camera and vehicle control data;

[0015] The driver's facial and posture features are extracted from the in-vehicle camera data, and the vehicle control feature vector representing the driver's control behavior is extracted from the vehicle control data.

[0016] The driver's facial and posture features are fused with the vehicle control feature vector to generate a fused feature vector;

[0017] Abnormal states of the driver in this vehicle are identified based on fused feature vectors.

[0018] Furthermore, S2 includes:

[0019] Obtain fleet formation information describing the vehicle formation order and roles in the cooperative driving fleet;

[0020] The vehicle's assigned position in the formation sequence is determined from the vehicle formation information.

[0021] Based on the rule that the lead vehicle in a cooperative driving fleet is always at the head of the formation sequence, determine whether the vehicle's position in the formation sequence meets the conditions for a lead vehicle.

[0022] Furthermore, S3 includes:

[0023] Based on the pattern and intensity of the abnormal state, determine its expected impact range and probability distribution on vehicle control commands.

[0024] Based on the expected scope of impact and its probability distribution, combined with the current vehicle motion state, multiple random sampling simulations are used to generate multiple possible motion trajectories of the vehicle in the future within a preset time period, forming the probability distribution of the future motion trajectory.

[0025] From the probability distribution of future motion trajectories, we statistically identify the motion trajectories that lead to violations of the kinematic constraints of safe following of the vehicle fleet, and calculate their proportion to characterize the degree of conflict.

[0026] Based on the degree of conflict, a comprehensive risk assessment result for the cooperative driving fleet under abnormal conditions is obtained by quantification.

[0027] Furthermore, based on the pattern and intensity of the abnormal state, the expected impact range and probability distribution of its impact on vehicle control commands are determined. This is achieved by: based on the pre-established correspondence between abnormal states and vehicle control parameter disturbances, querying the expected impact range and probability distribution of the longitudinal and lateral control commands of the vehicle that match the pattern and intensity of the current abnormal state.

[0028] Furthermore, S4 includes:

[0029] Based on the comprehensive risk assessment results, select at least two preset fleet control strategies from the pre-set strategy set;

[0030] For each preset fleet control strategy, simulate the vehicle state change process after the vehicle executes the preset fleet control strategy;

[0031] Analyze the consistency between the simulated vehicle state change process and the kinematic constraints for safe following of the fleet;

[0032] Based on the applicable conditions, evaluate the expected improvement effect of each preset fleet control strategy on meeting the kinematic constraints of safe following of the fleet.

[0033] Furthermore, the simulation results of vehicle state changes are analyzed to determine their compliance with the kinematic constraints of safe following of the convoy. This is achieved by comparing the time-series state data of the simulated vehicle state changes with the threshold conditions constituting the kinematic constraints of safe following of the convoy point by point, and statistically quantifying the degree of compliance with each threshold condition.

[0034] Furthermore, S5 includes:

[0035] Compare the expected improvement effects evaluated for each preset fleet control strategy;

[0036] From the simulated preset fleet control strategies, the preset fleet control strategy with the best expected improvement effect is selected as the recommended control strategy;

[0037] Generate fleet risk information that includes recommended control strategies and their corresponding expected improvement effects.

[0038] Furthermore, S6 includes:

[0039] Based on the platooning structure of the cooperative driving fleet, the following vehicles as receivers are determined;

[0040] The fleet risk information, which includes the recommended control strategy and its corresponding expected improvement effect, is encapsulated into a data frame that conforms to the vehicle-to-vehicle communication protocol format.

[0041] Encapsulated data frames are sent to the designated following vehicles via vehicle-to-vehicle communication links to complete the transmission of fleet risk information.

[0042] On the other hand, the present invention provides an abnormal driving behavior recognition system based on multimodal data fusion, comprising:

[0043] The anomaly detection module is used to collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data.

[0044] The role determination module is used to determine whether the vehicle is the lead vehicle in the collaborative driving fleet;

[0045] The risk assessment module is used to simulate the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state based on the pattern and intensity of the abnormal state, and based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, to obtain the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet.

[0046] The strategy simulation module is used to simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet, based on the comprehensive risk assessment results.

[0047] The information generation module is used to select the preset fleet control strategy with the best expected improvement effect from the simulated preset fleet control strategies as the recommended control strategy, and generate fleet risk information containing the recommended control strategy.

[0048] The information sending module is used to send fleet risk information to the following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. By deeply integrating driver status monitoring with the fleet collaborative control system, the problem of information gaps in driver abnormality risks within the fleet is effectively solved. When the lead vehicle identifies a driver abnormality, the method can quantify and assess the systemic safety threat posed to the entire fleet based on the deduction of the abnormality's development trend. This assessment result is then transformed into specific, clear, and immediately executable fleet collaborative control instructions. This transforms risk warnings that were originally isolated within a single vehicle into collaborative action signals that can be understood and responded to by the entire fleet, achieving full-chain connectivity of risk information from perception to decision-making and execution.

[0051] 2. Because the following vehicle can obtain instruction information containing specific control strategies in advance, the fleet system has the ability to proactively respond to non-dynamic risks. The entire fleet can shift from a reactive mode of passively receiving changes in the motion state of the vehicle in front to a proactive collaborative prevention and control mode based on risk prediction. This not only significantly improves the collective safety of the fleet when the lead vehicle driver experiences abnormalities and effectively curbs the spread and amplification of a single risk in a tightly coupled fleet, but also enhances the robustness and reliability of the collaborative driving system as a whole, providing key technical support for achieving higher-level and safer automated fleet collaborative operations. Attached Figure Description

[0052] Figure 1 This is a flowchart of the abnormal driving behavior recognition method based on multimodal data fusion according to the present invention;

[0053] Figure 2 This is a schematic diagram of the abnormal driving behavior recognition system based on multimodal data fusion of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Example 1: Figure 1 The present invention provides an abnormal driving behavior recognition method based on multimodal data fusion, comprising:

[0056] S1. Collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data;

[0057] S2. Determine whether this vehicle is the lead vehicle in the cooperative driving fleet;

[0058] S3. When the vehicle is the lead vehicle and an abnormal state is detected, based on the pattern and intensity of the abnormal state, the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state is simulated through probability evolution. Based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet is obtained.

[0059] S4. Based on the comprehensive risk assessment results, simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet.

[0060] S5. From the simulated preset fleet control strategies, select the preset fleet control strategy with the best expected improvement effect as the recommended control strategy, and generate fleet risk information containing the recommended control strategy.

[0061] S6. Send fleet risk information to following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

[0062] According to an embodiment of the present invention, the specific implementation of acquiring in-vehicle camera data and vehicle control data in step S1 is as follows: In-vehicle camera data is acquired through a camera installed in the vehicle's driver's cab and positioned directly in front of the driver. The camera acquires video image sequences containing the driver's face and upper body at a frequency of, for example, 30 frames per second. Vehicle control data is acquired through the vehicle's controller area network (CLAN) bus. Specifically, this involves real-time reading from the CLAN bus of data signals directly related to the driver's operating behavior, measured and broadcast by vehicle sensors. These data signals include steering wheel angle signals, accelerator pedal opening signals, brake pedal opening signals, and turn signal switch status signals.

[0063] The specific implementation method for extracting the driver's facial and posture features from the in-vehicle camera data in step S1 is as follows: For each frame of video image, a pre-trained face detection model is used to determine the location of the driver's facial region. Multiple key feature points are located within the facial region. These key feature points include the outer corner of the left eye, the inner corner of the left eye, the inner corner of the right eye, the outer corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth. A set of visual features is calculated based on the pixel coordinates of the key feature points. These visual features include eye aspect ratio features, mouth aspect ratio features, and head posture angle features. The eye aspect ratio feature is calculated as follows: first, the ratio of the height to the width of the left eye contour is calculated based on the coordinates of the outer and inner corners of the left eye to obtain the left eye aspect ratio; then, the ratio of the height to the width of the right eye contour is calculated based on the coordinates of the outer and inner corners of the right eye to obtain the right eye aspect ratio; finally, the left and right eye aspect ratios are added together and divided by 2 to obtain the eye aspect ratio feature value. The aspect ratio of the mouth is calculated by determining the ratio of the mouth's height to its width based on the coordinates of the left and right corner points. The head pose angle is calculated using a pose estimation algorithm based on feature points. The 2D image coordinates of detected key facial feature points are aligned with the 3D coordinates of corresponding feature points in a standard 3D face model to determine the rotation angle of the driver's head relative to the camera coordinate system. The rotation angle includes head pitch, head yaw, and head roll.

[0064] The specific implementation of extracting the vehicle control feature vector representing the driver's operating behavior from the vehicle control data in step S1 is as follows: The vehicle control feature vector is constructed based on vehicle control data within a sliding time window of a preset length. For example, the length of the sliding time window is 5 seconds. For the steering wheel angle signal sequence within the sliding time window, the standard deviation of the sequence is calculated as the steering wheel operation fluctuation feature. For the accelerator pedal opening signal sequence within the sliding time window, the average value of the sequence is calculated as the accelerator pedal operation average feature. For the brake pedal opening signal sequence within the sliding time window, the maximum value of the sequence is calculated as the brake operation intensity feature. The number of times the turn signal switch status signal changes from off to on within the sliding time window is counted as the steering intention frequency feature. The steering wheel operation fluctuation feature, accelerator pedal operation average feature, brake operation intensity feature, and steering intention frequency feature are sequentially combined into a four-dimensional vehicle control feature vector.

[0065] The specific implementation of fusing the driver's facial and posture features with the vehicle control feature vector in step S1 to generate the fused feature vector is as follows: The eye aspect ratio feature values, mouth aspect ratio feature values, head pitch angle feature values, head yaw angle feature values, and head roll angle feature values ​​contained in the driver's facial and posture features are arranged in order to form a five-dimensional visual feature vector. The four-dimensional vehicle control feature vector is concatenated with the five-dimensional visual feature vector, i.e., the nine feature values ​​are connected into a nine-dimensional fused feature vector in the order of visual feature vector first, followed by vehicle control feature vector. Before the concatenation operation, each feature value in both the visual feature vector and the vehicle control feature vector is standardized. The standardization process uses the Z-score normalization method. For any feature dimension in the visual feature vector, the mean and standard deviation of all sample values ​​in that dimension are pre-calculated from the historical normal driving dataset. The currently acquired feature value of that dimension is subtracted from the corresponding mean and then divided by the corresponding standard deviation to obtain the standardized value of that visual feature dimension. For any feature dimension in the vehicle handling feature vector, the same Z-score normalization method is used for processing, and its mean and standard deviation are also calculated based on the corresponding dimension of the historical normal driving dataset. The standardized visual feature vector and the vehicle handling feature vector are then concatenated to generate the final nine-dimensional fused feature vector.

[0066] The specific implementation of step S1, which identifies the abnormal state of the driver based on the fused feature vector, is as follows: The identification process is completed by a pre-trained support vector machine (SVM) classifier. The input to the SVM classifier is a nine-dimensional fused feature vector, and the output is the driver's state belonging to one of the predefined categories. The predefined categories include normal driving state, fatigued driving state, and distracted driving state. The SVM classifier is trained by collecting historical driving datasets containing samples of normal driving state, fatigued driving state, and distracted driving state. Each sample contains its corresponding nine-dimensional fused feature vector and manually labeled true state. Using all samples in the historical driving dataset, a quadratic programming problem is solved to find the hyperplane that optimally separates samples of different state categories, thereby determining the model parameters of the SVM classifier. During real-time identification, the currently generated fused feature vector is input into the pre-trained SVM classifier, which calculates the state category to which the fused feature vector belongs based on its decision function. For fused feature vectors classified as fatigued driving state or distracted driving state, the geometric distance from its decision function value to the classification hyperplane is further calculated. The geometric distance is converted into a value between 0 and 100 using a linear mapping function, which serves as the anomaly intensity score. The linear mapping function is determined as follows: during model training, the distance from all correctly classified anomaly samples in the training set to the hyperplane is calculated, with the minimum distance mapped to a score of 0, the maximum distance mapped to a score of 100, and intermediate distance values ​​mapped linearly. The anomaly category obtained in real-time identification, together with the anomaly intensity score, constitutes the anomaly information of the driver identified in step S1.

[0067] According to an embodiment of the present invention, the specific implementation of obtaining fleet formation information describing the platooning order and roles of vehicles in the cooperative driving fleet in step S2 is as follows: Fleet formation information is periodically received from a logical entity responsible for fleet coordination management via a vehicle-to-vehicle communication link. This logical entity may be the lead vehicle in the cooperative driving fleet. The fleet formation information is broadcast or multicast in the form of a structured data message. This data message contains an ordered list of vehicle identifiers. Each vehicle identifier is a unique code that identifies each vehicle in the cooperative driving fleet. For example, a vehicle identifier is a short identifier obtained by hashing the vehicle identification number extracted from the vehicle controller area network bus. In the vehicle identifier list, the order of each vehicle identifier directly corresponds to the vehicle's position in the platooning sequence within the cooperative driving fleet. The vehicle corresponding to the first vehicle identifier in the list is designated as the lead vehicle. The vehicles corresponding to subsequent vehicle identifiers in the list are designated as follower vehicles. The data message also contains a sequence number field. The sequence number field is used to identify the version of the fleet formation information. When fleet members or the order changes, the value of the sequence number field increments to facilitate the vehicle's recognition of the updated formation information.

[0068] The specific implementation of parsing the assigned formation position of the vehicle from the fleet formation information in step S2 is as follows: When initializing or joining a cooperative driving fleet, the vehicle has already obtained its own vehicle identifier through configuration or registration. Upon receiving the fleet formation information data message, the vehicle first verifies the integrity and validity of the data message. Integrity verification includes checking the message's cyclic redundancy check (CRC) code. After successful verification, the vehicle parses the data message and reads the list of vehicle identifiers. The vehicle compares its own vehicle identifier with each identifier in the list sequentially. The comparison operation uses a byte-level exact match method. When an entry that completely matches its own vehicle identifier is found in the list, its index number in the list is recorded. The starting value for the index number is set to 1, meaning the index number of the first position in the list is 1, and the index number of the second position is 2. The recorded index number is then parsed as the assigned formation position of the vehicle. If its own vehicle identifier is not found in the list, it is determined that the vehicle does not currently belong to the cooperative driving fleet, and an invalid formation position value is output. An invalid formation position value is, for example, 0.

[0069] In step S2, based on the rule that the lead vehicle in a cooperative driving fleet is always at the head of the platoon, the specific implementation of determining whether the vehicle's platoon position meets the lead vehicle condition is as follows: The lead vehicle condition is a predefined logical judgment criterion. The lead vehicle condition is: if a vehicle's platoon position is equal to 1, then the vehicle meets the lead vehicle condition. After obtaining the vehicle's platoon position, an equivalence comparison operation is performed. The vehicle's platoon position is compared with the number 1. The comparison operation is implemented using the equality operator in a programming language. If the comparison result is true, that is, the vehicle's platoon position is equal to 1, then the judgment conclusion is generated that the vehicle meets the lead vehicle condition. If the comparison result is false, that is, the vehicle's platoon position is not equal to 1, then the judgment conclusion is generated that the vehicle does not meet the lead vehicle condition. This judgment conclusion will be stored and output as a logical flag. The logical flag is used to control the logical branches of subsequent steps S3 to S6.

[0070] According to an embodiment of the present invention, in step S3, determining the expected range of influence and its probability distribution on vehicle control commands based on the pattern and intensity of the abnormal state is achieved in the following way: A correspondence table between abnormal states and vehicle control parameter disturbances is pre-established. The correspondence table defines the influence parameters of different abnormal state patterns and intensities on the longitudinal and lateral control commands of the vehicle. Abnormal state patterns include fatigue driving mode and distracted driving mode. The intensity of the abnormal state is represented by the abnormal state intensity score calculated in step S1. The establishment of the correspondence table is based on statistical analysis of historical driving data. A large amount of data containing different abnormal state patterns, different abnormal state intensity scores, and changes in control commands generated by the driver's actual operation of the vehicle during the corresponding time period is collected. For each abnormal state pattern, the abnormal state intensity score is divided into multiple intervals. For example, the abnormal state intensity score is divided into the 0-20 point interval, the 21-40 point interval, the 41-60 point interval, the 61-80 point interval, and the 81-100 point interval. For all data samples falling into each intensity score interval, the adjustment range of the driver's longitudinal acceleration command to the vehicle within that interval is statistically analyzed. The adjustment range of longitudinal acceleration commands is represented by minimum and maximum values. For example, in the abnormal state mode of fatigue driving and with an abnormal state intensity score between 61 and 80, the adjustment range of longitudinal acceleration commands might be -1.5 m / s² to +0.5 m / s². Negative values ​​represent deceleration commands. For all data samples falling into each intensity score interval, the adjustment range of the driver's steering wheel angle commands within that interval is statistically analyzed. The adjustment range of steering wheel angle commands is represented by the maximum left turn value and the maximum right turn value. Simultaneously, for all data samples falling into each intensity score interval, the probability distribution of the specific values ​​of longitudinal acceleration commands within that interval is statistically analyzed. The probability distribution is obtained through fitting. For example, the distribution of longitudinal acceleration commands might conform to a normal distribution. A normal distribution is described by mean and standard deviation parameters. Based on this statistical analysis results, a correspondence table is generated. Each record in the corresponding relationship table includes the following fields: Abnormal State Mode, Abnormal State Intensity Score Lower Limit, Abnormal State Intensity Score Upper Limit, Minimum Value of Longitudinal Acceleration Command, Maximum Value of Longitudinal Acceleration Command, Probability Distribution Type of Longitudinal Acceleration Command, Distribution Parameter of Longitudinal Acceleration Command, Maximum Value of Left Turn of Steering Wheel Angle Command, Maximum Value of Right Turn of Steering Wheel Angle Command, Probability Distribution Type of Steering Wheel Angle Command, and Distribution Parameter of Steering Wheel Angle Command.In real-time applications, based on the currently identified abnormal state pattern and abnormal state intensity score, the corresponding relationship table is queried to find records that match the current abnormal state pattern and whose current abnormal state intensity score is between its lower and upper limits. The minimum and maximum values ​​of the longitudinal acceleration command defined in the record are taken as the expected influence range of the longitudinal control command, and the probability distribution type and distribution parameters of the longitudinal acceleration command defined in the record are taken as the possible distribution of the longitudinal control command. The maximum left and right values ​​of the steering wheel angle command defined in the record are taken as the expected influence range of the lateral control command, and the probability distribution type and distribution parameters of the steering wheel angle command defined in the record are taken as the possible distribution of the lateral control command.

[0071] In step S3, based on the expected impact range and its probability distribution, and combined with the current vehicle motion state, multiple possible motion trajectories of the vehicle within a preset future time period are generated through multiple random sampling simulations, forming a probability distribution of the future motion trajectory. This is achieved through the following method: The current vehicle motion state includes the vehicle's current speed, current longitudinal acceleration, current yaw rate, and lateral position within the current lane. The simulation of the vehicle's motion trajectory within the preset future time period is performed using a discrete-time recursive approach. The preset future time period is, for example, 5 seconds. The discrete time step is, for example, 0.1 seconds. The recursive calculation starts from the current moment and calculates the vehicle motion state after 50 consecutive time steps. Each complete recursive calculation generates one possible motion trajectory. To form the probability distribution, multiple random sampling simulations are performed, for example, 500 independent recursive calculations are performed, generating 500 possible motion trajectories. In the initialization phase of each recursive calculation, the initial state is set to the vehicle's current motion state. In the calculation of each time step, one random sampling is performed. The object of random sampling is the vehicle control command. For longitudinal control commands, sampling is performed based on the expected impact range and probability distribution of the retrieved longitudinal control commands. If the probability distribution is normal, the mean and standard deviation of this normal distribution are used as parameters to generate a random number that satisfies this normal distribution, which serves as the longitudinal acceleration command increment for that time step. The longitudinal acceleration command increment needs to be constrained within the expected impact range formed by the minimum and maximum values ​​of the longitudinal acceleration command. For lateral control commands, random sampling is performed in the same way to obtain a steering wheel angle command increment. Then, based on the vehicle dynamics model, the vehicle motion state of the previous time step and the longitudinal acceleration command increment and steering wheel angle command increment sampled in the current time step are used as inputs to calculate the vehicle motion state at the end of the current time step. A simplified vehicle dynamics model is used. For longitudinal motion, the uniform acceleration motion formula is used. The velocity at the end of the current time step is equal to the velocity at the end of the previous time step plus the longitudinal acceleration command increment multiplied by the time step. The longitudinal position at the end of the current time step is equal to the longitudinal position at the end of the previous time step plus the average velocity multiplied by the time step. The average speed is the sum of the speed at the end of the previous time step and the speed at the end of the current time step, divided by 2. For lateral movement, the yaw rate is calculated based on parameters such as the steering wheel angle command increment and vehicle wheelbase, and then the vehicle's lateral position is updated based on the yaw rate and vehicle speed. This process is repeated for 50 time steps, recording the vehicle's longitudinal and lateral positions at the end of each time step. The sequence of these positions constitutes a possible trajectory. This process is repeated independently 500 times, resulting in a set of 500 possible trajectories. This set represents the probability distribution of the vehicle's future trajectory.

[0072] In step S3, the probability distribution of future trajectories is used to statistically identify the trajectories that violate the kinematic constraints of the convoy safety following, and their proportion is calculated to characterize the degree of conflict. This is achieved through the following method: The kinematic constraints of the convoy safety following are defined by a set of inequalities. These conditions include a minimum safe distance condition, a lane-keeping condition, and a maximum deceleration condition. The minimum safe distance condition requires that the distance between the vehicle and a preset forward virtual reference point must always be greater than a minimum safe distance threshold. The position of the forward virtual reference point is calculated based on the desired cruise speed of the cooperative driving convoy and a preset safe headway. The safe headway is, for example, 2 seconds. The desired cruise speed is obtained from the convoy cooperative control command. The forward virtual reference point is located in front of the vehicle, at a distance equal to the desired cruise speed multiplied by the safe headway. The lane-keeping condition requires that the absolute value of the lateral offset of the vehicle's lateral position from the lane centerline must always be less than a lane departure threshold. The maximum deceleration condition requires that the longitudinal acceleration of the vehicle must always be greater than a maximum deceleration threshold. For each possible trajectory obtained through multiple random sampling simulations, the vehicle state at each time step on the trajectory is checked to ensure that all inequality conditions are met simultaneously. The checking process is as follows: for a possible trajectory, starting from the first time step and ending at the last, the vehicle state at each point is checked sequentially. If, at a certain time point, the vehicle state is found to violate any of the minimum safe distance condition, lane keeping condition, or maximum deceleration condition, the check on that trajectory is immediately terminated, and this possible trajectory is marked as a trajectory that violates the kinematic constraints of safe platoon following. If the vehicle state of a possible trajectory satisfies all inequality conditions from the first time point to the last time point, this possible trajectory is marked as a trajectory that does not violate the constraints. After checking all possible trajectories, the number of trajectories marked as violating the kinematic constraints of safe platoon following is counted. The degree of conflict is characterized by calculating the proportion of this number to the total number of simulated trajectories. For example, if 150 out of 500 trajectories are marked as violating constraints, the degree of conflict is 150 / 500 = 0.3.

[0073] In step S3, the comprehensive risk assessment result of the abnormal state on the collaborative driving fleet is quantified based on the degree of conflict, achieved as follows: The comprehensive risk assessment result is represented by a predefined risk level. Risk levels include low risk, medium risk, and high risk. Two risk level thresholds are defined: a low risk threshold and a high risk threshold. The risk level classification rules are as follows: If the conflict degree calculated in step S3 is less than or equal to the low risk threshold, the comprehensive risk assessment result is low risk; if the conflict degree is greater than the low risk threshold but less than or equal to the high risk threshold, the comprehensive risk assessment result is medium risk; if the conflict degree is greater than the high risk threshold, the comprehensive risk assessment result is high risk. By querying the interval where the conflict degree falls and mapping it to the corresponding risk level, the comprehensive risk assessment result of the abnormal state on the collaborative driving fleet is quantified.

[0074] The minimum safe distance threshold is a critical value used to ensure a safe buffer distance between the vehicle and a virtual reference point ahead. This threshold is calculated by combining the expected cruising speed of the cooperative driving convoy with a preset safe headway. For example, if the safe headway is set to 2 seconds and the expected cruising speed of the convoy is 20 meters per second, then the minimum safe distance threshold is calculated as 20 meters / second × 2 seconds = 40 meters. This calculation method is based on the dynamic relationship between vehicle braking distance and reaction time, ensuring the rationality of the distance setting.

[0075] The lane departure threshold is a critical lateral position value used to determine whether a vehicle has exceeded the safe driving range of its lane. This threshold is obtained by multiplying the standard lane width by half the vehicle width, by a safety factor. For example, if the standard lane width is 3.75 meters and the vehicle width is 2 meters, then the maximum permissible deviation of the vehicle's centerline is (3.75 meters - 2 meters) / 2 = 0.875 meters. To allow for a safety margin, a safety factor of 0.8 is used, resulting in a final lane departure threshold of 0.875 meters × 0.8 ≈ 0.7 meters. This setting is based on road design and vehicle geometry.

[0076] The maximum deceleration threshold is a critical value of longitudinal acceleration used to limit the braking intensity of the vehicle to prevent loss of control. This threshold is obtained by referencing the product of the typical coefficient of friction between the vehicle's tires and the road surface and gravitational acceleration, while taking into account a safety margin. For example, the typical coefficient of friction for a dry asphalt road surface is 0.8, and the gravitational acceleration is 9.8 m / s², with a theoretical maximum deceleration absolute value of approximately 7.84 m / s². To retain a safety margin, the maximum deceleration threshold is set to approximately 0.75 times the theoretical value, or approximately -6 m / s². This setting is based on vehicle dynamics principles.

[0077] The low-risk and high-risk thresholds are critical values ​​used to map conflict intensity values ​​to different risk levels. These two thresholds are obtained through statistical analysis of a large amount of historical convoy driving safety data. Specifically, conflict intensity data samples generated from multiple simulations of normal convoy driving are collected, and the conflict intensity values ​​are sorted in ascending order. The value corresponding to the 90th percentile is taken as the low-risk threshold, for example, 0.1. Similarly, conflict intensity data samples corresponding to historical accidents or dangerous scenarios are collected, and the value corresponding to the 10th percentile is taken as the high-risk threshold, for example, 0.5. This setting is based on the probability distribution characteristics of historical data.

[0078] According to an embodiment of the present invention, in step S4, at least two preset fleet control strategies are selected from a preset strategy set based on the comprehensive risk assessment results, which is achieved in the following manner. The preset strategy set is a data structure stored in the vehicle control system. The preset strategy set contains a variety of selectable preset fleet control strategies. The preset fleet control strategies include at least a lead vehicle deceleration strategy, a follow distance increase strategy, and a fleet deceleration strategy. Each preset fleet control strategy is associated with one or more risk levels. The association is established based on the matching principle between the intervention intensity of the strategy and the risk level. For example, the preset fleet control strategy associated with a low risk level may be a follow distance increase strategy. The preset fleet control strategy associated with a medium risk level may be a lead vehicle deceleration strategy. The preset fleet control strategy associated with a high risk level may be a fleet deceleration strategy. The selection process is as follows: read the risk level corresponding to the comprehensive risk assessment results obtained in step S3. Based on the association relationship, select all preset fleet control strategies associated with the current risk level from the preset strategy set. If there are fewer than two associated preset fleet control strategies, another preset fleet control strategy with the closest intervention intensity is selected from the preset strategy set to ensure that at least two preset fleet control strategies are selected. For example, if the current comprehensive risk assessment result is a medium risk level, a lead vehicle deceleration strategy is selected based on the correlation. To supplement this with at least two strategies, a strategy of increasing following distance with the next lower intervention intensity is selected from the preset strategy set.

[0079] In step S4, for each preset platoon control strategy, the vehicle's state change process after executing that strategy is simulated, achieved as follows: The simulation process is based on a discrete-time recursive framework similar to the probabilistic evolution simulation in step S3, but the generation method of control commands differs. The initial state of the simulation is the vehicle's current motion state, including its current speed, current longitudinal acceleration, current yaw rate, and lateral position within the current lane. The future simulation time period remains consistent with the preset future time period in step S3, for example, 5 seconds. The time step remains consistent with the time step in step S3, for example, 0.1 seconds. For each selected preset platoon control strategy, a deterministic control command sequence is generated according to the definition of the preset platoon control strategy, replacing the random sampling process in step S3. Taking the lead car deceleration strategy as an example, the lead car deceleration strategy is defined as follows: During the future simulation period, the longitudinal acceleration command of the lead car is kept at a constant negative value, for example, -1 m / s², until the vehicle speed decreases to a preset target safe speed. Afterward, the lead car maintains the target safe speed, and the lateral control command remains the command to maintain the current lane center. Taking the increased following distance strategy as an example, the increased following distance strategy is defined as follows: Adjust the target following distance between the lead car and the virtual reference point ahead, for example, increasing it from 2 seconds to 3 seconds. Based on this target following distance and the current relative distance, a proportional-derivative controller calculates the required longitudinal acceleration command in real time, and the lateral control command remains the command to maintain lane center. At each time step, a determined increment of longitudinal acceleration command and steering wheel angle command is calculated according to the preset fleet control strategy definition. Using the same vehicle dynamics model and recursive formula as in step S3, the vehicle state of the previous time step and the deterministic control command increment calculated in the current time step are used as inputs to calculate the vehicle state at the end of the current time step. This process is repeated until the calculations for all time steps within the entire future simulation period are completed, recording the vehicle state at each time step, including longitudinal position, lateral position, velocity, and longitudinal acceleration. These states, arranged chronologically, constitute the vehicle state change process after implementing the preset platoon control strategy. The above simulation process is executed independently once for each selected preset platoon control strategy.

[0080] Step S4 analyzes the conformity between the simulated vehicle state change process and the kinematic constraints for safe following of the convoy, achieved as follows: The kinematic constraints for safe following of the convoy are consistent with those defined in step S3, including minimum safe distance conditions, lane keeping conditions, and maximum deceleration conditions. The minimum safe distance condition uses the minimum safe distance threshold defined in step S3. The lane keeping condition uses the lane departure threshold defined in step S3. The maximum deceleration condition uses the maximum deceleration threshold defined in step S3. For the simulated vehicle state change process corresponding to a preset convoy control strategy, time-series state data is extracted during the vehicle state change process. The time-series state data includes the distance between the vehicle and the virtual reference point ahead, the offset of the vehicle's lateral position from the lane centerline, and the vehicle's longitudinal acceleration at each time step. The analysis is performed point-by-point. For the first time point in the vehicle state change process, it is checked whether the distance between the vehicle and the virtual reference point ahead at this time point is greater than the minimum safe distance threshold. It is checked whether the absolute value of the vehicle's lateral position offset at this time point is less than the lane departure threshold. It is checked whether the vehicle's longitudinal acceleration at this time point is greater than the maximum deceleration threshold. If all three checks at a given time point are "yes," then that time point is recorded as conforming to the constraints. Move to the next time point and repeat the same three checks. Iterate through all time points in the vehicle state change process; for example, 50 time points corresponding to a 5-second simulation duration and a 0.1-second step size, and count the number of time points where all checks are "yes." Conformity is quantified using a conformity rate metric. The conformity rate equals the number of time points conforming to the constraints divided by the total number of time points in the vehicle state change process. For example, if the total number of time points is 50 and the number of time points conforming to the constraints is 45, then the conformity rate is 45 / 50 = 0.9. This conformity rate characterizes the conformity of the simulated vehicle state change process to the kinematic constraints of the convoy safety following.

[0081] In step S4, based on compliance, the expected improvement effect of each preset fleet control strategy on meeting the kinematic constraints of safe fleet following is evaluated, which is achieved as follows: The expected improvement effect is quantitatively evaluated by calculating an improvement score. The calculation of the improvement score requires two inputs. The first input is the conflict level calculated in step S3, which reflects the baseline risk level without any intervention. The second input is the compliance rate calculated in step S4 for a specific preset fleet control strategy, which reflects the expected safety level after implementing the preset fleet control strategy. The formula for calculating the improvement score is: Improvement Score = Compliance Rate - Conflict Level. For example, if the conflict level is 0.3 and the compliance rate is 0.9, then the improvement score = 0.9 - 0.3 = 0.6. The improvement score is a numerical value. The larger the improvement score, the better the expected improvement effect of the preset fleet control strategy relative to the baseline risk level. If the improvement score is negative, it means that the expected effect of the preset fleet control strategy is worse than no intervention. For each preset fleet control strategy selected in step S4, its corresponding improvement score is calculated independently. All calculated improvement scores constitute a quantitative evaluation of the expected improvement effect of each preset fleet control strategy. This evaluation result is used for strategy comparison and selection in the subsequent step S5.

[0082] According to an embodiment of the present invention, the comparison of the expected improvement effect evaluated for each preset fleet control strategy in step S5 is implemented in the following manner. The expected improvement effect has been quantified by improvement scores in step S4. The comparison process involves sorting all improvement scores calculated in step S4. Each improvement score corresponds one-to-one with each preset fleet control strategy selected in step S4. The comparison operation is performed in a data structure. Each element in the data structure contains two parts. The first part stores the identifier of the preset fleet control strategy. The second part stores the improvement score corresponding to the preset fleet control strategy. The identifier is the name or internal code of the preset fleet control strategy. The comparison logic is to find the maximum value among all improvement scores. The comparison algorithm starts from the first element in the data structure and temporarily sets the improvement score stored in the first element as the current maximum value. Then, the second element in the data structure is accessed. The improvement score stored in the second element is compared with the current maximum value. If the improvement score stored in the second element is greater than the current maximum value, the improvement score stored in the second element is updated to the new current maximum value, and the preset fleet control strategy identifier corresponding to the second element is recorded. If the improvement score stored in the second element is less than or equal to the current maximum value, the current maximum value is kept unchanged. The process continues by accessing the next element in the data structure, repeatedly comparing the improvement score stored in the element with the current maximum value and updating the current maximum value as needed, until all elements in the data structure have been accessed. After traversal, the current maximum value recorded is the best value among all improvement scores, and the corresponding preset fleet control strategy identifier is also recorded. This process compares the expected improvement effects of all preset fleet control strategies, identifying the best value and its corresponding strategy. For example, in step S4, a lead car deceleration strategy and a follow distance increase strategy were simulated, and improvement scores of 0.6 and 0.4 were calculated respectively. Therefore, the above comparison process determines that an improvement score of 0.6 is the maximum value, corresponding to the lead car deceleration strategy.

[0083] In step S5, the preset fleet control strategy with the best expected improvement effect is selected as the recommended control strategy from the simulated preset fleet control strategies. This is achieved in the following way: The selection operation is directly based on the results of the comparison process described above. During the comparison process, the maximum improvement score and its corresponding preset fleet control strategy identifier have been identified. The selection operation formally determines the preset fleet control strategy represented by this identifier as the output of this step, i.e., the recommended control strategy. If multiple preset fleet control strategies have the same improvement score and are all the maximum value, one of these strategies is selected as the recommended control strategy according to the preset priority rules. The priority rules are predefined. One way to define the priority rules is to assign a static priority value to each preset fleet control strategy. The static priority value is stored in a configuration file. The smaller the static priority value, the higher the priority. When multiple strategies have the same improvement score, the preset fleet control strategy with the smallest static priority value is selected as the recommended control strategy. Another way to define the priority rules is based on the intervention intensity of the preset fleet control strategy. The intervention intensity is ranked according to the severity of the change in the fleet driving state caused by the strategy. For example, the intervention intensity of the lead car deceleration strategy is higher than that of the follow distance increase strategy. The rule is defined as selecting the preset fleet control strategy with the weakest intervention intensity. Once selected, the recommended control strategy is stored in a variable for subsequent steps to generate fleet risk information. This process ensures that the selected recommended control strategy is the optimal choice based on quantitative assessment.

[0084] Step S5 generates fleet risk information containing the recommended control strategy and its corresponding expected improvement effect information, achieved as follows: Fleet risk information is a structured data object used to encapsulate key decision-making information that needs to be passed to following vehicles. This data object contains multiple fields. The first field is the recommended control strategy field. The content of the recommended control strategy field is the complete definition or unique identifier of the recommended control strategy selected in step S5. The complete definition includes the strategy type and specific parameters. For example, the strategy type is the lead vehicle deceleration strategy, and the specific parameters include the target deceleration value and the target safe speed value. If a unique identifier is used, it is necessary to ensure that the following vehicles can query the complete strategy definition based on the unique identifier through the locally stored same strategy library. The second field is the expected improvement effect information field. The content of the expected improvement effect information field is the improvement score corresponding to the recommended control strategy. The improvement score is the value calculated in step S4 and confirmed to match the recommended control strategy by comparison in step S5. The third field is an optional risk level field. The content of the risk level field is the comprehensive risk assessment result obtained in step S3, i.e., the risk level, such as a medium risk level. Including this information helps the following vehicles to have a more comprehensive understanding of the risk background. The fourth field is an optional timestamp field. The timestamp field records the system time that generated this fleet risk information, used for information synchronization and timeliness assessment. The generation process involves assembling the above fields and their values ​​according to a predetermined data format. This predetermined data format can be JavaScript object notation. The assembly operation combines the names and values ​​of the recommended control strategy field, expected improvement effect information field, risk level field, and timestamp field into a string according to the syntax rules of JavaScript object notation. After assembly, the resulting string is the final fleet risk information. This fleet risk information is the content entity sent via vehicle-to-vehicle communication in subsequent step S6. For example, the generated fleet risk information might include the following: the recommended control strategy is the lead vehicle's deceleration strategy and parameters: target deceleration = -1 m / s² and target speed = 15 m / s²; the expected improvement effect information is an improvement score of 0.6; the risk level is medium risk; and the timestamp is 2025-10-27T08:30:00Z. This information provides clear action instructions and decision-making basis for the following vehicles.

[0085] According to an embodiment of the present invention, the determination of the following vehicle as the receiver in step S6 based on the formation structure of the cooperative driving fleet is achieved in the following manner. The formation structure information of the cooperative driving fleet is derived from the fleet formation information obtained in step S2. The fleet formation information contains an ordered list of vehicle identifiers. The process of determining the receiver first requires reading the vehicle identifier of the current vehicle. The vehicle identifier of the current vehicle has been obtained during the parsing process in step S2. Next, the list of vehicle identifiers in the fleet formation information is traversed. For each vehicle identifier in the list of vehicle identifiers, it is compared with the vehicle identifier of the current vehicle. The comparison operation uses byte-level exact matching. If a vehicle identifier does not match the vehicle identifier of the current vehicle, the vehicle corresponding to that vehicle identifier is identified as a potential receiver following vehicle. If a vehicle identifier matches the vehicle identifier of the current vehicle, that vehicle identifier is skipped. By traversing the entire list of vehicle identifiers, all vehicle identifiers that do not match the vehicle identifier of the current vehicle are collected. The set of these collected vehicle identifiers is determined as the receiver following vehicle for this communication. For example, the vehicle identifier list in the convoy information contains three identifiers, such as 1001, 1002, and 1003. This vehicle's identifier is 1001. By iterating and comparing these identifiers, it is determined that 1002 and 1003 are inconsistent with this vehicle's identifier. Therefore, the two vehicles corresponding to 1002 and 1003 are identified as the receiving vehicles. This method ensures that information is sent to all other vehicles in the convoy except for this vehicle.

[0086] In step S6, the fleet risk information, including the recommended control strategy and its corresponding expected improvement effect information, is encapsulated into a data frame conforming to the vehicle-to-vehicle communication protocol format. This is achieved through the following method: The vehicle-to-vehicle communication protocol format is a predefined structured format used to ensure that both communicating parties can correctly parse the data. The data frame is the basic transmission unit of this format. The encapsulation operation begins with the fleet risk information generated in step S5. The fleet risk information is a structured data object. The encapsulation process first creates an empty data frame buffer. The data frame consists of three parts: a frame header, a payload, and a frame trailer. The frame header needs to be filled with several fields. The first field is the message type field. The message type field identifies that the data frame carries fleet risk information. The message type field uses a predefined enumerated value, for example, defining the value 100 as the fleet risk information type. The second field is the source address field. The source address field is filled with the address of the vehicle's communication module. The address of the vehicle's communication module is, for example, the media access control address of the vehicle's wireless communication module. The third field is the destination address field. The processing of the destination address field depends on the communication mode. If broadcast mode is used, the broadcast address is filled in. The broadcast address is, for example, a media access control address consisting entirely of 255. If unicast or multicast mode is used, the list of recipient vehicle addresses determined in step S6 needs to be processed and filled in. The frame header also includes a sequence number field and a data length field. The sequence number field is used to identify the message sending order. The sequence number field increments by 1 with each data frame sent. The data length field is used to indicate the byte length of the subsequent payload portion. The payload portion is used to store the actual fleet risk information to be transmitted. The fleet risk information data object generated in step S5 is serialized into a byte stream according to its predetermined data format. The predetermined data format is, for example, JavaScript object notation. The serialization operation converts each field name and value in the fleet risk information into a string conforming to JavaScript object notation syntax, and then converts the string into a byte stream. This byte stream is completely placed into the payload portion of the data frame buffer. The frame tail portion contains a cyclic redundancy check (CRC) code for error detection. The CRC code is calculated based on all bytes in the frame header and payload portion. A predetermined CRC algorithm is used to calculate the check value. The predetermined cyclic redundancy check (CRC) algorithm is, for example, the CRC-32 algorithm. When calculating the CRC check code, starting with the first byte of the frame header, each byte of the header and each byte of the payload are processed sequentially, generating a 4-byte check value through iterative calculation. This 4-byte check value is then filled into the end portion of the data frame buffer. At this point, a data frame containing a complete header, payload, and end portion is encapsulated.For example, the encapsulated data frame may include a frame header with the following content: message type equal to 100, source address equal to MAC_A, destination address equal to broadcast address, sequence number equal to 5, and data length equal to 150; payload content is a serialized fleet risk information JSON string; and frame trailer content is a CRC32 checksum equal to 0x89ABCDEF.

[0087] In step S6, encapsulated data frames are sent to the designated following vehicles via the vehicle-to-vehicle communication link to complete the transmission of fleet risk information. This is achieved through the following method: The vehicle-to-vehicle communication link refers to the wireless transmission channel used for direct communication between vehicles. The physical layer and link layer of the vehicle-to-vehicle communication link follow specific communication standards, such as dedicated short-range communication technology based on IEEE 802.11p. The transmission process is executed in the vehicle's onboard communication unit. First, the encapsulated data frame from step S6 is submitted from the memory buffer to the driver of the communication unit. The driver is responsible for further processing the data frame according to the requirements of the underlying communication protocol. Further processing includes adding a physical layer preamble and performing channel coding. Then, the communication unit determines the transmission mode based on the destination address field in the data frame header. If the destination address is a broadcast address, the communication unit transmits the data frame in broadcast mode on the predetermined communication channel. In broadcast mode, all vehicles within the communication signal coverage area can receive the data frame. If the destination address is a specific unicast or multicast address, the communication unit sends the data frame to one or a group of designated receiving following vehicles according to the routing table. The transmission operation requires setting transmit power and communication frequency parameters. The transmit power is dynamically adjusted based on communication distance and environment. This dynamic adjustment follows relevant communication standards, such as using lower transmit power within a 100-meter range. The communication frequency uses a specific frequency band allocated for vehicle-to-vehicle safety messages, such as the 5.9 GHz band. Data frames are modulated onto a carrier wave and converted into radio waves by an antenna before transmission. After transmission, the sender starts an acknowledgment timer. The duration of this timer is, for example, set to 100 milliseconds. If no acknowledgment message is received from the receiver before the timer expires, the sender decides whether to retransmit the data frame according to a retransmission strategy. This retransmission strategy may involve a maximum of three retransmissions. After the receiving vehicle's antenna receives the radio signal, it demodulates and decodes it to reconstruct the data frame. The receiver verifies data integrity by checking the cyclic redundancy check (CRC) code at the end of the frame. Upon successful verification, the receiver parses the data frame, extracts the convoy risk information from the payload, and completes the full transmission of convoy risk information from the lead vehicle to the following vehicles. This process enables the reliable distribution of critical decision-making information within the convoy.

[0088] Example 2: Figure 2A schematic diagram of the abnormal driving behavior recognition system based on multimodal data fusion of the present invention is provided. The abnormal driving behavior recognition system based on multimodal data fusion includes:

[0089] The anomaly detection module is used to collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data.

[0090] The role determination module is used to determine whether the vehicle is the lead vehicle in the collaborative driving fleet;

[0091] The risk assessment module is used to simulate the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state based on the pattern and intensity of the abnormal state, and based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, to obtain the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet.

[0092] The strategy simulation module is used to simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet, based on the comprehensive risk assessment results.

[0093] The information generation module is used to select the preset fleet control strategy with the best expected improvement effect from the simulated preset fleet control strategies as the recommended control strategy, and generate fleet risk information containing the recommended control strategy.

[0094] The information sending module is used to send fleet risk information to the following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

[0095] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0096] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0097] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. Computer-readable storage media can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0098] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0100] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0102] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0104] In conclusion, the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying abnormal driving behavior based on multimodal data fusion, characterized in that, include: S1. Collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data; S2. Determine whether this vehicle is the lead vehicle in the cooperative driving fleet; S3. When the vehicle is the lead vehicle and an abnormal state is detected, based on the pattern and intensity of the abnormal state, the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state is simulated through probability evolution. Based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet is obtained. S4. Based on the comprehensive risk assessment results, simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet. S5. From the simulated preset fleet control strategies, select the preset fleet control strategy with the best expected improvement effect as the recommended control strategy, and generate fleet risk information containing the recommended control strategy. S6. Send fleet risk information to following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

2. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S1 includes: Collect data from the vehicle's in-vehicle camera and vehicle control data; The driver's facial and posture features are extracted from the in-vehicle camera data, and the vehicle control feature vector representing the driver's control behavior is extracted from the vehicle control data. The driver's facial and posture features are fused with the vehicle control feature vector to generate a fused feature vector; Abnormal states of the driver in this vehicle are identified based on fused feature vectors.

3. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S2 include: Obtain fleet formation information describing the vehicle formation order and roles in the cooperative driving fleet; The vehicle's assigned position in the formation sequence is determined from the vehicle formation information. Based on the rule that the lead vehicle in a cooperative driving fleet is always at the head of the formation sequence, determine whether the vehicle's position in the formation sequence meets the conditions for a lead vehicle.

4. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S3 include: Based on the pattern and intensity of the abnormal state, determine its expected impact range and probability distribution on vehicle control commands. Based on the expected scope of impact and its probability distribution, combined with the current vehicle motion state, multiple random sampling simulations are used to generate multiple possible motion trajectories of the vehicle in the future within a preset time period, forming the probability distribution of the future motion trajectory. From the probability distribution of future motion trajectories, we statistically identify the motion trajectories that lead to violations of the kinematic constraints of safe following of the vehicle fleet, and calculate their proportion to characterize the degree of conflict. Based on the degree of conflict, a comprehensive risk assessment result for the cooperative driving fleet under abnormal conditions is obtained by quantification.

5. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 4, characterized in that, Based on the pattern and intensity of the abnormal state, determine its expected impact range and probability distribution on vehicle control commands. This is achieved by: based on the pre-established correspondence between abnormal states and vehicle control parameter disturbances, querying the expected impact range and probability distribution on longitudinal and lateral control commands of the vehicle that match the pattern and intensity of the current abnormal state.

6. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S4 includes: Based on the comprehensive risk assessment results, select at least two preset fleet control strategies from the pre-set strategy set; For each preset fleet control strategy, simulate the vehicle state change process after the vehicle executes the preset fleet control strategy; Analyze the consistency between the simulated vehicle state change process and the kinematic constraints for safe following of the fleet; Based on the applicable conditions, evaluate the expected improvement effect of each preset fleet control strategy on meeting the kinematic constraints of safe following of the fleet.

7. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 6, characterized in that, The analysis of the consistency between the simulated vehicle state change process and the kinematic constraints of safe following of the convoy is achieved by comparing the time-series state data of the simulated vehicle state change process with the threshold conditions constituting the kinematic constraints of safe following of the convoy point by point, and statistically quantifying the degree of consistency with each threshold condition.

8. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S5 include: Compare the expected improvement effects evaluated for each preset fleet control strategy; From the simulated preset fleet control strategies, the preset fleet control strategy with the best expected improvement effect is selected as the recommended control strategy; Generate fleet risk information that includes recommended control strategies and their corresponding expected improvement effects.

9. The abnormal driving behavior recognition method based on multimodal data fusion according to claim 1, characterized in that, S6 include: Based on the platooning structure of the cooperative driving fleet, the following vehicles as receivers are determined; The fleet risk information, which includes the recommended control strategy and its corresponding expected improvement effect, is encapsulated into a data frame that conforms to the vehicle-to-vehicle communication protocol format. Encapsulated data frames are sent to the designated following vehicles via vehicle-to-vehicle communication links to complete the transmission of fleet risk information.

10. An abnormal driving behavior recognition system based on multimodal data fusion, used to implement the abnormal driving behavior recognition method based on multimodal data fusion as described in any one of claims 1-9, characterized in that, include: The anomaly detection module is used to collect multimodal sensor data of the vehicle and identify abnormal states of the driver based on the multimodal sensor data. The role determination module is used to determine whether the vehicle is the lead vehicle in the collaborative driving fleet; The risk assessment module is used to simulate the probability distribution of the vehicle's future trajectory under the continuous action of the abnormal state based on the pattern and intensity of the abnormal state, and based on the degree of conflict between the probability distribution and the kinematic constraints of the fleet's safe following, to obtain the comprehensive risk assessment result of the abnormal state on the cooperative driving fleet. The strategy simulation module is used to simulate the expected improvement effect of at least two preset fleet control strategies on meeting the kinematic constraints of safe following of the fleet, based on the comprehensive risk assessment results. The information generation module is used to select the preset fleet control strategy with the best expected improvement effect from the simulated preset fleet control strategies as the recommended control strategy, and generate fleet risk information containing the recommended control strategy. The information sending module is used to send fleet risk information to the following vehicles in the cooperative driving fleet via vehicle-to-vehicle communication.

Images