Man-machine co-driving control method considering intention of driver

Abstract

The invention discloses a man-machine co-driving control method considering driver intention, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining visual data of a driver in a historical driving process, marking a lane changing intention, forming a training data set, training a CNN network through the training data set, and obtaining a driver lane changing intention prediction model; and obtaining visual data of the driver in real time and predicting a lane changing intention, if the prediction result is that the lane changing intention exists, calculating a man-machine intervention penalty factor, and measuring the conflict degree between the driver and the automatic driving system. And based on the man-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, the operation priority is dynamically adjusted, and man-machine co-driving control over the target vehicle is achieved. According to the method, the visual data of the driver is obtained, the intention of the driver is accurately predicted in combination with the CNN network, the man-machine control weight is dynamically adjusted, safer and more efficient co-driving experience is achieved, man-machine conflicts are reduced, and the adaptability and reliability of an automatic driving system are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a human-machine co-driving control method that takes the driver's intention into consideration. Background Art

[0002] With the rapid development of intelligent transportation systems, human-machine cooperative driving (HMCD) technology, as an important form of combining autonomous driving with manual driving, has attracted increasing attention. In HMCD, the driver and the autonomous driving system dynamically collaborate to jointly control the vehicle. This model not only improves driving safety but also enhances driving flexibility and user experience. However, current HMCD systems still face numerous technical challenges in practical application, the most significant of which is human-machine conflict caused by inaccurate driver intent recognition.

[0003] In existing human-machine co-driving systems, the driver and the autonomous driving system may disagree on the priority of vehicle operations in specific scenarios (such as lane changes and obstacle avoidance). For example, when the driver plans to change lanes and the autonomous driving system fails to detect the driver's intention in time, the autonomous driving system may continue to maintain the original path planning, causing the driver to feel that the system is "uncooperative" or even forcibly intervene. This human-machine conflict may not only cause discomfort to the driver but also increase the risk of traffic accidents. The main reason for this is that the existing autonomous driving system has insufficient methods for recognizing driver intentions, especially in complex driving scenarios.

[0004] Therefore, there is an urgent need for a human-machine co-driving control method that takes the driver's intention into consideration to solve the problem of inaccurate driver intention recognition in existing human-machine co-driving systems. Summary of the Invention

[0005] The purpose of this application is to provide a human-machine co-driving control method that takes the driver's intention into consideration, which can improve the intention recognition ability and collaborative control performance of the human-machine co-driving system, enhance the adaptability, and is suitable for various driving scenarios such as urban roads, highways and congestion, and has wide application value.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] This application provides a human-machine co-driving control method that takes the driver's intention into consideration, including:

[0008] Acquire visual data of multiple drivers during historical driving and annotate lane change intentions; the visual data includes the driver's head posture, gaze direction, and eye movement trajectory; the lane change intention includes "intention to change lanes" and "no intention to change lanes"; the "intention to change lanes" includes "intention to change lanes to the left" and "intention to change lanes to the right";

[0009] Combine the visual data and labeled lane-changing intentions of multiple drivers during their historical driving process Figure 1 To form a training data set;

[0010] Using the training data set to train a CNN network to obtain a driver lane change intention prediction model;

[0011] Acquire the driver's predicted visual data of the target vehicle in real time during driving, and input it into the driver's lane change intention prediction model to obtain the lane change intention prediction result;

[0012] If the lane change intention prediction result indicates that there is a lane change intention, a human-machine intervention penalty factor is calculated based on the lane change intention prediction result; the human-machine intervention penalty factor is used to measure the degree of conflict between the driver and the automatic driving system control in the target vehicle;

[0013] Based on the human-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, the operation priorities of the driver and the automatic driving system during the driving process of the target vehicle are dynamically adjusted to perform human-machine co-driving control of the target vehicle; the two-degree-of-freedom vehicle dynamics model is used to obtain the target vehicle status in real time and is also used to predict the target vehicle status based on the lane change intention prediction result; the target vehicle status includes lateral displacement, yaw angular velocity, lateral acceleration, and path trajectory.

[0014] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0015] This application provides a human-machine co-driving control method that takes the driver's intention into account. By acquiring the driver's visual data from historical driving experiences and annotating lane change intentions, this method addresses the inaccurate driver intention recognition problem in the prior art and enables accurate prediction of the driver's lane change intention. This data is used to train a CNN (Convolutional Neural Network) to generate a driver lane change intention prediction model. This model can acquire and analyze the driver's visual data in real time to predict lane change intention. This process improves the accuracy of lane change intention recognition and provides a reliable basis for dynamically adjusting human-machine control weights. By calculating a human-machine intervention penalty factor, this method addresses the issue of how to measure and handle conflicts between the driver and the autonomous driving system in a human-machine co-driving environment, achieving the goals of reducing conflicts and improving driving safety. When the prediction result indicates a lane change intention, the operation priorities of the driver and the autonomous driving system are dynamically adjusted based on this factor. By dynamically adjusting the operation priorities based on the human-machine intervention penalty factor and a two-degree-of-freedom vehicle dynamics model, this method addresses the issue of how to balance driver operation and autonomous driving system control to achieve a smooth transition during the lane change process, achieving smooth control of the target vehicle and improving co-driving comfort. The two-degree-of-freedom vehicle dynamics model is not only used to obtain the target vehicle state in real time, but also to predict the target vehicle state based on the lane change intention prediction results, including lateral displacement, yaw rate, lateral acceleration, path trajectory, etc., thereby achieving more precise and accurate vehicle control. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a human-machine co-driving control method that takes the driver's intention into consideration, provided in one embodiment of the present application;

[0018] Figure 2 A flowchart of a human-machine co-driving control method that takes the driver's intention into consideration, provided in another embodiment of the present application;

[0019] Figure 3 A schematic diagram of the CNN network principle of the driver lane change intention prediction model provided in one embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] Research on driver intention recognition in related technologies primarily relies on vehicle dynamic parameters (such as steering wheel angle and pedal operation) and physiological data (such as heart rate and brain waves) to infer the driver's operating intention. Although these methods are effective under ideal conditions, they face the following problems in practical applications:

[0022] 1. Sensing equipment is complex and costly: Intent recognition methods that rely on physiological data typically require the use of complex sensor arrays, such as brainwave sensors or heart rate monitoring devices, which not only increases hardware costs but also places high demands on vehicle modifications.

[0023] 2. Susceptible to environmental interference and individual differences: Physiological data and dynamic parameters are easily affected by the driving environment (such as road vibration and light changes), and there are significant differences in the operating habits and physiological characteristics of different drivers, resulting in insufficient versatility and stability of the driver lane change intention prediction model.

[0024] 3. Insufficient analysis of driver head and eye behavior: Related technologies fail to fully utilize key features that directly reflect driver attention and intent, such as head posture and eye movement. These features can provide more direct information about driver intent in specific scenarios, such as lane changes and obstacle avoidance.

[0025] 4. Although driving intention recognition methods based on head and eye information have made some progress, there is still room for improvement in robustness in complex dynamic scenes, cross-individual adaptability, and multimodal data fusion.

[0026] Therefore, there is an urgent need for a comprehensive method that combines optical perception equipment and a driver-in-the-loop experimental platform to establish an efficient and accurate driver intention recognition model and a human-machine collaborative control mechanism, thereby providing a safer and more efficient solution for human-machine co-driving in complex traffic environments.

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] In an exemplary embodiment, Figure 1As shown, a human-machine co-driving control method considering the driver's intention is provided, including the following steps 101 to 106. Among them:

[0030] Step 101: Acquire visual data of multiple drivers during historical driving processes and annotate lane change intentions; the visual data includes the driver's head posture, gaze direction, and eye movement trajectory; the lane change intention includes having a lane change intention and not having a lane change intention; the lane change intention includes having a lane change intention to the left and having a lane change intention to the right.

[0031] Step 102: Combine the visual data and the lane change intentions of multiple drivers during their historical driving process. Figure 1 constitute the training data set.

[0032] Step 103: Use the training data set to train a CNN network to obtain a driver lane-changing intention prediction model.

[0033] Step 104 , obtaining the driver's visual data to be predicted during the target vehicle driving process in real time, and inputting the data into the driver's lane change intention prediction model to obtain a lane change intention prediction result.

[0034] Step 105: If the lane change intention prediction result indicates that there is a lane change intention, a human-machine intervention penalty factor is calculated based on the lane change intention prediction result; the human-machine intervention penalty factor is used to measure the degree of conflict between the driver and the automatic driving system control in the target vehicle.

[0035] Step 106, based on the human-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, dynamically adjust the operation priorities of the driver and the automatic driving system during the driving process of the target vehicle, and perform human-machine co-driving control of the target vehicle; the two-degree-of-freedom vehicle dynamics model is used to obtain the target vehicle state in real time and is also used to predict the target vehicle state based on the lane change intention prediction result; the target vehicle state includes lateral displacement, yaw angular velocity, lateral acceleration, and path trajectory.

[0036] By implementing the above steps 101 to 106, the present application can effectively combine the driver's intentions and the capabilities of the autonomous driving system to achieve a more intelligent, safe and comfortable driving experience.

[0037] In another exemplary embodiment of the present application, marking the lane change intention in step 101 specifically includes: using a fixed time window and a heading angle threshold to mark the lane change behavior.

[0038] If the change in heading angle exceeds a threshold within a fixed time window and meets preset lane-changing conditions, it is marked as a lane-changing behavior. The preset lane-changing conditions include the driver turning his head to the left or right of the target lane; the eye movement trajectory showing the target lane is fixed for more than a preset time threshold; and the lateral displacement of the target vehicle deviates toward the target lane.

[0039] In another exemplary embodiment of the present application, Figure 2 As shown, the CNN network in step 103 includes: an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer connected in sequence. The convolutional layer is the detection layer, and the pooling layer is also called the downsampling layer.

[0040] The input layer receives visual data and generates a raw visual data matrix. As a data entry point, the input layer can accept digital or image signals, retaining the original features of the input image. The image processing network performs preliminary data processing, which can optimize training accuracy, prevent data contamination, reduce training time, and increase convergence rate.

[0041] The convolutional layer is used to extract the multi-dimensional features of the original visual data matrix using the following formula to obtain a feature map:

[0042]

[0043] Among them, a i，j is the value of the i-th row and j-th column in the feature map, x i+m,j+n is the visual data of the i+mth row and j+nth column in the original visual data matrix, ω m,n is the weight value of the convolution kernel in the mth row and nth column, ω b is the first bias term, and f is the sigmoid activation function.

[0044] Among them, in this implementation method, the convolution layer extracts the feature signal of the original data through convolution calculation to promote subsequent operations, and can deeply analyze the local data block to obtain more abstract features. Different convolution kernels can filter different feature data. It is a translation linear operation, which is formed by the local weighting of itself and the input layer. The weight set is selected according to the nature of the input signal, and the input signal is convolved according to the set step size to obtain a new feature result. In this embodiment, the convolution layer extracts features from the input data through the convolution kernel (Filters), and the output new features are specifically: (1) Local spatial features: such as edge contours in the head posture image (such as the trend of changes in the head turning angle); the gaze point concentration area in the eye movement trajectory map (such as the high-frequency gaze area of the target lane). (2) Abstract semantic features: through the stacking of multiple layers of convolution, the driver's lane change intention prediction model can extract higher-level semantic information, such as the combination pattern of "head left tilt + gaze at the left lane"; "multiple glances at the right lane + steering wheel right turn" temporal association. These features abstract patterns that are strongly related to lane change intention from the original data, providing a basis for discrimination for subsequent classification.

[0045] The pooling layer is used to compress the feature map output by the convolutional layer, producing a compressed feature map. The pooling layer compresses and extracts the convolutional layer output, improving computational efficiency. This can be understood as reducing the amount of feature input data. While it preserves the characteristics of the original data, it can blur the features. Common pooling methods include maximum, random, and average pooling. Maximum pooling takes the maximum value of the convolutional output matrix elements to form a new matrix; random pooling constructs a matrix by randomly extracting elements; and average pooling averages the matrix values to form a new matrix. Pooling can effectively enhance the generalization capability of driver lane change intention prediction models.

[0046] The fully connected layer is used to perform weighted combination of different features in the compressed feature map using the following formula:

[0047] y i′ =f(∑ω i′ x i′ +b).

[0048] Among them, y i′ is the fully connected layer output of the i′th feature, ω i′ is the weight of the i′th feature, x i′ is the value of the i′th feature in the compressed feature map, and b is the second bias term. The multi-dimensional features (such as head posture features and eye movement trajectory features) extracted by the convolution layer and the pooling layer are expanded into a one-dimensional vector.

[0049] The output layer is used to generate the probabilities of three types of lane change intentions through a Softmax function based on the output of the fully connected layer.

[0050] As an optional implementation, the output layer is used to generate the probabilities of three types of lane change intentions through a Softmax function, specifically including:

[0051]

[0052] Among them, P k′ is the probability of the k′th lane change intention, k′={1,2,3} corresponds to no lane change intention, left lane change intention, and right lane change intention, respectively, and K is the total number of lane change intention categories. is the exponential operation of the original output probability of the k′th lane-changing intention, is the exponential sum of all lane-changing intention probabilities.

[0053] If P2>preset probability threshold, it is determined to be an intention to change lanes to the left.

[0054] If P3>preset probability threshold, it is determined to be an intention to change lanes to the right.

[0055] If both of the above conditions are not met, it is determined that there is no lane change intention.

[0056] As an optional implementation, the training termination condition of the driver lane change intention prediction model is that the comprehensive performance indicators meet the standards, rather than simply relying on the number of iterations.

[0057] Condition 1: Convergence of validation set accuracy. When the lane change intention recognition accuracy of the driver lane change intention prediction model on the validation set does not improve for N consecutive epochs (e.g., 5), training is terminated.

[0058] Condition 2: The loss function is stable. The training loss (e.g., cross entropy loss) drops to a preset threshold (e.g., 0.05) and the fluctuation range is less than ε (e.g., ±0.01).

[0059] Condition 3: Passing real-world scenario testing. In the driver-in-the-loop test platform, the driver's lane change intention prediction model must achieve a minimum success rate (e.g., 95%) in simulated lane change scenarios (e.g., highways and congested roads) to be considered trained.

[0060] As an optional implementation method, the CNN network is trained using a transfer learning method to reduce intention recognition errors caused by individual differences and improve the versatility and robustness of the driver's lane change intention prediction model.

[0061] In another exemplary embodiment of the present application, in step 105, the human-machine intervention penalty factor is calculated based on the lane change intention prediction result, specifically including:

[0062] When the lane change intention prediction result is consistent with the lane change intention of the autonomous driving system Figure 1 When the lane change probability exceeds the preset lane change threshold, the human-machine intervention penalty factor is set to 0.

[0063] When the lane change intention prediction result is inconsistent with the lane change intention of the autonomous driving system, or the lane change probability is less than the preset lane change threshold, the human-machine intervention penalty factor is calculated according to the following formula:

[0064] F q =k(1-C q ).

[0065] Among them, F q is the human-machine intervention penalty factor at the qth moment, C q is the consistency coefficient between the lane changing intention prediction result at the qth moment and the lane changing intention of the automatic driving system, and k is the penalty factor adjustment coefficient.

[0066] In another exemplary embodiment of the present application, the lane change probability is obtained based on the target lane condition.

[0067] The target lane conditions include: whether the target lane has enough space to accommodate the target vehicle changing lanes; whether the target lane line is a dotted line; whether the relative speed of the front vehicle and the rear vehicle in the target lane is less than a preset maximum speed threshold, and whether the distance between the front vehicle and the rear vehicle in the target lane is greater than a preset minimum safety distance threshold; whether the predicted collision time between the target lane vehicle and the target vehicle is greater than a preset safety time threshold; whether the current speed of the target vehicle is less than a safety speed threshold; whether the traffic signal allows the lane change operation; whether the weather and road conditions are suitable for the lane change operation; and whether the lateral acceleration of the target vehicle during the predicted lane change is less than a preset lateral acceleration threshold and whether the yaw angular velocity is less than a preset yaw angular velocity threshold.

[0068] As an optional implementation method, "target lane conditions are met" means that the autonomous driving system confirms that the target lane meets the conditions for safe lane change in terms of physical space, traffic dynamics, and vehicle status through multi-source data fusion and real-time analysis. Specifically, it includes the following core elements:

[0069] (1) Physical space safety of the target lane.

[0070] ① Available space detection: Vehicle-mounted sensors (such as millimeter-wave radar and lidar) or cameras are used to detect in real time whether the target lane has sufficient space to accommodate the vehicle changing lanes, ensuring that there are no static obstacles (such as broken-down vehicles and roadblocks) or dynamic obstacles (such as motorcycles and bicycles) occupying the lane change path.

[0071] ② Lane marking recognition: Confirm that the target lane marking is a dotted line (lane changing is allowed). If it is a solid line or a double yellow line, it is determined that "conditions are not met" and lane changing is prohibited.

[0072] (2) Dynamic interaction safety with surrounding vehicles.

[0073] ① Relative speed and distance. Calculate the relative speed and distance between the leading and following vehicles in the target lane to ensure that the minimum safe distance is met (e.g., a dynamic safety model based on vehicle speed). For example, if the following vehicle is approaching quickly and the distance is insufficient, the autonomous driving system will determine it as high risk and prohibit lane changes. However, if the leading vehicle is significantly slower than the current speed and the distance is sufficient, the autonomous driving system will allow lane changes.

[0074] ② Time to Collision (TTC) prediction: By predicting the collision time between the vehicle in the target lane and the current vehicle, if the TTC is less than a threshold (e.g., 3 seconds), it is judged as dangerous and lane changing is prohibited.

[0075] (3) Adaptability of vehicle dynamics state.

[0076] ① Vehicle speed and stability. The lateral acceleration and yaw rate during lane changes are predicted based on the vehicle's current speed and a dynamics model (e.g., a two-degree-of-freedom vehicle dynamics model) to ensure they do not exceed the stability threshold (e.g., lateral acceleration < 0.3g).

[0077] ② Steering capability verification: Combine the steering system status (such as electric power steering torque) to determine whether there is sufficient steering capability to complete the lane change.

[0078] (4) Compliance with environmental and traffic regulations.

[0079] ① Traffic signals and signs: The visual recognition module detects traffic signals (such as no lane change signs) or road markings (such as solid lines) to ensure that lane changes comply with traffic regulations.

[0080] ② Weather and road conditions: If severe weather (such as heavy rain or fog) or a slippery road (low friction coefficient) is detected, the autonomous driving system will increase the safety threshold or recommend delaying lane changes.

[0081] (5) System real-time decision logic.

[0082] ① Multimodal data fusion: Integrating the target lane spatial data, surrounding vehicle dynamics, vehicle status, and environmental information, a convolutional neural network model is used to generate a real-time “lane change probability.”

[0083] ② Threshold determination: If the lane change probability exceeds a preset threshold (e.g., 90%), it is determined that "the target lane conditions are met," triggering assisted lane change; otherwise, the driver maintains the current lane or waits for an opportunity.

[0084] (6) Adaptability of the method.

[0085] ① Driver-in-the-Loop (DIL) test platform verification. Simulating different scenarios (such as overtaking on highways and changing lanes in congested roads) in virtual simulations, and optimizing the "target lane condition met" decision logic through large-scale training data.

[0086] ② Two-degree-of-freedom vehicle dynamics model linkage: Real-time prediction of lane change trajectory and vehicle dynamic response. If the predicted result conflicts with the target lane space, the lane change operation is immediately terminated.

[0087] In another exemplary embodiment of the present application, step 106 specifically includes:

[0088] When the human-machine intervention penalty factor is 0, the autonomous driving system takes priority in controlling the target vehicle to perform steering operations.

[0089] When the human-machine intervention penalty factor is not 0, the human-machine intervention penalty factor is updated according to the target vehicle state obtained in real time by the two-degree-of-freedom vehicle dynamics model, and the operation priority of the driver and the automatic driving system is switched through a smooth transition algorithm to control the target vehicle to perform steering operations.

[0090] As an optional implementation, the two-degree-of-freedom vehicle dynamics model includes: a lateral motion equation and a steering angle dynamics equation of the target vehicle.

[0091] The lateral motion equation is:

[0092]

[0093] The steering angle dynamic equation is:

[0094]

[0095] Where m is the mass of the target vehicle, v y is the lateral velocity of the target vehicle, v x is the longitudinal velocity of the target vehicle, is the yaw rate of the target vehicle, F yf and F yr are the front wheel lateral force and rear wheel lateral force of the target vehicle, respectively, l f and l r are the distance from the target vehicle’s center of mass to the front axle and the distance from the center of mass to the rear axle, I z is the moment of inertia of the target vehicle around its center of mass.

[0096] As an optional implementation, the driver's operation priority during the driving of the target vehicle is:

[0097]

[0098] Among them, W d is the driver's operation priority at the qth moment, F q is the human-machine intervention penalty factor at the qth moment.

[0099] The operational priorities of the autonomous driving system are:

[0100] W a =1-W d .

[0101] Among them, W a is the operational priority of the autonomous driving system at moment q.

[0102] In another exemplary embodiment of the present application, step 106 further includes:

[0103] If the predicted yaw rate or lateral acceleration of the target vehicle exceeds a preset yaw rate threshold or a preset lateral acceleration threshold, the automated driving system takes optimal intervention measures to adjust the target vehicle's trajectory. These optimal intervention measures may include adjusting the target vehicle's steering angle, acceleration, or deceleration. Specifically, in situations where the risk of control conflict is high, the rapid intervention algorithm adjusts the vehicle's trajectory to avoid safety hazards caused by operational conflicts. Specifically, the rapid intervention algorithm may include the following steps: ① Real-time monitoring: Continuously monitoring the vehicle's surroundings and the driver's operational behavior through sensors and cameras. ② Conflict prediction: Utilizing a driver lane change intention prediction model to assess the conflict risk between the current driver's operation and the automated driving system's intention. ③ Intervention decision: When a high-risk conflict is predicted, the automated driving system calculates and determines the optimal intervention measures, such as adjusting the steering angle, vehicle speed, or acceleration or deceleration. ④ Execution: Implementing these decisions and adjusting the vehicle's trajectory or speed to avoid potential conflicts. It is important to note that the design of the rapid intervention algorithm should consider the driver's intention and comfort to avoid excessive intervention that may cause driver discomfort or distrust in the automated driving system. Therefore, rapid intervention algorithms are often combined with a driver lane change intention prediction model to achieve smooth coordinated control.

[0104] In summary, the rapid intervention algorithm proactively adjusts vehicle behavior through real-time perception and prediction to avoid safety hazards caused by operational conflicts, rather than simply reducing the driver's control weight.

[0105] In another exemplary embodiment of the present application, Figure 3 As shown, a human-machine co-driving control method considering the driver's intention is provided, which specifically includes:

[0106] 1. Build a perception system based on optical cameras and a driver-in-the-loop experimental platform. A high-precision optical camera collects multimodal data, including the driver's head posture, gaze direction, and eye movement trajectory, in real time. Combining a virtual driving simulation module with a real-time data acquisition module, this system provides a multi-scenario simulation environment (e.g., urban roads, highways, and congested environments), providing a foundation for the development and validation of a driver lane change intention prediction model.

[0107] The driver-in-the-loop experimental platform's perception system consists of the following devices:

[0108] (1) Laser calibrator: It can be used to perform precise position calibration on relevant experimental equipment, ensuring the accuracy of each device in terms of spatial position, etc., and providing a basic guarantee for subsequent accurate data collection. For example, it can determine the accurate reference position for optical cameras and other equipment, so as to more accurately obtain the accurate situation of the driver's related action information in space.

[0109] (2) Optical camera: This is used to collect important information such as the driver's head movement trajectory, gaze direction, and eye movement trajectory. By collecting this visual data, we can gain a deeper understanding of the driver's visual focus and head movements during driving, thereby providing key visual behavioral evidence for analyzing their driving intentions, especially lane-changing intentions. The optical camera module can be equipped with optional high-frame-rate, low-latency camera equipment to ensure the accuracy of the driver's head and eye features; and through multimodal fusion technology, it can further enhance the ability to recognize driver intentions in complex scenarios.

[0110] (3) Freedom Driving Platform: It can simulate the changes in vehicle posture under various actual driving scenarios, such as acceleration, deceleration, turning, tilting and other different driving states of the vehicle, allowing the driver to operate under a driving experience close to the real one, so as to more comprehensively and accurately record the dynamic behavior data under various simulated road conditions.

[0111] In the entire perception system, optical cameras collect driver vision-related data, and a driver-in-the-loop (DIL) experimental platform records the driver's dynamic behavior data in real time. This multimodal data is then fused and processed through a convolutional neural network to ultimately establish a driver lane change intention prediction model, which is used to identify in real time whether the driver intends to change lanes. The multimodal data includes the following two categories: ① Driver vision-related data. Collected by the optical camera, specifically including: head posture (such as rotation angle and yaw direction); gaze direction (lateral deviation of the line of sight from the lane); and eye movement trajectory (sequence of gaze points and gaze duration). ② Driver dynamic behavior data. Recorded in real time by the DIL experimental platform, specifically including: steering wheel angle; accelerator / brake pedal application force; vehicle lateral acceleration and longitudinal speed; and physical feedback from lane change operations (such as steering torque).

[0112] In summary, multimodal data is a fusion of visual data (head, eyes) and dynamic behavior data (vehicle status, operation signals), which is jointly analyzed through convolutional neural networks to improve the comprehensiveness of intent recognition.

[0113] 2. Develop a driver lane change intention prediction model based on a convolutional neural network. By deeply extracting and integrating head and eye movement trajectory features, a prediction mechanism for lane change intention recognition (including left or right lane changes) is established. The driver lane change intention prediction model significantly reduces reliance on traditional physiological data and complex sensors, while improving recognition accuracy and real-time performance in complex scenarios. When the driver intends to change lanes, the head and eye movement trajectories need to exhibit the following characteristics:

[0114] (1) Head movement characteristics:

[0115] ① Change in gaze direction: The driver's head will turn toward the target lane, and the angle between the gaze and the vehicle's direction of travel will increase. (Analysis of training data from the driver-in-the-loop experimental platform shows that the head rotation angle for lane change intention is typically concentrated in the range of 15° to 45°, and the specific value is dynamically adjusted based on the driving scenario and individual differences.)

[0116] ② Head rotation angle: Before changing lanes, the head rotation angle usually increases, especially when the viewpoint shifts significantly. The correlation between head rotation and gaze angle is strong.

[0117] (2) Eye movement characteristics:

[0118] ① Gaze point offset: The eye's gaze point will shift from the front of the current lane to the front of the target lane. The offset of the gaze point is related to the intensity of the lane change intention.

[0119] ② Gaze time: When the intention to change lanes is clear, the time the eyes spend looking at the target lane area will increase significantly. Gaze time refers to the cumulative time the driver spends focusing on a specific target area (such as the target lane), and has the following two meanings:

[0120] Single gaze time: the duration of a driver's single continuous gaze on the target area.

[0121] Multiple gaze time superposition: the cumulative time of the driver's multiple short gazes on the same target area in different time periods.

[0122] In order to accurately identify the driver's lane-changing intention, the following method is needed to eliminate the influence of other random factors during the training process of the driver's lane-changing intention prediction model to ensure the accuracy of the lane-changing intention.

[0123] Method 1: Use a fixed time window and heading angle threshold to mark lane changes to improve the accuracy and consistency of the marking. The marking of lane changes must meet the following two conditions:

[0124] Condition 1: Continuous monitoring within a fixed time window. Driving behavior data is collected within a fixed time window (e.g., 3 seconds before a lane change to 1 second after the lane change). If coordinated changes in the driver's head posture, eye movement, and vehicle dynamic parameters are detected within this window, the annotation process begins.

[0125] Condition 2: The heading angle changes beyond a preset threshold. The heading angle (HeadingAngle) refers to the angle between the vehicle's direction of travel and a reference direction (such as the centerline of the current lane). When the heading angle change (ΔΨ) exceeds a preset heading angle threshold (e.g., 10° to 20°, the specific value varies depending on the scenario), and the direction of change is consistent with the target lane, it is considered a lane change.

[0126] If the heading angle change continues to increase and exceeds a threshold within a fixed time window, and is accompanied by the following features, it is marked as a lane change behavior: the driver's head turns toward the target lane (such as the left or right); the eye movement trajectory shows a significant increase in the gaze time on the target lane; and the vehicle's lateral displacement deviates toward the target lane.

[0127] Method 2: Feature selection and normalization: Extract key features from multimodal data, such as head rotation angle and eye movement trajectory, and normalize these features to reduce the impact of individual differences and environmental interference.

[0128] Method 3: Model Validation and Cross-Validation: Cross-validation is used to evaluate the driver lane change intention prediction model to ensure its generalization ability across different datasets and reduce the risk of overfitting.

[0129] Method 4: Multimodal data fusion: Combining head and eye motion features and performing deep fusion through convolutional neural networks to improve the accuracy and robustness of lane change intention recognition.

[0130] Among them, in implementing this implementation method, a convolutional neural network is combined with an attention mechanism to extract the driver's gaze features on specific targets during the lane change process to enhance the robustness of intention recognition.

[0131] Specific targets include: a. Target lane: the lane the driver plans to change to; b. Surrounding vehicles: especially the preceding or following vehicle in the target lane; c. Traffic signs and signals: such as speed limit signs and traffic lights, which may affect the lane change decision; d. Pedestrians or obstacles: road obstacles or pedestrians that may affect lane change safety;

[0132] Gaze signatures distinguish specific targets that the driver is looking at by:

[0133] a. Target detection and positioning: The vehicle's onboard camera or lidar detects the position of the target lane, surrounding vehicles, traffic signs, and obstacles in real time and classifies them as left or right targets.

[0134] b. Gaze point matching: Map the gaze point coordinates of the driver's eye movement trajectory to the 3D scene coordinate system, match them with the position of the detection target, and determine the gaze target category (left / right lane, left / right vehicle, etc.).

[0135] c. Feature extraction and association: Gaze duration: Counts the percentage of time the driver spends looking at the left / right target lane; Gaze frequency: Counts the number of glances at the left / right vehicle or obstacle; Gaze angle: Quantifies the intensity of attention to the left / right target through the horizontal angle between the line of sight and the vehicle centerline; Gaze shift: Analyzes the path and frequency of gaze shifts from the current lane to the left / right target lane.

[0136] 3. Data Collection and Model Training and Validation. Dynamic driving data is collected on a driver-in-the-loop (DIL) test platform. Virtual scenarios are used to simulate various lane-changing scenarios (e.g., dense traffic conditions, inclement weather conditions, etc.) to conduct large-scale training and validation of the driver lane-change intention prediction model. This method utilizes real-world traffic scenarios and multimodal feature fusion to improve the robustness and cross-individual adaptability of the DIL prediction model in dynamic traffic environments.

[0137] 4. Design of human-machine intervention penalty factor. To address operational conflicts that may arise from inconsistencies between the driver's intentions and the autonomous driving system's decisions, a human-machine intervention penalty factor is designed to mitigate human-machine conflicts by adjusting the control weight distribution between the driver and the autonomous driving system. The calculation formula for the human-machine intervention penalty factor is:

[0138] F q =k(1-C q ).

[0139] Among them, F q is the human-machine intervention penalty factor at the qth moment, C q is the consistency coefficient between the lane changing intention prediction result at the qth moment and the lane changing intention of the automatic driving system, and k is the penalty factor adjustment coefficient.

[0140] In this implementation, the human-machine intervention penalty factor is dynamically adjusted based on the following parameters:

[0141] C q The calculation of is based on the matching degree between the driver's gaze direction and the vehicle's driving target; the adjustment of k is based on the changes in vehicle speed and road complexity; F P The values are continuously updated over time, ensuring smooth collaboration between the driver and the automated driving system.

[0142] 5. Construction of a shared human-machine steering control system. Based on the human-machine intervention penalty factor and vehicle dynamic parameters, a shared human-machine steering control system is constructed. This shared human-machine steering control system utilizes a two-degree-of-freedom vehicle dynamics model to monitor the vehicle's lateral motion and steering behavior in real time, and dynamically allocates control weights based on the consistency of the driver's and the autonomous driving system's intentions:

[0143] When the driver's intentions conflict with the autonomous driving system's intentions Figure 1 When the vehicle is in a state of emergency, the automatic driving system takes priority in steering operations.

[0144] For example, during driving, the human-machine steering shared control system accurately captures the driver's lane-changing intention through multi-source data collection and analysis. Once it is determined that the driver's lane-changing intention is consistent with the preset plan of the autonomous driving system, the autonomous driving system will be immediately triggered to prioritize the steering operation. When the autonomous driving system is in action, it will monitor the vehicle's yaw rate, lateral acceleration and other key parameters in real time at a very high frequency, and use intelligent optimization algorithms to dynamically adjust these parameters. In this way, the smoothness of the lane change process is effectively ensured, and the possibility of unnecessary shaking and deviation from the preset trajectory of the vehicle is greatly reduced. While improving lane-changing efficiency, the continuity and smoothness of the entire driving process are fully guaranteed, ultimately achieving the effect of the vehicle being able to switch to the target lane quickly and safely.

[0145] When intentions are inconsistent, the control weights are smoothly adjusted based on the penalty factor to avoid violent conflicts.

[0146] For example, when the shared human-machine steering control system detects a discrepancy between the driver's intended steering and the automated driving system's planned steering, it immediately initiates a dynamic adjustment mechanism based on a penalty factor. The shared human-machine steering control system first rapidly collects and analyzes the driver's steering input, incorporating the vehicle's current speed information. Based on a preset penalty factor algorithm, which comprehensively considers the impact of vehicle speed on steering stability and the degree of deviation between the driver's operation and the automated driving system's planned steering, it calculates the steering control weights to be allocated to the driver and the automated driving system, respectively. For example, if the vehicle is traveling at high speed, the shared human-machine steering control system will appropriately increase the automated driving system's control weight to ensure steering safety and stability. If the driver's deviation is minor and the vehicle speed is moderate, the shared human-machine steering control system will grant the driver a certain percentage of control authority, allowing for appropriate steering intervention. Throughout this dynamic adjustment process, the shared human-machine steering control system utilizes a smooth transition algorithm to ensure a smooth and natural transition between control authority and the automated driving system, avoiding sudden changes in control authority that could cause drastic steering fluctuations or instability. This effectively reduces potential conflicts and enables harmonious coexistence and collaboration between the driver and the automated driving system in steering control.

[0147] As an optional implementation, the two-degree-of-freedom vehicle dynamics model includes: a lateral motion equation and a steering angle dynamics equation of the target vehicle.

[0148] The lateral motion equation is:

[0149]

[0150] The steering angle dynamic equation is:

[0151]

[0152] Where m is the mass of the target vehicle, v y is the lateral velocity of the target vehicle, v x is the longitudinal velocity of the target vehicle, is the yaw rate of the target vehicle, F yf and F yr are the front wheel lateral force and rear wheel lateral force of the target vehicle, respectively, l f and l r are the distance from the target vehicle’s center of mass to the front axle and the distance from the center of mass to the rear axle, I z is the moment of inertia of the target vehicle around its center of mass.

[0153] The lateral forces on the front and rear wheels can be calculated from the tire slip angle and tire stiffness:

[0154] F yf =C f α f .

[0155]

[0156] F yr =C r α r .

[0157]

[0158] Among them, C f and C r are the cornering stiffness of the front and rear wheels of the target vehicle respectively; α f and α r are the sideslip angles of the front and rear wheels of the target vehicle, respectively.

[0159] By collecting vehicle parameters such as speed and steering angle in real time, the two-degree-of-freedom vehicle dynamics model is input into the control module to predict the dynamic response of the target vehicle during lane changing and ensure the smoothness of the target vehicle's lateral movement and steering.

[0160] 6. Implementation of collaborative control during lane changes. During lane changes, a two-degree-of-freedom vehicle dynamics model is used to monitor and control the vehicle in real time, ensuring smooth lateral movement and steering angles. Furthermore, an attention mechanism is used to deeply extract and analyze driver behavior characteristics, further improving safety and comfort during lane changes.

[0161] During the lane change process, the following target vehicle states are predicted in real time using a two-degree-of-freedom vehicle dynamics model:

[0162] (1) Lateral Displacement: This predicts the lateral movement of the vehicle relative to the current lane during the lane change process to ensure that the trajectory is smooth and consistent with the target lane position.

[0163] (2) Yaw Rate: Predicts the vehicle's rotational angular velocity around the vertical axis, used to determine whether the steering is excessive or insufficient.

[0164] (3) Lateral Acceleration: Predicts changes in the vehicle's lateral acceleration to prevent vehicle instability caused by over-sharp steering.

[0165] (4) Trajectory Deviation: Predicts the deviation between the actual trajectory and the preset lane change path to ensure lane change accuracy.

[0166] The predicted target vehicle state is directly used in the following steps:

[0167] ① Dynamic control weight allocation. When the predicted lateral displacement is consistent with the driver's intention (such as changing lanes to the left), the automatic driving control weight is increased and the steering operation is performed first. If the predicted trajectory deviation exceeds the safety threshold, the penalty factor (F p ) Reduce the driver's control weight and force the autonomous driving system to correct the trajectory.

[0168] ② Rapid Intervention Algorithm. If the predicted yaw rate or lateral acceleration exceeds the stability threshold (e.g., yaw rate > 0.5 rad / s), the rapid intervention algorithm is triggered to adjust the steering angle or vehicle speed to avoid the risk of loss of control. For example, when changing lanes at high speed, if the predicted trajectory deviates too much, the autonomous driving system will proactively fine-tune the steering wheel angle to ensure the vehicle smoothly enters the target lane.

[0169] This application also provides an application scenario that utilizes the aforementioned human-machine co-driving control method that considers driver intention. Specifically, the human-machine co-driving control method that considers driver intention, provided in this embodiment, can be applied in highway driving scenarios. Highway driving scenarios include a lane change decision phase, a lane change execution phase, and a lane change completion phase. From the lane change decision phase, the vehicle enters the lane change execution phase, where it undergoes real-time monitoring and adjustment by the human-machine co-driving control method to achieve a smooth lane change operation, and then enters the lane change completion phase. The human-machine co-driving control method that considers driver intention, provided in this embodiment, is a key technology in both the lane change decision phase and the lane change execution phase. Specifically, in the lane change decision phase, the method acquires the driver's visual data in real time and uses a CNN network to predict the driver's lane change intention and determine whether there is an intention to change. In the lane change execution phase, the operation priorities of the driver and the autonomous driving system are dynamically adjusted based on the prediction results and the human-machine intervention penalty factor to achieve a safe and smooth lane change operation. This application achieves accurate judgment of whether the driver has lane change intention during highway driving and performs corresponding human-machine collaborative control accordingly, thereby improving driving safety and efficiency.

[0170] This application fully considers the potential interference of the driving environment on data collection and analysis, and significantly reduces the impact of environmental and individual differences through the following technical means, thereby improving the versatility and stability of the driver lane change intention prediction model.

[0171] (1) High-precision optical camera: Dynamic exposure compensation technology and adaptive light adjustment algorithms are used to ensure that the driver's head posture and eye movement can be clearly captured under complex lighting conditions such as strong light, backlight, nighttime lighting, and tunnels. Example: When driving at night, the camera uses enhanced infrared fill light to avoid image blur caused by low light.

[0172] (2) Multimodal data fusion dynamically corrects ambient noise by fusing vehicle dynamic parameters (such as speed and steering wheel angle) with visual data (head and eye features). For example, if a bumpy road surface causes a brief head shake, the vehicle's lateral acceleration data can be combined to determine whether it is random jitter rather than an intention to change lanes.

[0173] (3) Feature normalization and denoising: Normalize features such as head rotation angle and gaze offset to eliminate baseline offsets caused by environmental vibration or individual differences in habits. Example: Normalize the head rotation angle to the driver's baseline posture (such as the head position during normal driving) to reduce the impact of individual physiological differences.

[0174] (4) Attention Mechanism: The attention mechanism dynamically focuses on key features that are strongly related to the lane-changing intention (such as the target lane gaze time) and ignores secondary noise caused by environmental interference (such as short-term gaze deviation).

[0175] (4) Driver-in-the-loop experimental platform training: A variety of scenarios (such as bad weather, congested roads, and nighttime driving) are simulated in a virtual simulation environment. Large-scale data is used to train the driver lane change intention prediction model to adapt to behavioral patterns in different environments. For example, in rainy and snowy scenarios, the driver lane change intention prediction model learns to distinguish between slight head movements caused by wiper operation and actual lane change intentions.

[0176] (5) Linking the heading angle with the two-degree-of-freedom vehicle dynamics model: The two-degree-of-freedom vehicle dynamics model verifies the intention recognition results in real time. For example, if the head tilts to the left but the vehicle does not move to the left, it is determined to be environmental interference (such as the driver adjusting his sitting position) rather than an intention to change lanes.

[0177] (6) Dynamic adjustment of penalty factor: When environmental interference causes a temporary deviation in intention recognition, the human-machine intervention penalty factor (F P ) automatically reduces the driver's control weight, and the autonomous driving system temporarily takes over to maintain stability.

[0178] (7) Transfer learning technology: The driver lane change intention prediction model uses transfer learning pre-training, extracting common features from general driving behavior data. It then fine-tunes the model with a small amount of personalized data to quickly adapt to the habits of different drivers. For example, during pre-training, the driver lane change intention prediction model learns the general relationship between head rotation angle and lane change direction. When fine-tuning the model for a specific driver, it only needs to adapt to the individual differences in the rotation amplitude.

[0179] (8) Multimodal feature screening: Extract features from multimodal data that are strongly correlated with intent and less affected by individual differences. For example, gaze time percentage: The percentage of cumulative gaze time to total driving time, rather than the absolute time value, reduces the impact of individual differences in scanning frequency.

[0180] In summary, although the driving environment (such as light changes, road vibrations) and individual differences (such as head flexibility) may have a certain impact on the acquisition of raw data, this application has significantly improved the anti-interference ability of head posture and eye movement trajectory analysis through hardware optimization (high-precision camera), algorithm enhancement (attention mechanism, multimodal fusion), dynamic scene training and real-time verification (two-degree-of-freedom vehicle dynamics model linkage). Compared with traditional methods that rely on physiological data (such as heart rate and brain waves), this application achieves higher robustness and cross-scene adaptability through the direct capture and dynamic correction of visual behavioral characteristics, ultimately achieving the core goals of reducing experimental costs and improving versatility.

[0181] This application has the following advantages:

[0182] 1. By using optical cameras to collect behavioral characteristics such as the driver's head posture and eye movement in real time, it can directly reflect the driver's attention distribution and operating intentions in different driving scenarios, avoiding errors caused by indirect inference.

[0183] 2. Utilizing high-precision cameras, multimodal data collection is possible, reducing the need for expensive physiological monitoring equipment and, consequently, hardware costs. Combined with a CNN network, this effectively improves the accuracy, real-time performance, and robustness of driver intent recognition, significantly reducing reliance on complex physiological sensors.

[0184] 3. By designing a human-machine intervention penalty factor and dynamically adjusting the human-machine control weight distribution, the control conflict between the driver and the autonomous driving system during lane change operations is significantly reduced, thereby improving collaborative efficiency.

[0185] 4. Based on a two-degree-of-freedom vehicle dynamics model and a human-machine steering shared control system, it ensures the vehicle's stability and safety during lane changes and adapts to complex dynamic traffic environments.

[0186] 5. Combined with dynamic simulation experiments on the driver-in-the-loop test platform, the driver's lane change intention prediction model can be trained and verified in a variety of complex traffic environments, such as urban roads, highways, and congested driving scenarios, significantly improving the adaptability and stability of the driver's lane change intention prediction model in actual scenarios.

[0187] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used for visual data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a human-machine co-driving control method that takes into account the driver's intention is realized.

[0188] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0189] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0190] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0191] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0192] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0193] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0194] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0195] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A human-machine co-driving control method considering the driver's intention, characterized by: The human-machine co-driving control method considering the driver's intention includes: Acquire visual data of multiple drivers during historical driving and annotate lane change intentions; the visual data includes the driver's head posture, gaze direction, and eye movement trajectory; the lane change intention includes "intention to change lanes" and "no intention to change lanes"; the "intention to change lanes" includes "intention to change lanes to the left" and "intention to change lanes to the right"; The training dataset is composed of visual data from multiple drivers' historical driving processes and annotated lane change intentions. Using the training data set to train a CNN network to obtain a driver lane change intention prediction model; Acquire the driver's predicted visual data of the target vehicle in real time during driving, and input it into the driver's lane change intention prediction model to obtain the lane change intention prediction result; If the lane change intention prediction result indicates that there is a lane change intention, a human-machine intervention penalty factor is calculated based on the lane change intention prediction result; the human-machine intervention penalty factor is used to measure the degree of conflict between the driver and the automatic driving system control in the target vehicle; Based on the human-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, the operation priorities of the driver and the automatic driving system during the driving process of the target vehicle are dynamically adjusted to perform human-machine co-driving control of the target vehicle; the two-degree-of-freedom vehicle dynamics model is used to obtain the target vehicle status in real time and is also used to predict the target vehicle status based on the lane change intention prediction result; the target vehicle status includes lateral displacement, yaw angular velocity, lateral acceleration, and path trajectory.

2. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: The human-machine intervention penalty factor is calculated based on the lane change intention prediction results, including: When the lane change intention prediction result is consistent with the lane change intention of the autonomous driving system and the lane change probability exceeds the preset lane change threshold, the human-machine intervention penalty factor is set to 0; When the lane change intention prediction result is inconsistent with the lane change intention of the autonomous driving system, or the lane change probability is less than the preset lane change threshold, the human-machine intervention penalty factor is calculated according to the following formula: F q =k(1-C q ); Among them, F q is the human-machine intervention penalty factor at the qth moment, C q is the consistency coefficient between the lane changing intention prediction result at the qth moment and the lane changing intention of the automatic driving system, and k is the penalty factor adjustment coefficient.

3. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: The method dynamically adjusts the operation priorities of the driver and the automatic driving system during the driving of the target vehicle based on the human-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, and performs human-machine co-driving control of the target vehicle, specifically including: When the human-machine intervention penalty factor is 0, the autonomous driving system takes priority in controlling the target vehicle to perform steering operations; When the human-machine intervention penalty factor is not 0, the human-machine intervention penalty factor is updated according to the target vehicle state obtained in real time by the two-degree-of-freedom vehicle dynamics model, and the operation priority of the driver and the automatic driving system is switched through a smooth transition algorithm to control the target vehicle to perform steering operations.

4. The human-machine co-driving control method considering the driver's intention according to claim 3 is characterized in that: The two-degree-of-freedom vehicle dynamic model includes: the lateral motion equation and steering angle dynamic equation of the target vehicle; The lateral motion equation is: The steering angle dynamic equation is: Where m is the mass of the target vehicle, v y is the lateral velocity of the target vehicle, v x is the longitudinal velocity of the target vehicle, is the yaw rate of the target vehicle, F yf and F yr are the front wheel lateral force and rear wheel lateral force of the target vehicle, respectively, l f and l r are the distance from the target vehicle’s center of mass to the front axle and the distance from the center of mass to the rear axle, I z is the moment of inertia of the target vehicle around its center of mass.

5. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: The driver's operation priority during the driving of the target vehicle is: Among them, W d is the driver's operation priority at the qth moment, F q is the human-machine intervention penalty factor at the qth moment; The operational priorities of the autonomous driving system are: IN a =1-W d ; Among them, W a is the operational priority of the autonomous driving system at moment q.

6. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: Based on the human-machine intervention penalty factor and the two-degree-of-freedom vehicle dynamics model, the system dynamically adjusts the operation priorities of the driver and the autonomous driving system during the target vehicle's driving process, and performs human-machine co-driving control of the target vehicle. This also includes: If the predicted yaw rate in the target vehicle state exceeds a preset yaw rate threshold or the lateral acceleration exceeds a preset lateral acceleration threshold, the autonomous driving system takes optimal intervention measures to adjust the target vehicle trajectory; the optimal intervention measures include adjusting the steering angle, acceleration or deceleration of the target vehicle.

7. The human-machine co-driving control method considering the driver's intention according to claim 2 is characterized in that: The lane change probability is obtained based on the target lane condition; The target lane conditions include: whether the target lane has enough space to accommodate the target vehicle changing lanes; whether the target lane line is a dotted line; whether the relative speed of the front vehicle and the rear vehicle in the target lane is less than a preset maximum speed threshold, and whether the distance between the front vehicle and the rear vehicle in the target lane is greater than a preset minimum safety distance threshold; whether the predicted collision time between the target lane vehicle and the target vehicle is greater than a preset safety time threshold; whether the current speed of the target vehicle is less than a safety speed threshold; whether the traffic signal allows the lane change operation; whether the weather and road conditions are suitable for the lane change operation; and whether the lateral acceleration of the target vehicle during the predicted lane change is less than a preset lateral acceleration threshold and whether the yaw angular velocity is less than a preset yaw angular velocity threshold.

8. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: The CNN network includes: an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer connected in sequence; The input layer is used to receive visual data and obtain an original visual data matrix; The convolutional layer is used to extract the multi-dimensional features of the original visual data matrix using the following formula to obtain a feature map: Among them, a i，j is the value of the i-th row and j-th column in the feature map, x i+m,j+n is the visual data of the i+mth row and j+nth column in the original visual data matrix, ω m,n is the weight value of the convolution kernel in the mth row and nth column, ω b is the first bias term, f is the activation function; The pooling layer is used to compress the feature map output by the convolutional layer to obtain a compressed feature map; The fully connected layer is used to perform weighted combination of different features in the compressed feature map using the following formula: y i′ =f(∑ω i′ x i′ +b); Among them, y i′ is the fully connected layer output of the i′th feature, ω i′ is the weight of the i′th feature, x i′ is the value of the i′th feature in the compressed feature map, and b is the second bias term; The output layer is used to generate the probabilities of three types of lane change intentions through a Softmax function based on the output of the fully connected layer.

9. The human-machine co-driving control method considering the driver's intention according to claim 8 is characterized in that: The output layer is used to generate the probabilities of three types of lane change intentions through the Softmax function, specifically including: Among them, P k′ is the probability of the k′th lane change intention, k′={1,2,3} corresponds to no lane change intention, left lane change intention, and right lane change intention, respectively, and K is the total number of lane change intention categories. is the exponential operation of the original output probability of the k′th lane-changing intention, is the exponential sum of all lane-changing intention probabilities; If P2>preset probability threshold, it is determined to be an intention to change lanes to the left; If P3>preset probability threshold, it is determined to be an intention to change lanes to the right; If both of the above conditions are not met, it is determined that there is no lane change intention.

10. The human-machine co-driving control method considering the driver's intention according to claim 1 is characterized in that: The lane-changing intention annotation specifically includes: using a fixed time window and heading angle threshold to annotate the lane-changing behavior; If the change in heading angle exceeds a threshold within a fixed time window and meets preset lane-changing conditions, it is marked as a lane-changing behavior. The preset lane-changing conditions include the driver turning his head to the left or right of the target lane; the eye movement trajectory showing the target lane is fixed for more than a preset time threshold; and the lateral displacement of the target vehicle deviates toward the target lane.

Images