An automatic-to-manual driving control transition method based on driver takeover capability

Abstract

The application relates to an automatic-to-manual driving control transition method based on driver takeover ability; firstly, driver eye movement data and driver manipulation data are collected through experiments, and a takeover ability factor is designed according to the takeover performance of the driver; secondly, a takeover ability random forest model based on driver eye movement behavior is trained with eye movement indexes as inputs and corresponding takeover ability factors as outputs; finally, an automatic-to-manual driving control transition model is constructed; the method can reduce the phenomenon that the driver is unstable in steering the steering wheel when the driver's takeover ability is poor in the traditional control right switching process; by considering the takeover ability of the driver, the takeover process can be more stable, and the takeover comfort and safety are improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology and relates to a method for transitioning from automatic to manual driving control based on driver takeover capability. In particular, it considers a method that calculates the takeover capability in real time based on the driver's eye movement data during autonomous driving and adjusts the torque applied to the steering wheel by the system during the switch from autonomous driving to manual driving based on the takeover capability, so as to achieve the safety and stability of the autonomous driving takeover process. Background Technology

[0002] In recent years, autonomous driving technology has developed rapidly, promising to reduce driver workload, free drivers from complex driving tasks, and alleviate traffic congestion, carbon emissions, and road fatalities. However, due to technological limitations, fully autonomous driving is not possible in the short term. When the autonomous driving system exceeds its operating range, the driver needs to take over the vehicle. However, sudden changes in torque applied to the steering wheel can make takeover operations unstable, affecting safety. This is especially true when the driver's takeover ability is poor, hindering a smooth transition of control. Therefore, this invention designs an automatic-to-manual driving control transition method based on the driver's takeover ability to improve the safety and stability of takeover. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to overcome the poor safety and stability of the transition from automatic to manual driving control in the prior art, and to provide a method for the transition from automatic to manual driving control based on the driver's takeover ability.

[0004] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution, which is described below in conjunction with the accompanying drawings:

[0005] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0006] First, the experiment collected driver eye-tracking and driver control data, and designed a takeover capability factor based on the driver's takeover performance. Second, using eye-tracking indicators as input and the corresponding takeover capability factor as output, a random forest model of takeover capability based on driver eye-tracking behavior was trained. Finally, a transition model from automatic to manual driving control was constructed.

[0007] The present invention relates to two models: a random forest model for takeover ability based on driver eye movement behavior and an automatic-to-manual driving control transition model.

[0008] The random forest model for takeover ability based on driver eye movement behavior is a prior art. Multiple decision tree models are trained, and the mode of the results of all decision tree models is taken as the result of the random forest model. The decision tree model needs to continuously traverse all possible split points of the feature subset of this tree at each node to find the split point of the feature with the smallest Gini index coefficient. When traversing the split point of each feature, the feature is divided into two parts, namely D1 (such as satisfying the feature value A < a) and D2 (not satisfying the feature value A < a). Then the Gini index of the set D under the condition of the feature value A < a is (indicating the uncertainty of the set D after being split by a):

[0009]

[0010] A represents a selected feature value (here, one of the eye movement features);

[0011] a represents a split point of feature A (the split point of each feature needs to be traversed).

[0012] The automatic-to-manual driving control transition model is a prior art. The automatic driving system gradually withdraws its control of the vehicle by reducing the torque on the steering wheel. The torque exerted by the automatic driving system on the steering wheel is:

[0013]

[0014] where K P represents the control weight coefficient of the automatic driving system, t represents the time calculated from the start of the control right switch, θ represents the actual steering wheel angle, represents the steering wheel angular velocity; θ d represents the target steering wheel angle, and K d is a constant. Design K p , and adjust the reduction speed of K p according to the takeover ability. The formula is as follows:

[0015]

[0016] "It" represents K p .

[0017] C represents the takeover ability factor. For high takeover ability, C = 3; for medium takeover ability, C = 2; for low takeover ability, C = 1;

[0018] The present invention relates to an automatic-to-manual driving control transition method based on driver takeover ability, including the following steps:

[0019] Step 1: Design the experimental plan, determine the driver type, select the working conditions, design the experimental process, and conduct the experiment, collecting driver eye-tracking data and control data after the driver takes over in real time.

[0020] Step two involves analyzing and processing the collected driver eye movement data to obtain eye movement indicators that affect the driver's takeover performance.

[0021] Step 3: Analyze and process the collected operational data after the driver takes over, evaluate the driver's takeover ability, and design a takeover ability factor.

[0022] Step 4: Use the random forest algorithm to classify the driver's eye movement indicators and takeover ability factors before the driver takes over, and obtain a random forest model of takeover ability based on the driver's eye movement behavior.

[0023] Step 5: Design an automatic-to-manual driving control transition model to gradually reduce the torque applied to the steering wheel and disengage from vehicle control.

[0024] Step 6: Collect the driver's eye movement data in real time during autonomous driving (within 9 seconds), input it into the random forest model of takeover capability based on the driver's eye movement behavior to obtain the takeover capability factor; then input the takeover capability factor into the automatic to manual driving control transition model to complete the transition of control from autonomous driving to manual driving.

[0025] In step one, the scenario is set as a curve where the driver takes over control of the vehicle. When the driver touches the steering wheel, the autonomous driving system disengages from control of the vehicle, and the driver begins manual driving. If the driver does not take over for more than 10 seconds, the system adopts the minimum risk strategy (MRM) and applies emergency braking.

[0026] In step two, eye movement indicators that affect the driver's takeover performance include: the proportion of time spent looking straight ahead, the number of saccades, the number of blinks, the percentage of eye closure per unit time, and the duration of blinks.

[0027] In step two, the collected driver eye movement data is analyzed and processed to obtain eye movement indicators within 9 seconds before takeover that affect the driver's takeover performance.

[0028] In step three, the evaluation of the driver's takeover ability is divided into three levels: high, medium, and low.

[0029] Low insertion capability: When the insertion time exceeds 5 seconds and the maximum lateral deviation of the centerline exceeds (w) r -w v At ) / 2, the driver has low takeover capability, where w r For road width, w v For vehicle width;

[0030] High takeover ability: When the takeover time is less than 5 s and the maximum lateral deviation of the center line is less than (w r -w v ) / 2, the driver has high takeover ability;

[0031] Medium takeover ability: When the takeover time is more than 5 s and the maximum lateral deviation of the center line is less than (w r -w v ) / 2, or when the takeover time is less than 5 s and the maximum lateral deviation of the center line is more than (w r -w v ) / 2, the driver has medium takeover ability.

[0032] Furthermore, the high takeover ability corresponds to the takeover ability factor C = 3, the medium takeover ability corresponds to the takeover ability factor C = 2, and the low takeover ability corresponds to the takeover ability factor C = 1.

[0033] In step four, the random forest algorithm is an ensemble algorithm. By composing multiple weak classifiers, the mode or mean is finally taken as the result of the strong classifier, that is, the driver's takeover ability factor.

[0034] Finally, the mode or mean is taken as the result of the strong classifier, which is the takeover ability factor 1, 2, or 3.

[0035] The strong classifier is a classification model for takeover ability based on the driver's eye movement. The result of the strong classifier is the result of the classification model.

[0036] In step five, the autonomous driving system gradually exits the control of the vehicle by reducing the torque on the steering wheel. The torque exerted by the autonomous driving system on the steering wheel is:

[0037]

[0038] where K P represents the control weight coefficient of the autonomous driving system, t represents the time calculated from the start of the control right switch, θ represents the actual steering wheel angle, and θ d represents the target steering wheel angle, and K d is a constant.

[0039] Design K p , and adjust the decreasing speed of K p according to the takeover ability. The formula is as follows:

[0040]

[0041] "It" represents K p . Adjust the decreasing speed of K p according to the takeover ability.

[0042] Autonomous driving systems also control the vehicle through the steering wheel, so the system applies torque to the steering wheel. The transition from automatic to manual driving is that the torque applied by the driver to the steering wheel gradually increases, while the torque applied by the system to the steering wheel gradually decreases.

[0043] The goal is to gradually reduce the torque applied to the steering wheel, thus gradually disengaging control of the vehicle. The automatic-to-manual driving control transition model is based on this gradual reduction of the torque applied to the steering wheel.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] This invention proposes a method for the transition from automatic to manual driving control based on driver takeover capability. First, driver eye-tracking and manipulation data are experimentally collected, and a takeover capability factor is designed based on the driver's performance. Second, a random forest model of takeover capability based on driver eye-tracking behavior is trained using eye-tracking indicators as input and the corresponding takeover capability factor as output. Finally, an automatic-to-manual driving control transition model is constructed. This method can alleviate the instability of steering wheel operation caused by poor driver takeover capability during traditional control switching processes. By considering the driver's takeover capability, the takeover process can be made smoother, improving comfort and safety. Attached Figure Description

[0046] The invention will now be further described with reference to the accompanying drawings:

[0047] Figure 1 This is a framework diagram of the automatic-to-manual driving control transition method based on driver takeover capability described in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this invention. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The embodiments of this invention will be described in detail below with reference to the accompanying drawings.

[0049] In the description of this invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the scope of protection of this invention.

[0050] The present invention will now be described in detail with reference to the accompanying drawings:

[0051] A method for transitioning from automatic to manual driving control based on driver takeover capability includes the following steps:

[0052] Step 1: Design the experimental plan, determine the experimental sample (driver type), select the working conditions, design the experimental process, and conduct the experiment, collecting driver eye-tracking data and driver control data in real time.

[0053] Step 2: Analyze and process the collected driver eye movement data to obtain the eye movement index (TOR first 9s) that affects the driver's takeover performance;

[0054] Step 3: Analyze and process the collected operational data after the driver takes over, evaluate the driver's takeover ability (high, medium, low), and design a takeover ability factor.

[0055] Step 4: Use the random forest algorithm to classify the driver's eye movement indicators and takeover ability factors before the driver takes over, and obtain a classification model of the driver's takeover ability based on eye movements.

[0056] Step 5: Construct a transition model for automatic to manual driving control.

[0057] Step 6: Collect the driver's eye movement data in real time during autonomous driving (within 9 seconds), input it into the driver takeover capability classification model to obtain the takeover capability factor; then input the takeover capability factor into the automatic to manual driving control transition model to complete the transition of control from autonomous driving to manual driving.

[0058] In step one above, the scenario is set as a curve where the driver takes over control of the vehicle. The driver touches the steering wheel, and the autonomous driving system disengages from control of the vehicle, allowing the driver to begin manual driving. If no intervention occurs for more than 10 seconds, the system adopts the minimum risk strategy (MRM), initiating emergency braking.

[0059] In step two above, the eye movement indicators that affect the driver’s takeover performance include (TOR first 9s): the proportion of time spent looking straight ahead, the number of saccades, the number of blinks, the percentage of eye closure per unit time, and the duration of blinks.

[0060] In step three above, the driver's takeover ability is evaluated as high, medium, or low based on the takeover time and the maximum lateral deviation from the centerline. The takeover time threshold is 5 seconds, and the threshold for the maximum lateral deviation from the centerline is: (w r -w v ) / 2, w r For road width, w v For vehicle width. Low takeover capability: When the takeover time exceeds 5 seconds and the maximum lateral deviation of the centerline exceeds (w r -w v When the takeover time is 5 seconds, the driver has low takeover capability; medium takeover capability: when the takeover time is higher than 5 seconds, the maximum lateral deviation of the centerline is lower than (w / 2). r -w v ) / 2 or the take-off time is less than 5 seconds, and the maximum lateral deviation of the centerline is higher than (w r -w v When the takeover time is less than 5 seconds and the maximum lateral deviation of the centerline is less than (w / 2), the driver has medium takeover capability; high takeover capability: when the takeover time is less than 5 seconds and the maximum lateral deviation of the centerline is less than (w / 2). r -w v When the value is ) / 2, the driver has a high takeover capability; high takeover capability corresponds to a takeover capability factor C=3, medium takeover capability corresponds to a takeover capability factor C=2, and low takeover capability corresponds to a takeover capability factor C=1.

[0061] In step four above, random forest is an ensemble algorithm that combines multiple weak classifiers and finally obtains the final result by taking the mode or mean.

[0062] In step five above, the torque applied to the steering wheel by the autonomous driving system is:

[0063]

[0064] Among them, K P The control weight coefficient of the autonomous driving system is represented by θ, where t represents the time calculated from the start of the control handover, and θ represents the actual steering wheel angle. d K represents the target turning angle of the steering wheel. d It is a constant. K p The formula is as follows:

[0065]

[0066] The process of the present invention will be further described in detail below with reference to the accompanying drawings.

[0067] See appendix Figure 1 The present invention proposes a method for transitioning from automatic to manual driving control based on driver takeover capability, which includes the following described process:

[0068] Step 1: Experimental design and data acquisition.

[0069] Experimental equipment: Eye tracker for acquiring driver gaze points (image coordinate system) and driver perspective video (video captured by eye tracker); angle sensor for real-time acquisition of driver control behavior information.

[0070] Experimental Samples: Considering that this invention uses a head-mounted eye tracker, the drivers participating in the experiment must meet the following requirements:

[0071] ①Drive with normal vision or habitually without wearing glasses;

[0072] ② You have obtained a C1 or higher driver's license.

[0073] At the same time, the gender, age, and driving experience of the test subjects need to be considered, and the overall average level should be close to the average level of theoretical drivers.

[0074] Experimental working condition design:

[0075] The experimental environment was a curve. Before a takeover request was issued, the autonomous driving system traveled at a speed of 50 km / h and displayed various environmental elements to the participants, including pedestrians, non-motorized vehicles, surrounding vehicles, subtle obstacles, road signs, and stationary vehicles. If no takeover was initiated for more than 10 seconds, the system adopted the minimum risk strategy (MRM) and applied emergency braking.

[0076] During autonomous driving, drivers are likely to engage in non-driving tasks. Different non-driving tasks correspond to different eye-tracking states of the driver, resulting in varying takeover responses. The driver's tasks during autonomous driving are as follows:

[0077] ① Do not engage in non-driving related tasks;

[0078] ② Listen to music;

[0079] ③ Watch videos;

[0080] ④ Typing and chatting;

[0081] ⑤ Play games;

[0082] Step two: Based on the driver's takeover time and maximum lateral deviation from the centerline, the driver's takeover ability is evaluated as high, medium, or low. The takeover time threshold is 5 seconds, and the threshold for maximum lateral deviation from the centerline is: (w r -w v ) / 2, w r For road width, w v For vehicle width. Low takeover capability: When the takeover time exceeds 5 seconds and the maximum lateral deviation of the centerline exceeds (w r -w v) / 2, the driver has low takeover ability; Medium takeover ability: When the takeover time is higher than 5s and the maximum lateral deviation of the center line is lower than (w r -w v ) / 2 or the takeover time is lower than 5s and the maximum lateral deviation of the center line is higher than (w r -w v ) / 2, the driver has medium takeover ability; High takeover ability: When the takeover time is lower than 5s and the maximum lateral deviation of the center line is lower than (w r -w v ) / 2, the driver has high takeover ability; The high takeover ability corresponds to the takeover ability factor C = 3, the medium takeover ability corresponds to the takeover ability factor C = 2, and the low takeover ability corresponds to the takeover ability factor C = 1;

[0083] Step 3: Construct a random forest model for takeover ability based on driver eye movement;

[0084] Taking driver eye movement as the input and the takeover ability factor as the output, use the random forest algorithm to analyze and process (classify) the eye movement indicators (the proportion of time looking ahead, the number of saccades, the number of blinks, the percentage of eye closure per unit time, the blink duration) before the driver takes over (9s) and the takeover ability factor, and obtain a random forest model for takeover ability based on driver eye movement.

[0085] Random forest is an ensemble algorithm that combines multiple weak classifiers and obtains the result by taking the mean or the majority. Each weak classifier is randomly sampled from the dataset for training, and finally, each weak classifier is integrated together to obtain a strong classifier. The weak classifier of the random forest uses the CART decision tree, also known as the classification and regression tree. The dependent variable of the dataset in this invention is a discrete numerical value, so it is a classification tree. Feature selection based on the Gini index is adopted. When the Gini coefficient is the smallest, the purity is the highest, the uncertainty is the smallest, and the effect of segmentation using this feature is the best. The CART tree is a binary tree. If the probability that a sample belongs to one of the classes is P, then the Gini index is (representing the uncertainty of set D):

[0086] Gini(p) = 2p(1 - p)

[0087] When traversing the splitting points of each feature, the feature is divided into two parts, namely D1 (such as satisfying the feature value A < a) and D2 (not satisfying the feature value A < a). Then the Gini index of set D under the condition of feature value A < a is (representing the uncertainty of set D after being split by a):

[0088]

[0089] By iterating through all possible split points of the feature subset of this tree, the split point with the smallest Gini coefficient is found. The mode of the results of each weak classifier is then used to obtain the result of the strong classifier.

[0090] A strong classifier is a classification model based on the driver's eye-tracking ability to take over. The result of a strong classifier is the result of the classification model. The entire random forest is a strong classifier, which is composed of multiple weak classifiers (decision trees).

[0091] Step four: Construct a transition model for automatic to manual driving control. The torque applied to the steering wheel by the automatic driving system is:

[0092]

[0093] Among them, K P The control weight coefficient of the autonomous driving system is represented by θ, where t represents the time calculated from the start of the control handover, and θ represents the actual steering wheel angle. d K represents the target turning angle of the steering wheel. d It is a constant. K p The formula is as follows:

[0094]

[0095] Step 5: Collect the driver's eye movement data in real time during autonomous driving (within 9 seconds), input it into the random forest model of takeover capability based on the driver's eye movement behavior to obtain the takeover capability factor; then input the takeover capability factor into the automatic to manual driving control transition model to complete the transition of control from autonomous driving to manual driving.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be included within the scope of protection of the present invention. Furthermore, all content not described in detail in this specification is prior art known to those skilled in the art.

Claims

1. A method for transitioning from automatic to manual driving control based on driver takeover capability, characterized in that, Includes the following steps: Step 1: Real-time acquisition of driver eye movement data and driver control data after takeover; Step two involves analyzing and processing the collected driver eye movement data to obtain eye movement indicators that affect the driver's takeover performance. Step three involves analyzing and processing the collected operational data after the driver takes over, evaluating the driver's takeover ability, and designing a takeover ability factor. ; Step 4: Use the random forest algorithm to classify the driver's eye movement indicators and takeover ability factors before the driver takes over, and obtain a random forest model of takeover ability based on the driver's eye movement behavior. Step 5: Design an automatic-to-manual driving control transition model to gradually reduce the torque applied to the steering wheel and disengage from vehicle control. Step 6: Collect driver's eye movement data in real time during autonomous driving and input it into a random forest model of takeover capability based on driver's eye movement behavior to obtain the takeover capability factor. Then, the takeover capability factor is input into the automatic to manual driving control transition model to complete the transition of control from automatic driving to manual driving; In step three, the evaluation of the driver's takeover ability is divided into three levels: high, medium, and low. Low insertion capability: When the insertion time exceeds 5 seconds and the maximum lateral deviation of the centerline exceeds [a certain value], [the following conditions apply]. At that time, the driver had low takeover capability, among which For the road width, For vehicle width; High take-off capability: When the take-off time is less than 5 seconds and the maximum lateral deviation of the centerline is less than... At that time, the driver possesses a high level of takeover capability; Centerline control capability: When the connection time exceeds 5 seconds, the maximum lateral deviation of the centerline is less than... Or the takeover time is less than 5 seconds, and the maximum lateral deviation of the centerline is higher than At that time, the driver has the ability to take over the command. In step five, the autonomous driving system gradually disengages from vehicle control by reducing the torque applied to the steering wheel. The torque applied to the steering wheel by the autonomous driving system is: in, Represents the control weight coefficient of the autonomous driving system. This represents the time calculated from the start of the transfer of control. This represents the actual angle of steering wheel rotation. Represents the target angle of the steering wheel. It is a constant; design Adjust according to the takeover capacity The rate of decrease; the formula is as follows: 。 2. The automatic-to-manual driving control transition method based on driver takeover capability according to claim 1, characterized in that: In step one, the scenario is set as a curve where the driver takes over control of the vehicle. When the driver touches the steering wheel, the autonomous driving system disengages from control of the vehicle, and the driver begins manual driving. If the driver does not take over for more than 10 seconds, the system adopts the minimum risk strategy (MRM) and applies emergency braking.

3. The automatic-to-manual driving control transition method based on driver takeover capability according to claim 1, characterized in that: In step two, eye movement indicators that affect the driver's takeover performance include: the proportion of time spent looking straight ahead, the number of saccades, the number of blinks, the percentage of eye closure per unit time, and the duration of blinks.

4. The automatic-to-manual driving control transition method based on driver takeover capability according to claim 1, characterized in that: In step two, the collected driver eye movement data is analyzed and processed to obtain eye movement indicators within 9 seconds before takeover that affect the driver's takeover performance.

5. The automatic-to-manual driving control transition method based on driver takeover capability according to claim 1, characterized in that: High takeover capability corresponds to takeover capability factor =3, the takeover capability factor corresponding to the intermediate takeover capability. =2, low takeover capability corresponds to takeover capability factor. =1.

6. The automatic-to-manual driving control transition method based on driver takeover capability according to claim 1, characterized in that, In step four, the random forest algorithm is an ensemble algorithm that combines multiple weak classifiers and finally takes the mode or mean as the result of the strong classifier, which is the driver's takeover ability factor.

Images