An automatic driving exit judgment method considering driver fatigue state

Abstract

The application is suitable for the automatic driving technical field, and provides an automatic driving exit judgment method considering the fatigue state of a driver. The method can more comprehensively and accurately evaluate the real-time fatigue state of the driver by fusing the muscle fatigue and facial visual fatigue features of the driver. The steering wheel input torque threshold is dynamically adjusted based on the fatigue state of the driver. When the fatigue degree of the driver is high, the takeover can be triggered with smaller operation force, and the risk of takeover difficulty caused by fatigue is reduced. When the state of the driver is good, larger operation force is required to avoid false triggering. Therefore, the intelligentization and humanization of the automatic driving exit judgment are realized under the premise of ensuring safety, and the safety and reliability of the man-machine co-driving are improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to an autonomous driving exit judgment method that takes into account driver fatigue. Background Technology

[0002] With the continuous development of autonomous driving technology, the transfer of vehicle control has become an increasingly important research topic. Especially in high-level autonomous driving systems, accurately and promptly assessing the driver's ability to take over control when the system detects that the driver needs to do so becomes crucial for ensuring driving safety.

[0003] In fatigue detection, existing technologies mostly employ recognition methods based on driver facial features, such as eye movements and yawning frequency, or detection methods based on physiological signals such as heart rate and brain waves. However, facial recognition is easily affected by factors such as lighting and occlusion, while physiological signal detection often requires wearing equipment, affecting driving comfort. On the other hand, fatigue detection methods based on vehicle behavior characteristics, such as lane departure warnings, are reactive interventions and are difficult to provide early warnings of fatigue.

[0004] In recent years, some studies have begun to extract fatigue characteristics from driver behavior, such as judging muscle fatigue levels through signals like steering wheel torque fluctuations and steering frequency. These methods have the advantages of being non-invasive and real-time, but they still lack integration with visual fatigue characteristics, making it difficult to comprehensively reflect the driver's overall fatigue state.

[0005] Therefore, existing technologies have not yet formed a multi-dimensional fatigue state judgment mechanism that can comprehensively consider driver muscle fatigue and facial features during the transfer of control in autonomous driving, resulting in the system lacking sufficient accuracy and safety when determining whether to exit autonomous driving mode. Summary of the Invention

[0006] The purpose of this invention is to provide a method for determining the exit from autonomous driving mode that takes into account driver fatigue, thereby addressing the problems mentioned in the background section.

[0007] The present invention is implemented as follows: a method for determining the exit from autonomous driving mode that takes into account driver fatigue includes the following steps:

[0008] Step 1: Acquire and fuse fatigue state signals from two different sources in real time to generate a driver's total fatigue state level signal. ;

[0009] Step 2: Based on the real-time total fatigue state level signal Dynamically set the steering wheel input torque threshold for disengaging autonomous driving. ;

[0010] Step 3: Monitor the driver's steering wheel input torque in real time. The system then performs filtering and outputs the filtered steering wheel input torque. Compared with the dynamically set steering wheel input torque threshold in step 2 The system compares the data and considers time constraints to determine whether to exit autonomous driving mode.

[0011] A further technical solution, wherein step 1 includes the following specific steps:

[0012] By collecting data on steering wheel angle and torque during vehicle operation, characteristic quantities used to characterize driver muscle fatigue are calculated, including the number of steering operations over a period of time. and steering torque fluctuation index ;

[0013] Steering torque fluctuation index Based on the collected steering wheel torque data The calculation shows that the process consists of three steps:

[0014] First, calculate the variance of the steering wheel torque within a single time window. Let the number of sampling points in this window be... The average torque is , No. The torque value at each sampling point is Then the formula for calculating variance is:

[0015] ;

[0016] Subsequently, in order to establish the individual torque baseline value for each driver Continuous measurement is required. torque variance over a time window And calculate their average value using the following formula:

[0017] ;

[0018] Ultimately, the steering torque fluctuation index The torque variance and torque baseline value within a specific time window The comparison shows that its value is a relative percentage change:

[0019]

[0020] Number of turn operations With steering torque fluctuation index By fusing the signals using preset weighting coefficients, the driver's muscle fatigue level signal is calculated. :

[0021] ;

[0022] in, The weighting coefficient for the number of turning operations. , is the weighting coefficient of the steering torque fluctuation index, and ; The numerical range is in between, It indicates that one is full of energy. This indicates severe muscle fatigue;

[0023] The driver's facial image is acquired by an in-vehicle camera. Image recognition technology is used to locate eye features, and PERCLOS (percentage of eye closure time) and EAR (eye aspect ratio) are calculated to determine the degree of fatigue, outputting a signal indicating the driver's visual fatigue level. ;

[0024] The PERCLOS criterion uses the P80 rule (eyelids covering more than 80% of the pupil area are considered closed). The formula for calculating the PERCLOS value is as follows:

[0025] ;

[0026] in, For the calculated PERCLOS value, The total number of sampling points within the selected time window. For the first The state of each sampling point; if the sampling point is determined to be "eyes closed", then Otherwise, it is 0;

[0027] To determine whether a sampling point is in a "closed eye" state, the EAR (Earnings Expiration) is used as the criterion; when the EAR value of a sampling point is lower than a certain threshold, the sampling point is considered to be in a "closed eye" state. );

[0028] Visual fatigue level signal According to PERCLOS value The calculation is as follows:

[0029] ;

[0030] Its numerical range is also within between, Indicates being conscious. This indicates severe visual fatigue;

[0031] Muscle fatigue level signal Visual fatigue level signal The signals are fused to calculate the total fatigue state level signal. The calculation formula is as follows:

[0032] ;

[0033] in, and These are the weighting coefficients for muscle fatigue and visual fatigue, respectively. .

[0034] In a further technical solution, in step 1, the number of steering operations is counted based on steering wheel angle data; the criterion is: when the steering wheel angle exceeds 20° and lasts for at least 1 second, and then returns to center, it is counted as a valid steering operation.

[0035] In a further technical solution, step 2 includes the following specific steps:

[0036] Establish and The mapping relationship satisfies: The higher the value, the more severe the driver fatigue. The higher the steering wheel input torque threshold set at this value, the more fatigued the driver becomes. The lower the value, the greater the functional relationship, which can be expressed as:

[0037] ;

[0038] in, The preset baseline takeover torque threshold represents the minimum steering wheel input torque required for the autonomous driving system to determine that the driver has a clear intention to take over when the driver is conscious. The value ranges from greater than or equal to [value missing]. and less than or equal to ; As a moderating factor of fatigue effects, , The larger the value, the more identical. corresponding The smaller.

[0039] In a further technical solution, step 3 includes the following specific steps:

[0040] Using a first-order low-pass filter After smoothing, its discrete form is expressed as:

[0041] ;

[0042] in, Input the filtered steering wheel torque at the current moment. This is the original torque value collected at the current moment. These are the filter coefficients, and their values ​​range from [value range missing]. ;

[0043] The filtered steering wheel input torque is obtained. ,Will and The system compares data and considers time constraints to determine whether to exit autonomous driving mode.

[0044] When the filtered steering wheel input torque is detected Continuously exceeding the current steering wheel input torque threshold and duration Reaching or exceeding the preset time threshold When the system determines that the driver has a valid intention to take over and a certain level of operational ability, it generates an automatic driving exit command.

[0045] The vehicle control system responds to this instruction and performs a smooth transfer of control from the autonomous driving system to the driver.

[0046] This invention provides an autonomous driving exit judgment method that takes into account driver fatigue. By integrating driver muscle fatigue and facial visual fatigue characteristics, this method can more comprehensively and accurately assess the driver's real-time fatigue state. Based on this, it dynamically adjusts the steering wheel input torque threshold, so that when the driver is highly fatigued, the takeover can be triggered with less operating force, reducing the risk of takeover difficulties caused by fatigue. When the driver is in good condition, a greater operating force is required to avoid false triggering. Thus, under the premise of ensuring safety, the autonomous driving exit judgment is made intelligent and humanized, improving the safety and reliability of human-machine co-driving. Attached Figure Description

[0047] Figure 1 A flowchart of an autonomous driving exit judgment method considering driver fatigue state provided in an embodiment of the present invention;

[0048] Figure 2 Input torque threshold for steering wheel With the overall fatigue state level signal Relationship diagram. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0051] like Figure 1As shown, an embodiment of the present invention provides a method for determining the exit of automated driving considering driver fatigue, comprising the following steps:

[0052] Step 1: Acquire and fuse fatigue state signals from two different sources in real time to generate a comprehensive and quantified driver total fatigue state level signal. .

[0053] By collecting data on steering wheel angle and torque during vehicle operation, characteristic quantities used to characterize driver muscle fatigue are calculated, including the number of steering operations over a period of time. and steering torque fluctuation index .

[0054] The number of steering operations is counted based on steering wheel angle data. The criterion is: when the steering wheel angle exceeds 20° and is maintained for at least 1 second, and then returns to center, it is counted as a valid steering operation.

[0055] Steering torque fluctuation index Based on the collected steering wheel torque data The calculation shows that the process consists of three steps:

[0056] First, calculate the variance of the steering wheel torque within a single time window. Let the number of sampling points in this window be... The average torque is , No. The torque value at each sampling point is Then the formula for calculating variance is:

[0057] ;

[0058] Subsequently, in order to establish the individual torque baseline value for each driver Continuous measurement is required. torque variance over a time window And calculate their average value using the following formula:

[0059] ;

[0060] Ultimately, the steering torque fluctuation index The torque variance and torque baseline value within a specific time window The comparison shows that its value is a relative percentage change:

[0061] ;

[0062] Number of turn operations With steering torque fluctuation index By fusing the signals using preset weighting coefficients, the driver's muscle fatigue level signal is calculated. :

[0063] ;

[0064] in, The weighting coefficient for the number of turning operations. , is the weighting coefficient of the steering torque fluctuation index, and , set here ; The numerical range is in between, It indicates that one is full of energy. This indicates severe muscle fatigue.

[0065] The driver's facial image is acquired by an in-vehicle camera, and image recognition technology is used to locate eye features. PERCLOS and EAR are calculated to determine the degree of fatigue, and a driver visual fatigue level signal is output. .

[0066] The PERCLOS criterion adopts the P80 criterion, and the formula for calculating the PERCLOS value is as follows:

[0067] ;

[0068] in, For the calculated PERCLOS value, The total number of sampling points within the selected time window. For the first The state of each sampling point. If the sampling point is determined to be "eyes closed", then Otherwise, it is 0.

[0069] The Earnings Expiration (EAR) metric is used to determine whether a sampling point is in a "closed eyes" state. A lower EAR value indicates a higher degree of eye closure. When the EAR value of a sampling point is below a certain threshold (set to 0.225 here), that sampling point is considered to be in a "closed eyes" state. ).

[0070] Visual fatigue level signal According to PERCLOS value The calculation is as follows:

[0071] ;

[0072] Its numerical range is also within between, Indicates being conscious. This indicates severe visual fatigue.

[0073] Muscle fatigue level signal Visual fatigue level signal The signals are fused to calculate the total fatigue state level signal. The calculation formula is as follows:

[0074] ;

[0075] in, and These are the weighting coefficients for muscle fatigue and visual fatigue, respectively. , here is set , .

[0076] Step 2: Based on the real-time total fatigue state level signal obtained in Step 1 Dynamically set the steering wheel input torque threshold for disengaging autonomous driving (i.e., requiring the driver to take over the vehicle). .

[0077] Establish and The mapping relationship. This mapping relationship satisfies: The higher the value, the more severe the driver fatigue. The higher the steering wheel input torque threshold set at this value, the more fatigued the driver becomes. The lower the value, the better. Its functional relationship can be expressed as:

[0078] ;

[0079] in, The preset baseline takeover torque threshold represents the minimum steering wheel input torque required for the automated driving system to determine that the driver has a clear intention to take over when the driver is conscious. It typically ranges from greater than or equal to [a certain value]. and less than or equal to , here is taken as , Fatigue effect moderating factor ( ), The larger the value, the more identical. corresponding The smaller the value, the better; here it is taken as 0.6.

[0080] Step 3: Monitor the driver's steering wheel input torque in real time. The signal is filtered to eliminate high-frequency noise and transient interference, thereby improving signal reliability and accuracy.

[0081] Specifically, a first-order low-pass filter is used for... After smoothing, its discrete form is expressed as:

[0082] ;

[0083] in, Input the filtered steering wheel torque at the current moment. This is the original torque value collected at the current moment. These are the filter coefficients, and their values ​​range from [value range missing]. Take this place .

[0084] The filtered steering wheel input torque is obtained. ,Will Compared with the dynamically set in step 2 The system compares data and considers time constraints to determine whether to exit autonomous driving mode.

[0085] When the filtered steering wheel input torque is detected Continuously exceeding the current steering wheel input torque threshold and duration Reaching or exceeding the preset time threshold (Set to 0.3 seconds here) When it is determined that the driver has a valid intention to take over and a certain operational ability, the system generates an automatic driving exit command.

[0086] The vehicle control system responds to this instruction and performs a smooth transfer of control from the autonomous driving system to the driver.

[0087] Figure 2 This is a graph showing the relationship between the steering wheel input torque threshold and the total fatigue state level signal. The total fatigue state level signal... When the driver is in a state of high alertness, as the muscle fatigue level signal and visual fatigue level signal increase, the total fatigue level signal also increases. Under the influence of this method, the steering wheel input torque required for the driver to take over driving gradually decreases. The steering torque required for the driver to disengage from automatic driving while in a fatigued state is reduced, so that the driver does not need a lot of force to take over vehicle control. At the same time, it is not too small, avoiding the system from making misjudgments due to the driver holding the steering wheel in automatic driving mode, which could lead to the dangerous situation of automatic driving disengaging without the driver's intention to take over vehicle control.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining automatic driving exit considering driver fatigue, characterized in that, Includes the following steps: Step 1: Acquire and fuse fatigue state signals from two different sources in real time to generate a driver's total fatigue state level signal. ; Step 2: Based on the real-time total fatigue state level signal Dynamically set the steering wheel input torque threshold for disengaging autonomous driving. ; Step 3: Monitor the driver's steering wheel input torque in real time. The system then performs filtering and outputs the filtered steering wheel input torque. Compared with the dynamically set steering wheel input torque threshold in step 2 The system compares the data and considers time constraints to determine whether to exit autonomous driving mode.

2. The method for determining automatic driving exit considering driver fatigue as described in claim 1, characterized in that, Step 1 includes the following specific steps: By collecting data on steering wheel angle and torque during vehicle operation, characteristic quantities used to characterize driver muscle fatigue are calculated, including the number of steering operations over a period of time. and steering torque fluctuation index ; Number of turn operations With steering torque fluctuation index By fusing the signals using preset weighting coefficients, the driver's muscle fatigue level signal is calculated. : ； in, The weighting coefficient for the number of turning operations. , is the weighting coefficient of the steering torque fluctuation index, and ; The numerical range is in between, It indicates that one is full of energy. This indicates severe muscle fatigue; The driver's facial image is acquired by an in-vehicle camera, and image recognition technology is used to locate eye features. PERCLOS and EAR are calculated to determine the degree of fatigue, and a driver visual fatigue level signal is output. ; The PERCLOS criterion adopts the P80 criterion, and the formula for calculating the PERCLOS value is as follows: ； in, For the calculated PERCLOS value, The total number of sampling points within the selected time window. For the first The state of each sampling point; if it is determined to be "eyes closed", then Otherwise, it is 0; To determine whether a sampling point is in a "closed eye" state, the EAR is used as the criterion; when the EAR value of a sampling point is lower than a threshold, the sampling point is considered to be in a "closed eye" state. Visual fatigue level signal According to PERCLOS value The calculation is as follows: ； Its numerical range is also within between, Indicates being conscious. This indicates severe visual fatigue; Muscle fatigue level signal Visual fatigue level signal The signals are fused to calculate the total fatigue state level signal. The calculation formula is as follows: ； in, and These are the weighting coefficients for muscle fatigue and visual fatigue, respectively. .

3. The method for determining automatic driving exit considering driver fatigue as described in claim 2, characterized in that, In step 1, the number of steering operations is counted based on steering wheel angle data; the criterion is: when the steering wheel angle exceeds 20° and lasts for at least 1 second, and then returns to center, it is counted as a valid steering operation.

4. The method for determining automatic driving exit considering driver fatigue as described in claim 2, characterized in that, In step 1, the steering torque fluctuation index Based on the collected steering wheel torque data The calculation shows that the process consists of three steps: First, calculate the variance of the steering wheel torque within a single time window. Let the number of sampling points in this window be... The average torque is , No. The torque value at each sampling point is Then the formula for calculating variance is: ； Subsequently, in order to establish the individual torque baseline value for each driver Continuous measurement is required. torque variance over a time window And calculate their average value using the following formula: ； Ultimately, the steering torque fluctuation index The torque variance and torque baseline value within a specific time window The comparison shows that its value is a relative percentage change: 。 5. The method for determining automatic driving exit considering driver fatigue as described in claim 4, characterized in that, Step 2 includes the following specific steps: Establish and The mapping relationship satisfies: The higher the value, the more severe the driver fatigue. The higher the steering wheel input torque threshold set at this value, the more fatigued the driver becomes. The lower the value, the greater the functional relationship, which can be expressed as: ； in, The preset baseline takeover torque threshold represents the minimum steering wheel input torque required for the autonomous driving system to determine that the driver has a clear intention to take over when the driver is conscious. The value ranges from greater than or equal to [value missing]. and less than or equal to ; As a moderating factor of fatigue effects, , The larger the value, the more identical. corresponding The smaller.

6. The method for determining automatic driving exit considering driver fatigue as described in claim 5, characterized in that, Step 3 includes the following specific steps: Using a first-order low-pass filter After smoothing, its discrete form is expressed as: ； in, Input the filtered steering wheel torque at the current moment. This is the original torque value collected at the current moment. These are the filter coefficients, and their values ​​range from [value range missing]. ; The filtered steering wheel input torque is obtained. ,Will and The system compares data and considers time constraints to determine whether to exit autonomous driving mode. When the filtered steering wheel input torque is detected Continuously exceeding the current steering wheel input torque threshold and duration Reaching or exceeding the preset time threshold When the system determines that the driver has the valid intention and ability to take over, it generates an automatic driving exit command. The vehicle control system responds to this instruction and performs a smooth transfer of control from the autonomous driving system to the driver.

Images