Intelligent driving automobile hand-off recognition method and device, processor and storage medium

Abstract

The application belongs to the technical field of intelligent driving, and discloses an intelligent driving automobile hand-off recognition method and device, a processor and a storage medium: step 1, an actual steering wheel torque value is acquired through a steering wheel torque sensor, the actual steering wheel torque value is compared with a first preset value, and it is judged whether the steering wheel is twisted or not; step 2, a steering wheel touch sensing value is acquired through a steering wheel capacitance sensor, and it is judged whether a driver's hand is on the steering wheel based on the steering wheel touch sensing value; step 3, face recognition information is acquired through a driver monitoring camera, and line-of-sight information data is generated based on the face recognition information; and step 4, an advanced driver assistance system (ADAS) controller is controlled based on the judgment results of steps 1, 2 and 3. The application solves the problems of low safety and poor reliability of functions of the ADAS / AD system.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology, specifically relating to a method, device, processor, and storage medium for hands-free recognition in intelligent driving vehicles. Background Technology

[0002] With the development of intelligent driving vehicles, the level of driver assistance is getting higher and higher, gradually transitioning from the initial Advanced Driver Assistance Systems (ADAS) to Autonomous Driving Systems (AD). As an intermediate stage between ADAS and AD, Level 3 autonomous driving functions may require driver intervention in certain scenarios due to technological limitations. In order to ensure that the driver can receive the system's takeover request in a timely manner, the system's ability to accurately determine whether the driver is paying attention to the vehicle's driving status is particularly important.

[0003] Currently, there are several solutions for monitoring whether a driver is paying attention to the vehicle's driving status. One approach uses a torque sensor in the electric power steering (EPS) system to monitor the steering torque applied by the driver and determine if the driver's hands are on the steering wheel, thus indicating driver engagement. However, this solution requires the driver to apply a certain amount of torque to the steering wheel periodically, which can affect the performance of the autonomous driving's lateral functions, resulting in a poor user experience. Another approach uses a capacitive sensor on the steering wheel to monitor whether the driver's hands are on the wheel, thus determining driver engagement. This solution avoids the problem of the first approach requiring the driver to periodically apply torque to the steering wheel, but it prevents true hands-free driving, especially for Level 3-4 autonomous driving, leading to a poor user experience. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method, device, processor, and storage medium for hands-free vehicle recognition in intelligent driving vehicles. It aims to accurately determine whether the driver is paying attention to the vehicle's driving status through multi-sensor combined monitoring for ADAS / AD systems below Level 4, and to inform the ADAS / AD system whether it needs to remind the driver of the vehicle's driving status via sound, vibration, or graphics, thereby ensuring the driver's driving safety.

[0005] In a first aspect, the present invention provides a method for hands-free recognition in intelligent driving vehicles, the method comprising:

[0006] Step 1: Obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with a first preset value, and determine whether the steering wheel is twisting.

[0007] Step 2: Obtain the steering wheel touch sensing value through the steering wheel capacitive sensor, and determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing value;

[0008] Step 3: Obtain facial recognition information through the driver monitoring camera, and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed.

[0009] Step 4: Control the Advanced Driver Assistance System (ADAS) controller based on the judgment results of Step 1, Step 2, and Step 3.

[0010] Specifically, when the actual torque value of the steering wheel is greater than the first preset value, if the driver's hands and the driver's line of sight meet the first preset conditions, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The first preset conditions include the driver's line of sight being on the road surface, or the driver's line of sight being unconfirmed when the driver's hands are on the steering wheel.

[0011] Specifically, when the actual torque value of the steering wheel is greater than the first preset value, if the driver's hands and the driver's line of sight meet the second preset condition, then the duration T2 of meeting the second preset condition is obtained. If the duration T2 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable L3 and above functions. The second preset condition includes the driver's line of sight being unable to be confirmed when the driver's hands are on the steering wheel, or the driver's line of sight being unable to be confirmed or not on the steering wheel when the driver's hands are not on the steering wheel.

[0012] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the third preset condition, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The third preset condition includes the driver's line of sight being on the road surface when the driver's hands are on the steering wheel.

[0013] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fourth preset condition, then the duration T4 of meeting the fourth preset condition is obtained. If the duration T4 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable L3 and above functions. The fourth preset condition includes the driver's line of sight not being on the road or being unable to confirm when the driver's hands are on the steering wheel, or the driver's line of sight being on the road when the driver's hands are not on the steering wheel.

[0014] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fifth preset condition, then the duration T5 of meeting the fifth preset condition is obtained. If the duration T5 is less than the second preset time t5, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable functions below L3. The fifth preset condition includes the inability to confirm the driver's line of sight information or the driver's line of sight being off the road when the driver's hands are not on the steering wheel.

[0015] Specifically, when the duration T2 of satisfying the second preset condition or the duration T4 of satisfying the fourth preset condition is greater than or equal to the first preset time t2, or the duration T5 of satisfying the fifth preset condition is greater than or equal to the second preset time t5, the Advanced Driver Assistance System (ADAS) controller does not have permission to enable any function.

[0016] Secondly, the present invention also provides a hands-free recognition device for intelligent driving vehicles, the device comprising:

[0017] The steering wheel torque sensor module is used to obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with a first preset value, and determine whether the steering wheel is twisting.

[0018] The steering wheel capacitive sensor module is used to acquire steering wheel touch sensing values ​​through the steering wheel capacitive sensor, and to determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing values.

[0019] The driver monitoring camera module is used to acquire facial recognition information through the driver monitoring camera and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed.

[0020] The driver monitoring ECU module is used to control the advanced driver assistance system (ADAS) controller based on the judgment results of the steering wheel torque sensor module, the steering wheel capacitance sensor module, and the driver monitoring camera module.

[0021] Thirdly, the present invention provides a processor for running a program, wherein the program executes any of the above-mentioned intelligent driving vehicle hands-free recognition methods during runtime.

[0022] Fourthly, the present invention provides a computer storage medium storing program instructions, wherein the program instructions, when executed, control the device where the computer storage medium is located to execute any of the above-mentioned intelligent driving vehicle hands-free recognition methods.

[0023] This invention discloses a method, device, processor, and storage medium for hands-free recognition in intelligent driving vehicles. By fusing multiple sensors, including a steering wheel torque sensor, a steering wheel capacitance sensor, and a driver monitoring camera, it reduces system misjudgments and provides the ADAS / AD driving system with more levels of driver operation judgment for enabling different levels of autonomous driving functions, thereby improving the safety and reliability of various functions of the ADAS / AD system. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a flowchart of the intelligent driving vehicle hands-free recognition method of the present invention;

[0026] Figure 2 This is a schematic diagram of the intelligent driving vehicle hands-free recognition device of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the specific embodiments described herein are merely illustrative of the invention and represent only a portion, not all, of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0028] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0029] Figure 1 The diagram shown is a flowchart of an embodiment of a hands-free recognition method for intelligent driving vehicles provided by the present invention. The flowchart specifically includes:

[0030] Step 1: Obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with a first preset value, and determine whether the steering wheel is twisting.

[0031] Step 2: Obtain the steering wheel touch sensing value through the steering wheel capacitive sensor, and determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing value.

[0032] Step 3: Obtain facial recognition information through the driver monitoring camera, and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed.

[0033] Step 4: Control the Advanced Driver Assistance System (ADAS) controller based on the judgment results of Step 1, Step 2, and Step 3.

[0034] Specifically, when the actual torque value of the steering wheel is greater than the first preset value, if the driver's hands and the driver's line of sight meet the first preset conditions, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The first preset conditions include the driver's line of sight being on the road surface, or the driver's line of sight being unconfirmed when the driver's hands are on the steering wheel.

[0035] Specifically, when the actual torque value of the steering wheel is greater than the first preset value, if the driver's hands and the driver's line of sight meet the second preset condition, then the duration T2 of meeting the second preset condition is obtained. If the duration T2 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable L3 and above functions. The second preset condition includes the driver's line of sight being unable to be confirmed when the driver's hands are on the steering wheel, or the driver's line of sight being unable to be confirmed or not on the steering wheel when the driver's hands are not on the steering wheel.

[0036] Preferably, L3 and above refers to the level of autonomous driving that allows hands-free driving, while functions below L3 refer to driving assistance functions that do not allow hands-free driving.

[0037] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the third preset condition, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The third preset condition includes the driver's line of sight being on the road surface when the driver's hands are on the steering wheel.

[0038] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fourth preset condition, then the duration T4 of meeting the fourth preset condition is obtained. If the duration T4 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable L3 and above functions. The fourth preset condition includes the driver's line of sight not being on the road or being unable to confirm when the driver's hands are on the steering wheel, or the driver's line of sight being on the road when the driver's hands are not on the steering wheel.

[0039] Specifically, when the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fifth preset condition, then the duration T5 of meeting the fifth preset condition is obtained. If the duration T5 is less than the second preset time t5, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable functions below L3. The fifth preset condition includes the inability to confirm the driver's line of sight information or the driver's line of sight being off the road when the driver's hands are not on the steering wheel.

[0040] Specifically, when the duration T2 of satisfying the second preset condition or the duration T4 of satisfying the fourth preset condition is greater than or equal to the first preset time t2, or the duration T5 of satisfying the fifth preset condition is greater than or equal to the second preset time t5, the Advanced Driver Assistance System (ADAS) controller does not have permission to enable any function.

[0041] The detection strategy of this invention mainly integrates the actual torque value of the steering wheel provided by the steering wheel torque sensor, the judgment signal of whether the driver's hands are on the steering wheel provided by the steering wheel capacitance sensor, and the judgment signal of whether the driver's line of sight is on the road provided by the driver monitoring camera. Through signal fusion and cross-judgment, the ADAS / AD system accurately outputs the level of whether the driver is paying attention to the vehicle's driving status, so that the ADAS / AD system can make subsequent reminder logic.

[0042] For example, the detection strategy is shown in the table below:

[0043] When the actual steering wheel torque value is greater than the preset value T1, output A = 1; when the actual steering wheel torque value is less than or equal to the preset value T1, output A = 2. When the driver's hands are on the steering wheel, output B = 1; when the driver's hands are not on the steering wheel, output B = 2. When the driver's eyes are on the road, output C = 1; when the driver's eyes cannot be seen, output C = 2; when the driver's eyes are not on the road, output C = 3. The driver monitoring ECU performs logical judgments on the above A, B, and C to obtain the value of D, as shown in Table 1.

[0044] Table 1

[0045]

[0046]

[0047] When D=1, the Advanced Driver Assistance System (ADAS) controller has the authority to enable all levels of functions; when D=2, the duration of the D=2 state is obtained. If this duration is less than the first preset time t2, the ADAS controller has the authority to enable L3 and above functions; when D=3, the duration of the D=3 state is obtained. If this duration is less than the second preset time t5, the ADAS controller has the authority to enable functions below L3; when the duration of the D=2 state is greater than or equal to the first preset time t2 or the duration of the D=3 state is greater than or equal to the second preset time t5, D=4. In this case, the ADAS controller does not have the authority to enable any functions.

[0048] Figure 2 The diagram shown is a structural schematic of an embodiment of a hands-free recognition device for intelligent driving vehicles provided by the present invention. Figure 2 As shown, the device includes:

[0049] The steering wheel torque sensor module is used to obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with a first preset value, and determine whether the steering wheel is twisting.

[0050] The steering wheel capacitive sensor module is used to acquire steering wheel touch sensing values ​​through the steering wheel capacitive sensor, and to determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing values.

[0051] The driver monitoring camera module is used to acquire facial recognition information through the driver monitoring camera and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed.

[0052] The driver monitoring ECU module is used to control the advanced driver assistance system (ADAS) controller based on the judgment results of the steering wheel torque sensor module, the steering wheel capacitance sensor module, and the driver monitoring camera module.

[0053] According to another aspect of the present invention, a processor is provided for running a program, wherein the program executes the intelligent driving vehicle hands-free recognition method described above.

[0054] According to another aspect of the present invention, a computer storage medium is provided, which stores program instructions, wherein the program instructions, when executed, control the device where the computer storage medium is located to execute any of the above-described intelligent driving vehicle hands-free recognition methods.

[0055] The above-described embodiments merely illustrate preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for recognizing hands-free driving in intelligent vehicles, characterized in that, Includes the following steps: Step 1: Obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with the first preset value, and determine whether the steering wheel has been twisted. Step 2: Obtain the steering wheel touch sensing value through the steering wheel capacitive sensor, and determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing value; Step 3: Obtain facial recognition information through the driver monitoring camera, and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed. Step 4: Control the Advanced Driver Assistance System (ADAS) controller based on the judgment results of Step 1, Step 2, and Step 3; When the actual torque value of the steering wheel is greater than the first preset value, if the first preset condition is met, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The first preset condition includes the driver's hands being on the steering wheel and the driver's line of sight being unable to confirm the position.

2. The intelligent driving vehicle hands-free recognition method according to claim 1, characterized in that, The first preset condition also includes the driver's line of sight information being on the road surface.

3. The intelligent driving vehicle hands-free recognition method according to claim 1 or 2, characterized in that, When the actual torque value of the steering wheel is greater than the first preset value, if the driver's hands and the driver's line of sight meet the second preset condition, then the duration T2 of meeting the second preset condition is obtained. If the duration T2 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to activate L3 and above functions. The second preset condition is any of the following: the driver's hands are on the steering wheel and the driver's line of sight is not on the road; or, the driver's hands are not on the steering wheel and the driver's line of sight cannot be confirmed; or, the driver's hands are not on the steering wheel and the driver's line of sight is not on the road.

4. The intelligent driving vehicle hands-free recognition method according to claim 3, characterized in that, When the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the third preset condition, the Advanced Driver Assistance System (ADAS) controller has the authority to activate all levels of functions. The third preset condition includes the driver's line of sight being on the road surface when the driver's hands are on the steering wheel.

5. The intelligent driving vehicle hands-free recognition method according to claim 4, characterized in that, When the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fourth preset condition, then the duration T4 of meeting the fourth preset condition is obtained. If the duration T4 is less than the first preset time t2, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable L3 and above functions. The fourth preset condition includes the driver's line of sight not being on the road or being unable to confirm it when the driver's hands are on the steering wheel, or the driver's line of sight being on the road when the driver's hands are not on the steering wheel.

6. The intelligent driving vehicle hands-free recognition method according to claim 5, characterized in that, When the actual torque value of the steering wheel is less than or equal to the first preset value, if the driver's hands and the driver's line of sight meet the fifth preset condition, then the duration T5 of meeting the fifth preset condition is obtained. If the duration T5 is less than the second preset time t5, then the Advanced Driver Assistance System (ADAS) controller has the authority to enable functions below L3. The fifth preset condition includes the inability to confirm the driver's line of sight information or the driver's line of sight being off the road when the driver's hands are not on the steering wheel.

7. The intelligent driving vehicle hands-free recognition method according to claim 6, characterized in that, When the duration T2 of satisfying the second preset condition is greater than or equal to the first preset time t2, or the duration T4 of satisfying the fourth preset condition is greater than or equal to the first preset time t2, or the duration T5 of satisfying the fifth preset condition is greater than or equal to the second preset time t5, the Advanced Driver Assistance System (ADAS) controller does not have permission to enable any function.

8. A hands-free recognition device for intelligent driving vehicles, used to implement the hands-free recognition method for intelligent driving vehicles as described in any one of claims 1-7, characterized in that, include: The steering wheel torque sensor module is used to obtain the actual torque value of the steering wheel through the steering wheel torque sensor, compare the actual torque value of the steering wheel with a first preset value, and determine whether the steering wheel is twisting. The steering wheel capacitive sensor module is used to acquire steering wheel touch sensing values ​​through the steering wheel capacitive sensor, and to determine whether the driver's hands are on the steering wheel based on the steering wheel touch sensing values. The driver monitoring camera module is used to acquire facial recognition information through the driver monitoring camera and generate gaze information data based on the facial recognition information. The gaze information data is used to determine whether the driver's gaze is on the road surface, not on the road surface, or cannot be confirmed. The driver monitoring ECU module is used to control the advanced driver assistance system (ADAS) controller based on the judgment results of the steering wheel torque sensor module, the steering wheel capacitance sensor module, and the driver monitoring camera module.

9. A processor, characterized in that, The processor is used to run a program, wherein the program executes the hands-free recognition method for intelligent driving vehicles according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the intelligent driving vehicle hands-free recognition method according to any one of claims 1 to 7.

Images