Method and device for detecting a hands-on / hands-off state of a steering wheel of a vehicle

Abstract

The invention relates to a method for detecting a hands-on / hands-off state (6) of a steering wheel (51) of a vehicle (50), wherein at least one steering variable (4) is detected on the steering wheel (51), the hands-on / hands-off state (6) is estimated at least on the basis of the detected at least one steering variable (4) using a trained machine learning method (5), and the detection of the hands-on / hands-off state (6) is deactivated if the vehicle (50) is in a hands-off domain (7). The invention additionally relates to a device (1) for detecting a hands-on / hands-off state (6) of a steering wheel (51) of a vehicle (50).

Description

[0001] Description

[0002] Method and device for detecting a hands-on / off state on a steering wheel of a vehicle

[0003] The invention relates to a method and a device for detecting a hands-on / off state on a steering wheel of a vehicle.

[0004] Sensors such as a capacitive steering wheel are used in vehicles to monitor driver activity. Such a steering wheel detects when the driver has touched (“hands-on”) or not touched (“hands-off”) the steering wheel using a capacitive sensor. The result is transmitted to the functions being used, such as a longitudinal and / or lateral guidance assistance system. The contact of the hands on the steering wheel can be used to determine driver activity and the driver’s level of attention. For example, the system can remind the driver to put their hands on the steering wheel if it is detected that their hands have not been on the steering wheel for a specified time during lateral guidance.

[0005] To save additional costs for a capacitive sensor in the steering wheel, it is known to monitor driver activity using artificial neural networks based on a torque (hand torque) detected on the steering wheel. One such method is known, for example, from DE 102019211 016 A1. Another method is known from CN 115782892 A.

[0006] The invention is based on the object of improving a method and a device for detecting a hands-on / off state on a steering wheel.

[0007] The object is achieved according to the invention by a method having the features of patent claim 1 and a device having the features of patent claim 8. Advantageous embodiments of the invention emerge from the subclaims.

[0008] In particular, a method for detecting a hands-on / off state on a steering wheel of a vehicle is provided, wherein at least one steering variable is detected on the steering wheel, wherein the hands-on / off state is estimated based on at least the detected at least one steering variable by means of a trained machine learning method, wherein the detection of the hands-on / off state is deactivated when the vehicle is in a hands-off domain.

[0009] Furthermore, in particular, a device for detecting a hands-on / off state on a steering wheel of a vehicle is provided, comprising at least one steering variable sensor configured to detect at least one steering variable on the steering wheel, a control device, wherein the control device is configured to receive the detected at least one steering variable and to provide a trained machine learning method, wherein the trained machine learning method is trained to estimate the hands-on / off state based on at least the detected at least one steering variable, and wherein the control device is further configured to deactivate the detection of the hands-on / off state when the vehicle is in a hands-off domain.

[0010] The method and device make it possible to reliably detect a hands-on / off state on the steering wheel and yet save computing and memory resources when the detection of the hands-on / off state is not necessary because the vehicle is currently driving in a hands-off domain (partially automated or automated). This is achieved by deactivating the detection of the hands-on / off state within a hands-off domain. In the active state, the hands-on / off state is detected using a trained machine learning method (i.e., using a trained machine learning model), for example, using one or more trained artificial neural networks. In the deactivated state, computing and memory resources otherwise required for this trained machine learning method can be released and, after being released, used for other applications.The estimated hands-on / off state is then provided, in particular output and / or fed to a control unit of the vehicle.

[0011] A hands-off state is, in particular, a state in which the driver does not touch the steering wheel. In particular, none of the driver's fingers are in contact with the steering wheel. A hands-on state is, in particular, a state in which the driver does touch the steering wheel. Detecting the hands-on / off state can, in particular, comprise providing a hands-on / off state signal. This comprises, for example, a probability for a hands-on state and a probability for a hands-off state or coded signals for the states "hands-off detected" and "hands-on detected." Furthermore, the signal can also additionally comprise the state "no detection possible," for example if the uncertainty regarding detection is too great.

[0012] A hands-on domain refers specifically to an area, section, or region of the vehicle's environment in which the vehicle may only be driven while the driver has their hands on the steering wheel. The area, section, or region can be defined, for example, by a geographical region or by a road type (e.g., motorway, expressway, country road, village road, play street, etc.).

[0013] A hands-off domain refers specifically to an area, section, or region of the vehicle's environment in which the vehicle may be driven without the driver having their hands on the steering wheel. In particular, the vehicle then drives semi-automatically or automatically. The area, section, or region can be defined, for example, by a geographical region or by a road type (e.g., motorway, expressway, country road, village road, play street, etc.).

[0014] It can be provided that, in particular, it is regularly determined whether the vehicle is in a hands-on domain or a hands-off domain. This can be done, for example, by querying a navigation system which requests a geographical coordinate of the vehicle and / or a road type, etc. Based on the geographical coordinate, it can be checked, for example, whether the vehicle is in a certain geographical region or not. Based on the road type, it can be determined whether the vehicle is on a certain road type or not. For example, it can be provided in the future that a motorway is generally classified as a hands-off domain and that driving without hands on the steering wheel is therefore generally possible there. In general, however, the domain can also be defined individually for each region and / or each road type and / or an area arbitrarily defined by geographical coordinates.The information about which domain is currently present can be queried, for example, directly from a suitable navigation system based on the vehicle's current position. Alternatively or additionally, a map containing the domains with spatial resolution can be stored in the control unit's memory.

[0015] A steering variable is, in particular, a variable that represents and / or describes a current state of the steering wheel. A steering variable is, in particular, a torque, which is detected in particular by means of a torque sensor on the steering wheel. In principle, however, a steering variable can also be another variable detected directly or indirectly on the steering wheel. For example, it can be provided to detect a current in an electrical machine on the steering wheel and use it as a steering variable. The hands-on / off state can be detected exclusively on the basis of the steering variable detected on the steering wheel, in particular a detected torque. However, it is also possible, in particular, for the trained machine learning process to be provided with further (steering) variables that are detected on the steering wheel (e.g., a steering wheel angle and / or a steering wheel angular velocity, etc.).) and the trained machine learning process recognizes the hands-on / off state taking this additional variable(s) into account. Furthermore, variables that are not detected at the steering wheel can also be taken into account, such as vehicle speed, lateral acceleration, yaw rate, wheel ticks, damper information, and / or other driving dynamics variables, etc. In particular, however, no capacitive sensor is provided on the steering wheel.

[0016] The machine learning method is designed in particular as a neural network and comprises, in particular, several inner layers. The machine learning method is, in particular, an artificial recurrent neural network that processes the input data X at each time point t. t processed and a hands-off probability y t in [0,1] outputs: y t = p(x t| xo:ti). In particular, the neural network has a so-called memory h, in which information from previous magazines is stored and which can be used for the output of the current magazine. The output is further processed, for example, by filtering before the decreasing functions (e.g., a lateral guidance assistant) process it. In particular, it can be provided that, based on a comparison of the hands-off probability with a predetermined threshold value, a binary hands-on / off signal is provided (with the two states "hands-off detected" and "hands-off not detected"). In the same way, a hands-on probability can also be estimated. Taking both probabilities into account, a hands-on / off state signal can be generated and provided.In this case, for example, the state can be selected depending on which of the probabilities has the higher value. This hands-on / off state signal can, for example, comprise the following alternative states: "Hands-on detected" and "Hands-off detected." In addition, a "detection not possible / error" state can be provided. The hands-on / off state signal can also contain values ​​for the probabilities described above. During a training phase, the machine learning method is or was trained in particular using training data comprising pairs in which data of the at least one steering variable, in particular torque data, are each paired with a hands-on / off state (as ground truth).The data of the at least one steering variable, in particular the torque data, are in particular time series of the at least one steering variable detected at the steering wheel, in particular time series of torques detected at the steering wheel. The training data is obtained in particular with the aid of test drives and / or in simulators. In principle, the provision of training data can be carried out in particular according to the method described in DE 102019211016 A1. Training is carried out in a conventional manner, in particular by means of supervised learning.

[0017] Parts of the device, in particular the control device, can be implemented individually or collectively as a combination of hardware and software, for example, as program code executed on a microcontroller or microprocessor. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA). The control device comprises, in particular, at least one computing device and at least one memory.

[0018] In one embodiment, the detection of the hands-on / off state is activated proactively when the vehicle is in a hands-off domain and approaching a hands-on domain. This allows detection to occur or at least be prepared directly at the transition from the hands-off to the hands-on domain.

[0019] In one embodiment, activation occurs when the time until reaching the hands-on domain falls below a predefined time threshold and / or when the distance until reaching the hands-on domain falls below a predefined distance threshold. The remaining time can be determined, for example, using information retrieved and / or provided by a navigation system of the vehicle. The predefined time threshold can be, for example, a few seconds, e.g., 3, 5, or 10 seconds. The predefined distance threshold can be, for example, a few meters, e.g., 50 m, 75 m, or 100 m.The threshold values ​​may also be set taking into account a time required to activate the trained machine learning method, that is to say, in particular, the time required to load the trained machine learning method into a memory of the control device and to provide the functionality of the trained machine learning method.

[0020] In one embodiment, it is provided that at least during the transition to a hands-on domain and / or at the beginning of the hands-on domain, a trained machine learning method with higher sensitivity is activated and / or used as the trained machine learning method. This makes it possible to take into account that determining whether the driver has their hands on the steering wheel is particularly critical during the transition. For example, it can be provided to use a particularly large neural network or an ensemble of trained neural networks. Alternatively or additionally, it can be provided that a trained machine learning method is activated and used that is specifically trained and specialized for the context of a transition between a hands-off domain and a hands-on domain.Subsequently, for example after a specified time and / or distance after the transition, a (general) trained machine learning method, which may be less computationally and / or memory-intensive, can be used.

[0021] In one embodiment, a test torque is applied to the steering wheel at least at the beginning of the hands-on domain. This can improve the detection of the hands-on / off state, since a counter-torque of a hand placed on the steering wheel that counteracts the test torque can be used to more reliably determine whether a driver is touching the steering wheel or not. For this purpose, the device comprises, in particular, at least one actuator for applying the test torque to the steering wheel.

[0022] In one embodiment, a test torque is applied to the steering wheel if the trained machine learning method detects a hands-on state with a confidence value below a predetermined threshold. This ensures that a test torque is only applied if the hands-on state is detected with uncertainty. In other cases, however, the application, which may be perceived as disruptive by the driver, can be omitted.

[0023] In one embodiment, if the trained machine learning method detects a hands-off state in the hands-on domain, an acoustic and / or visual and / or haptic request to the driver to take control of the steering wheel is generated and output. This can provide a warning to the driver, prompting them to take control of the steering wheel. If the hands-off state is detected for a predetermined maximum duration, the vehicle can be automatically transferred to a safe state, for example by bringing the vehicle to a stop on a hard shoulder. For this purpose, the control device can transmit a control signal to a vehicle control system.

[0024] Further features of the device design will become apparent from the description of embodiments of the method. The advantages of the device are the same as those of the embodiments of the method.

[0025] Furthermore, in particular, a steering system is also created, comprising a device according to one of the described embodiments.

[0026] Furthermore, in particular, a vehicle is also created, comprising a steering system according to one of the described embodiments and / or a device according to one of the described embodiments.

[0027] The invention will be explained in more detail below using preferred embodiments with reference to the figures.

[0028] Fig. 1 is a schematic diagram illustrating embodiments of the device for detecting a hands-on / off state on a steering wheel of a vehicle;

[0029] Fig. 2 is a schematic flow diagram illustrating embodiments of the method for detecting a hands-on / off state on a steering wheel of a vehicle.

[0030] Fig. 1 shows a schematic representation to illustrate embodiments of the device 1 for detecting a hands-on / off state 6 on a steering wheel 51 of a vehicle 50. The device 1 is arranged in particular in a vehicle 50. The method described in this disclosure is illustrated and explained in more detail below using the device 1.

[0031] The device 1 comprises a steering variable sensor 2 and a control device 3. The

[0032] Steering variable sensor 2 is configured to detect a steering variable 4 at the steering wheel 51 of the vehicle 50. Steering variable sensor 2 is, for example, a torque sensor, and steering variable 4 is a torque.

[0033] The control device 3 comprises a computing device 3-1 and a memory 3-2. The computing device 3-1 is configured to perform the computing operations necessary for implementing measures of the method and can access data stored in the memory 3-2 for this purpose.

[0034] The control device 3 is configured to receive the detected steering variable 4 and to provide a trained machine learning method 5. The trained machine learning method 5 is trained to estimate the hands-on / off state 6 based on at least the detected steering variable 4.

[0035] The hands-off state 6 is fed, for example, as a state signal to a control unit 52 of the vehicle 50 for further processing. The control unit 52 can, for example, be a lateral guidance assistant or another assistance system. The state signal can, for example, be a hands-off probability or a binary state value with the two states "hands-off detected" and "hands-off not detected." Furthermore, the state signal can also include the alternative states "hands-off detected" and "hands-on detected." Additionally, a "detection not possible / error" state can be provided if, for example, a confidence value for the other two states is too low. It can also be provided that the trained machine learning method estimates and outputs the probabilities for the presence of the hands-on state and the presence of the hands-off state, which can then be provided as part of the state signal.

[0036] The control device 3 is further configured to deactivate the recognition of the hands-on / off state 6 when the vehicle 50 is in a hands-off domain. To this end, the control device 3 can, for example, query a navigation system 53 of the vehicle 50 as to whether the vehicle 50 is currently in a hands-off domain 7 or not. The navigation system 53 can, for example, provide a signal for a hands-off domain 7 or a signal for a hands-on domain 8, depending on the domain. Furthermore, the navigation system 53 can also transmit to the control device 3 how far the vehicle 50 is from a hands-on domain and / or how long the vehicle 50 still needs to reach the hands-on domain. It can be provided that the recognition of the hands-on / off state 6 is activated in advance when the vehicle 50 is in a hands-off domain 7 and is approaching a hands-on domain 8.

[0037] Activation can be provided when the time until reaching the hands-on domain 8 falls below a predefined time threshold 9 and / or when the distance until reaching the hands-on domain 8 falls below a predefined distance threshold 10. The thresholds 9, 10 can be specified, for example, by a driver or a manufacturer. The information regarding the time or distance until reaching the hands-on domain 8 can be queried from and / or provided by the navigation system 53, for example.

[0038] It can be provided that, at least during the transition to a hands-on domain 8 and / or at the beginning of the hands-on domain 8, a trained machine learning method with higher sensitivity is activated and / or used as the trained machine learning method 5. After the transition, a trained machine learning method 5 that requires fewer computing and / or memory resources can then be used again.

[0039] It can be provided that, at least at the beginning of the hands-on domain 8, a test torque is applied to the steering wheel 50. This is done in particular by means of an actuator 11 of the device 1 arranged directly or indirectly on the steering wheel. For this purpose, the control device 3 controls the actuator 11 accordingly so that it applies the test torque to the steering wheel 51.

[0040] It can be provided that a test torque is applied to the steering wheel 51 when the trained machine learning method 5 detects a hands-on state with a confidence value that is below a predetermined threshold value.

[0041] It can be provided that, in the case in which the trained machine learning method 5 detects a hands-off state in the hands-on domain 8, an acoustic and / or visual and / or haptic request to take over the steering wheel 51 is generated and output to the driver. This is done in particular by means of a suitable signal generator 12, which is controlled by the control device 3 in this case. The signal generator 12 can, for example, comprise a display device, a loudspeaker and / or a haptic actuator. The device 1 can, in particular, be part of a steering system 60 that comprises the device 1.

[0042] Figure 2 shows a schematic flowchart illustrating embodiments of the method for detecting a hands-on / off state on a steering wheel of a vehicle. The method is implemented, for example, using a device as described with reference to Figure 1.

[0043] In action 100, a domain in which the vehicle is currently located is determined. For this purpose, for example, a query is made to the vehicle's navigation system.

[0044] In step 101, the current domain is checked. If the domain is a hands-off domain, the process continues with step 200; if the domain is a hands-on domain, the process continues with step 102.

[0045] In measure 200, the trained machine learning method is deactivated (or deactivation is maintained). For this purpose, it can be provided that the execution of computational operations necessary to provide the trained machine learning method is terminated and memory areas in which the information about the trained machine learning method is stored are released for use by other applications.

[0046] In an action 201, a time and / or distance until reaching a hands-on domain is determined. This can be done using information provided by the vehicle's navigation system. A planned route can also be taken into account.

[0047] In measure 202, it is checked whether the specific time and / or the specific distance fall below a predetermined threshold (time threshold and / or distance threshold). If this is not the case, the system returns to measure 201. If this is the case, however, the trained machine learning method is reactivated in measure 203. For this purpose, information from the trained machine learning method is (again) loaded into the memory of the control device, and the computing operations necessary to provide the trained machine learning method are (again) performed. It can be provided that, at least during the transition to a hands-on domain and / or at the beginning of the hands-on domain, a trained machine learning method with higher sensitivity is activated and / or used as the trained machine learning method. In measure 102, at least one steering variable is recorded on the steering wheel.

[0048] In measure 103, the hands-on / off state is estimated based on at least one detected steering variable using the trained machine learning method. The estimated hands-on / off state is then provided, in particular output.

[0049] Then return to measure 100.

[0050] An optional measure 204 may provide for a test torque to be applied to the steering wheel at least at the beginning of the hands-on domain.

[0051] It can be provided that, in the case where the trained machine learning method detects a hands-off state in the hands-on domain, an acoustic and / or visual and / or haptic request to take over the steering wheel is generated and output to the driver. This can be verified in a measure 104. If a hands-off state is detected, the acoustic and / or visual and / or haptic request to take over the steering wheel is generated and output to the driver in measure 105.

[0052] Further embodiments of the method have already been described above with reference to the device.

[0053] List of reference symbols

[0054] device

[0055] Steering size sensor

[0056] Control device -1 Computing device -2 Memory

[0057] Steering variable trained machine learning method

[0058] Hands-Off / On state

[0059] Hands-off domain

[0060] Hands-on domain

[0061] Time threshold 0 Distance threshold 1 Actuator 2 Signal generator 0 Vehicle 1 Steering wheel 2 Control unit 3 Navigation system 0 Steering system 00-105 Measures of the procedure 00-204 Measures of the procedure

Claims

Patent claims 1. Method for detecting a hands-on / off state (6) on a steering wheel (51) of a vehicle (50), wherein at least one steering variable (4) is detected on the steering wheel (51), wherein the hands-on / off state (6) is estimated based on at least the detected at least one steering variable (4) by means of a trained machine learning method (5), wherein the detection of the hands-on / off state (6) is deactivated when the vehicle (50) is in a hands-off domain (7).

2. Method according to claim 1, characterized in that the recognition of the hands-on / off state (6) is activated in advance when the vehicle (50) is in a hands-off domain (7) and is approaching a hands-on domain (8).

3. Method according to claim 2, characterized in that the activation takes place when a time until reaching the hands-on domain (8) falls below a predetermined time threshold (9) and / or when a distance until reaching the hands-on domain (8) falls below a predetermined distance threshold (10).

4. The method according to claim 2 or 3, characterized in that at least during the transition to a hands-on domain (8) and / or at the beginning of the hands-on domain (8), a trained machine learning method with higher sensitivity is activated and / or used as the trained machine learning method (5).

5. Method according to one of the preceding claims, characterized in that at least at the beginning of the hands-on domain (8) a test torque is applied to the steering wheel (51).

6. Method according to one of the preceding claims, characterized in that a test torque is applied to the steering wheel (51) when the trained machine learning method (5) detects a hands-on state with a confidence value that is below a predetermined threshold value.

7. Method according to one of the preceding claims, characterized in that in the case in which the trained machine learning method (5) detects a hands-off state in the hands-on domain (8), an acoustic and / or visual and / or haptic request to take over the steering wheel (51) is generated and output to the driver.

8. Device (1) for detecting a hands-on / off state (6) on a steering wheel (51) of a vehicle (50), comprising: at least one steering variable sensor (2) configured to detect at least one steering variable (4) on the steering wheel, a control device (3), wherein the control device (3) is configured to receive the detected at least one steering variable (4), to provide a trained machine learning method (5), wherein the trained machine learning method (5) is trained to estimate the hands-on / off state (6) based on at least the detected at least one steering variable (4), and wherein the control device (3) is further configured to deactivate the detection of the hands-on / off state (6) when the vehicle (50) is in a hands-off domain (7).

9. Steering system (60) comprising a device (1) according to claim 8.

10. Vehicle (50) comprising a steering system (60) according to claim 9 and / or a device (1) according to claim 8.

Images