Method and device for detecting contact between hands and a vehicle steering wheel

Abstract

The invention relates to a method for detecting contact between hands and a steering wheel (52) of a vehicle (50), wherein, based on at least detected and / or received state data (10) of a steering system (51) of the vehicle (50), a decision (20) is made by means of at least one trained machine learning method (3) as to whether at least one hand is in contact with the steering wheel (52) or not, wherein the actuation of at least one control element (54-x) arranged on the steering wheel (52) is taken into account as an input variable (11) of the at least one trained machine learning method (3), and wherein a decision signal (21) is generated and provided. Furthermore, the invention relates to a device (1) for detecting contact between hands and a steering wheel (52) of a vehicle (50).

Description

[0001] The invention relates to a method and a device for detecting contact of hands with a steering wheel of a vehicle.

[0002] In modern vehicles, drivers are supported by driver assistance systems that provide driver assistance functions. Examples of these include automated or semi-automated longitudinal and / or lateral guidance of the vehicle. Currently, it is still common practice for the driver to place their hands on the steering wheel at regular intervals to enable automated or semi-automated operation. Otherwise, the driver assistance function is deactivated. Various sensors and / or methods can be used to detect at least one hand on the steering wheel (also known as hands-on detection).

[0003] For example, a capacitive sensor on a vehicle's steering wheel can be used to determine whether or not at least one of the driver's hands is on the steering wheel. The capacitively recorded sensor data can be used to determine a multi-stage status (e.g. no contact, lightly gripped left / right to firmly gripped on both sides, etc.), which is then transmitted to support functions such as a longitudinal and / or lateral guidance assistance system. For example, such a status can be fed into an adaptive cruise control (ACC) as an input. If contact between at least one hand and the steering wheel is detected, a pull-off procedure to follow a vehicle in front is activated, for example. Driver observation cameras that can detect driver activity are also known.

[0004] In addition, there are methods that detect driver activity, particularly with regard to hands-on detection, using artificial intelligence (machine learning) or rule-based approaches. For example, an artificial intelligence method, in particular an artificial neural network, can be trained to perform hands-on detection based on a torque detected on a steering system, a detected steering angle, and / or changes in these variables. Such a method is known, for example, from US 2022 / 0 161 846 A1. The rule-based approach, on the other hand, passively evaluates steering signals (e.g.The control system compares a steering torque (e.g., a steering torque, a steering speed, and / or a steering angle) with threshold values ​​and / or signal curves and / or actively operates with a torque feedforward function, in which a test signal is applied to the steering wheel by means of an actuator. This test signal is intended to generate a counter-torque from hands positioned on the steering wheel, which can be detected by a sensor. As soon as such a counter-torque is detected, it is assumed that at least one hand is positioned on the steering wheel. Such a method is known, for example, from DE 10 2013 209 459 A1.

[0005] DE 10 2019 211 455 A1 discloses a method for controlling a vehicle. It provides an additional wheel steering angle by manually actuating an additional control element on the vehicle's steering element, eliminating the need to shift the steering wheel with the hands to further increase the steering angle, even when cornering tighter.

[0006] DE 10 2019 211 738 B3 discloses a method for detecting a transfer of control by a driver of a vehicle controlled by an autonomous driving assistance system. The method comprises: receiving image data from at least one camera directed at the driver of the vehicle such that at least a portion of the driver is depicted in the image data; determining a steering wheel value from the image data, wherein the steering wheel value indicates whether the driver is gripping a steering wheel of the vehicle with one hand; determining an accelerator pedal value from the image data, wherein the accelerator pedal value indicates whether the driver is touching an accelerator pedal of the vehicle with one foot; and transferring control of the vehicle to the driver by the driving assistance system if the steering wheel value and the accelerator pedal value indicate that the driver is gripping the steering wheel and touching the accelerator pedal.

[0007] The invention is based on the object of improving a method and a device for detecting contact of hands with a steering wheel of a vehicle.

[0008] The object is achieved according to the invention by a method having the features of patent claim 1 and a device having the features of patent claim 10. Advantageous embodiments of the invention emerge from the subclaims.

[0009] In particular, a method for detecting contact of hands with a steering wheel of a vehicle is provided, wherein, based on at least recorded and / or received status data of a steering system of the vehicle, a decision is made by means of at least one trained machine learning method as to whether or not at least one hand is in contact with the steering wheel, wherein an actuation of at least one control element arranged on the steering wheel is taken into account as an input variable of the at least one trained machine learning method, and wherein a decision signal is generated and provided.

[0010] Furthermore, in particular, a device for detecting contact of hands with a steering wheel of a vehicle is provided, comprising a data processing device, wherein the data processing device is configured to receive detected and / or queried status data of a steering system of the vehicle and to provide at least one trained machine learning method which, based on at least the received status data, makes a decision as to whether or not at least one hand is in contact with the steering wheel, wherein an actuation of at least one control element arranged on the steering wheel is taken into account as an input variable of the at least one trained machine learning method, and to provide a decision signal.

[0011] The method and device make it possible to consider the actuation of at least one control element located on the steering wheel as an input variable when deciding whether a hand is in contact with the steering wheel or not (hands-on detection). This provides at least one trained machine learning method with an additional input datum or dimension of input data. Overall, this makes the decision-making process more robust because more data can be considered.

[0012] Steering system status data includes, in particular, a detected steering torque and / or a detected steering angle. Furthermore, changes in these variables can also be taken into account. The status data and the input variable are fed into the trained machine learning process as input data.

[0013] A trained machine learning method is, in particular, a trained neural network, and in particular a trained deep neural network. The trained machine learning method is trained to make a decision or estimate, based on the state data of the steering system, whether at least one hand is on the steering wheel or not. A result is provided as a decision signal. The decision can comprise at least two possible values ​​(e.g., contact yes / no or corresponding values ​​of a data packet or signal level). In principle, however, gradations in the decision are also possible (e.g., no hand on the steering wheel, one hand on the steering wheel, both hands on the steering wheel, etc.). The machine learning method is trained in a known manner based on training data. The training data consists, in particular, of pairs of state data as input data and known decision data (contact yes / no, etc.) as output data.Ground truth is used. Furthermore, information about the actuation of at least one control element is also used as input data. This means that the information about whether or not the at least one control element is actuated is also taken into account as input data during training (and later when applying the machine learning method).

[0014] A steering wheel control element is, in particular, a push-button, a slider, a button, or a potentiometer. Examples of steering wheel controls include a telephone control (e.g., buttons for "accept call," "end call," etc.), a multimedia control (e.g., buttons for "volume up," "quiet down," "next," "mute," etc.), a cruise control (e.g., buttons for "SET," "RESUME," etc.), and a gearshift lever (e.g., levers or buttons for "increase gear," "decrease gear," Tiptronic®, etc.).

[0015] Parts of the device, in particular the data processing device, can be implemented individually or collectively as a combination of hardware and software, for example, as program code executed on a microcontroller or microprocessor. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA).

[0016] In one embodiment, it is provided that at least a subset of the operating elements arranged on the steering wheel is taken into account individually. This allows each operating element on the steering wheel to be taken into account with regard to its individual properties when making a decision. In particular, it can be provided that all operating elements are taken into account individually. The trained machine learning method then additionally receives information as input data which uniquely identifies the at least one operating element actuated. The additional information can be used to improve decision-making; in particular, the decision-making process is more differentiated and thus more robust. When training the at least one machine learning method, the information which uniquely identifies the at least one operating element on the steering wheel is taken into account accordingly.

[0017] In one embodiment, at least a subset of the control elements arranged on the steering wheel is considered grouped by class. This can reduce the effort required to train the machine learning method, since training data no longer needs to be generated and provided for all control elements, but only for the classes. This reduces the effort and costs of the training phase. In particular, it can be provided to consider all control elements on the steering wheel grouped by class. Grouping by class can occur, for example, depending on the type of respective control element and / or the position of the respective control element on the steering wheel.

[0018] In one embodiment, when an actuation of the at least one control element is detected, a test signal is applied to the steering wheel, whereby a counter-torque caused by this is detected and taken into account in the decision. This makes it possible to check whether a hand is on the steering wheel or not when the at least one control element is actuated. The actuation of the at least one control element can also be improved, in particular in the form of a counter-torque, and taken into account in the decision. This makes it possible to further differentiate the decision-making process and thus make it more robust.

[0019] In one embodiment, it is provided that a type of the at least one control element actuated is taken into account. This also allows the type of the at least one control element actuated to be taken into account in the decision-making process. A type, for example, contains at least one of the following information: mechanical button, mechanical potentiometer, mechanical lever, capacitive button, etc. This allows the decision-making process to be further differentiated and thus made more robust.

[0020] In one embodiment, it is provided that a position of the at least one operating element on the steering wheel and / or a distance of the actuated at least one operating element from a rotation axis of the steering wheel is taken into account. This allows the position and / or distance to be taken into account in the decision-making process, in particular when a test signal is applied. The idea behind this is that when the test signal is applied, a counter-torque with the same actuating force with which the at least one actuating element is actuated will differ due to the position-dependent and / or distance-dependent leverage. For example, one button may be 5 cm from the rotation axis, while another button may be 10 cm. With the same test signal (torque), a detected counter-torque with the same actuating force on the buttons will differ by a factor of 2 due to the different lever arms.This allows decision-making to be further differentiated and thus made more robust.

[0021] In one embodiment, a correction factor is taken into account based on the at least one control element actuated. This makes it possible to take into account and / or correct a different leverage effect, particularly when applying the test signal. For example, it can be provided to correct a detected counter-torque using the correction factor and to feed the corrected counter-torque to the trained machine learning process as input data for decision-making. In the above example, for example, a correction factor of 2 could be provided for the button that is arranged at a distance of 5 cm from the axis of rotation, so that a detected counter-torque is ultimately doubled and is therefore equal to a detected counter-torque on the other button for the same actuation force.Such a correction factor can be determined and provided, for example, based on the position and / or distance of the at least one control element on the steering wheel being actuated. For example, a predefined rule for determining the correction factor can be stored in the data processing device. The correction factor can, for example, be a function of the distance: k = f(d) = c / d, where k is the correction factor, c is a (normalization) constant, and d is the distance from the axis of rotation. This allows the decision-making process to be further differentiated and thus made more robust.

[0022] In one embodiment, it is provided that when considering the actuated at least one control element, a driver profile is also taken into account. This allows individual characteristics of the driver to be taken into account. Individual characteristics (tall person, short person, etc.) can in particular influence the strength of the actuation. The driver profile can either be selected by the driver themselves or automatically selected based on recorded sensor data (e.g., from a weight sensor in the driver's seat and / or a driver camera, etc.). In particular, the correction factor described above can be determined, selected, and / or adjusted depending on the driver profile. This allows the decision-making process to be further differentiated and thus made more robust.

[0023] In one embodiment, it is provided that a driver is detected by means of at least one driver camera, wherein additionally at least one captured image from the at least one driver camera is evaluated and taken into account. This makes it possible, for example, to determine the driver profile described above. Furthermore, based on the at least one captured image, a driver's intention (operation of the control element is unintentional or intentional, driver is facing the steering wheel or away from it, etc.) can be determined and taken into account. An evaluation result is also fed to the trained machine learning process as input data and taken into account in the decision-making process. This makes it possible to further differentiate the decision-making process and thus make it more robust.

[0024] Further features of the device design will become apparent from the description of embodiments of the method. The advantages of the device are the same as those of the embodiments of the method.

[0025] The invention will be explained in more detail below using preferred embodiments with reference to the figure. Herein: Fig. 1 is a schematic diagram illustrating embodiments of the device for detecting contact of hands with a steering wheel of a vehicle.

[0026] The Fig. Figure 1 shows a schematic representation to illustrate embodiments of the device 1 for detecting contact between hands and a steering wheel 52 of a vehicle 50. The device 1 is arranged in particular in the vehicle 50. The device 1 is configured to carry out the method described in this disclosure. The method is explained in more detail below using the device 1.

[0027] The device 1 comprises a data processing device 2. The data processing device 2 comprises, for example, at least one computing device (not shown) and at least one memory (not shown). The at least one computing device performs the computing operations necessary to execute the method steps of the method.

[0028] The data processing device 2 is configured to receive detected and / or retrieved state data 10 of a steering system 51 of the vehicle 50, for example, from a steering controller 53, and to provide at least one trained machine learning method 3, which, based on at least the received state data 10, makes a decision 20 as to whether or not at least one hand is in contact with the steering wheel 52. The state data 10 can, for example, include a detected steering torque and / or a detected steering angle and / or changes in these variables. The at least one trained machine learning method 3 comprises, in particular, at least one trained neural network, in particular a trained deep neural network.

[0029] It is provided that an actuation of at least one control element 54-x arranged on the steering wheel 52 is taken into account as input variable 11 of the at least one machine learning method 3. For this purpose, the steering controller 53 can transmit a corresponding actuation signal 15 for the input variable 11 to the data processing device 2, which encodes the actuation of the at least one control element 54-x.

[0030] The data processing device 2 is further configured to provide a decision signal 21 based on the decision 20 made

[0031] The decision signal 21 can, for example, be fed to a driver assistance system 55 of the vehicle 50, which is activated or deactivated depending on the decision signal 21. Such a driver assistance system 55 can, for example, be an adaptive cruise control (ACC), wherein a following function, which enables following a vehicle in front after the vehicle 50 has come to a standstill, is activated or deactivated depending on whether a hand is in contact with the steering wheel 52 or not.

[0032] It can be provided that at least a subset of the control elements 54-x arranged on the steering wheel 52 is considered individually. The input variable 11 or the actuation signal 15 then additionally includes information that uniquely identifies the actuated control element 54-x.

[0033] It may further be provided that at least a subset of the control elements 54-x arranged on the steering wheel 52 is considered grouped by class. For example, it may be provided that the control elements 54-1, 54-4 arranged on the outside are considered in one class and the control elements 54-2, 54-3 arranged on the inside are considered in another class. The respective control elements 54-x are then no longer considered individually, but only according to their respective class.

[0034] It can be provided that, upon detecting an actuation of the at least one control element 54-x, a test signal 30 is impressed on the steering wheel 52, wherein a counter-torque 12 caused thereby is detected and taken into account in the decision 20. The impression is effected by means of an actuator (not shown) of the steering system 51 in a manner known per se. The counter-torque 12 can be detected, for example, by means of a torque sensor (not shown) of the steering system 51.

[0035] It can be provided that a type of the actuated at least one operating element 54-x is taken into account. For example, the operating elements 54-1, 54-2 can be designed as mechanical buttons, while the operating elements 54-3 and 54-4 can be designed as capacitive buttons. This difference is then fed to the trained machine learning method 3 as additional information or input data. The type can either be encoded as information in the input variable 11, or the type is stored individually or in groups for each operating element 54-x in the memory of the data processing device 2 and is determined based on information that uniquely identifies the respective operating element 54-x individually or in groups. When training the machine learning method 3, the type of the operating element 54-x is taken into account accordingly.

[0036] It can be provided that a position of the at least one operating element 54-x on the steering wheel 52 and / or a distance of the actuated at least one operating element 54-x from a rotation axis 56 of the steering wheel 52 is taken into account. Either the position and / or the distance can be encoded as information in the input variable 11, or the position and / or the distance is stored individually or in groups for each operating element 54-x in the memory of the data processing device 2 and is determined based on information that uniquely identifies the respective actuating element 54-x individually or in groups.

[0037] It can be provided that, based on the actuated at least one operating element 54-x, a correction factor k is selected and taken into account. The correction factor k can be determined, for example, based on information that uniquely identifies the respective actuating element 54-x individually or in groups. Alternatively, the correction factor k can be determined, for example, based on a respective position and / or a respective distance of the actuated at least one actuating element 54-x. It can be provided, for example, that the detected counter-torque 12 is corrected by means of the operating element-dependent correction factor k. The correction factor k is, in particular, a function of the position and / or the distance.

[0038] It can be provided that, when considering the actuated at least one control element 54-x, a driver profile 13 of a driver is also taken into account. In particular, it can be provided that the correction factor k is selected as a function of the driver profile 13 or that information about the driver profile 13 is used directly as input to the trained machine learning method 3. The driver profile 13 can, for example, be selected by a driver. Alternatively, the driver profile 13 can also be provided by a vehicle controller 57.

[0039] It can be provided that a driver is detected by means of at least one driver camera 58, wherein in addition at least one detected Fig.which is evaluated and taken into account by at least one driver camera 58. The evaluation can, for example, include a gaze direction detection and / or a recognition of a driver's intention, which is taken into account by the trained machine learning method 3 as part of the decision-making process or application of the trained machine learning method 3 and is supplied to the trained machine learning method 3 for this purpose as input data. List of reference symbols 1 device 2 Data processing facility 3 trained machine learning algorithm 10 Status data 11 Input variable 12 Counter torque 13 Driver profile 14 captured image 15 Actuation signal 20 Decision 21 Decision signal 30 test signal 50 vehicles 51 steering system 52 steering wheel 53 Steering control 54-x control element 55 Driver assistance system 56 axis of rotation 57 Vehicle control 58 Driver camera k Correction factor

Claims

[1] Method for detecting contact of hands with a steering wheel (52) of a vehicle (50), wherein, based on at least detected and / or received status data (10) of a steering system (51) of the vehicle (50), a decision (20) is made by means of at least one trained machine learning method (3) as to whether or not at least one hand is in contact with the steering wheel (52), wherein an actuation of at least one control element (54-x) arranged on the steering wheel (52) is taken into account as an input variable (11) of the at least one trained machine learning method (3), and wherein a decision signal (21) is generated and provided. [2] Method according to claim 1, characterized by that at least a subset of the control elements (54-x) arranged on the steering wheel (52) is taken into account individually. [3] Method according to claim 1 or 2, characterized bythat at least a subset of the operating elements (54-x) arranged on the steering wheel (52) is taken into account grouped by classes. [4] Method according to one of the preceding claims, characterized by that upon detection of an actuation of the at least one control element (54-x), a test signal (30) is impressed on the steering wheel (52), wherein a counter-torque (12) caused thereby is detected and taken into account in the decision (20). [5] Method according to one of the preceding claims, characterized by that a type of actuated at least one control element (54-x) is taken into account. [6] Method according to one of the preceding claims, characterized by that a position of the at least one operating element (54-x) on the steering wheel (52) and / or a distance of the actuated at least one operating element (54-x) from a rotation axis (56) of the steering wheel (56) is taken into account. [7] Method according to one of the preceding claims, characterized by that a correction factor (k) is taken into account based on the actuated at least one control element (54-x). [8] Method according to one of the preceding claims, characterized by that when taking into account the actuated at least one control element (54-x), a driver profile (13) of a driver is also taken into account. [9] Method according to one of the preceding claims, characterized by that a driver is detected by means of at least one driver camera (58), wherein additionally at least one detected image (14) of the at least one driver camera (58) is evaluated and taken into account. [10] Device (1) for detecting contact of hands with a steering wheel (52) of a vehicle (50), comprising: a data processing device (2), wherein the data processing device (2) is configured to receive detected and / or queried state data (10) of a steering system (51) of the vehicle (50) and to provide at least one trained machine learning method (3) which, based on at least the received state data (10), makes a decision (20) as to whether or not at least one hand is in contact with the steering wheel (52), wherein an actuation of at least one control element (54-x) arranged on the steering wheel (52) is taken into account as an input variable (11) of the at least one trained machine learning method (3), and to provide a decision signal (21).

Images