Method and device for detecting a hands-off state on a steering wheel

Abstract

The invention relates to a method for detecting a hands-off state (6) on a steering wheel (51), wherein a steering input (4) is detected on the steering wheel (51), the detected steering input (4) is supplied as input data to at least two trained machine learning models (5-x), the machine learning models (5-x) being trained to detect a hands-off state (6) based on at least the detected steering input (4) and outputting it as output data (20-x), the output data (20-x) being weighted, aggregated to represent a hands-off state (6), and the weighting values ​​(wx) being determined based on a current context (7-x). The invention further relates to a device (1) for detecting a hands-off state (6) on a steering wheel (51).

Description

[0001] The invention relates to a method and a device for detecting a hands-off state on a steering wheel. Furthermore, the invention relates to a steering system with such a device and a vehicle with such a steering system and / or with such a device.

[0002] Sensors such as a capacitive steering wheel are used in vehicles to monitor driver activity. Such a steering wheel detects whether the driver has touched or not touched the steering wheel ("hands-off") using a capacitive sensor. The result is transmitted to the functions being used, such as a longitudinal and / or lateral guidance assistance system. The contact of the hands on the steering wheel can be used to determine driver activity and the driver's level of attention. For example, the system can indicate to the driver to place their hands on the steering wheel if it is detected that their hands have not been on the steering wheel for a specified period of time during lateral guidance.

[0003] To save additional costs for a capacitive sensor in the steering wheel, it is known to monitor driver activity using artificial neural networks based on a torque (hand torque) detected on the steering wheel. Such a method is known, for example, from DE 10 2019 211 016 A1.

[0004] DE 10 2018 129 563 A1 discloses a method for determining the control mode of a steering wheel of a vehicle, wherein the control mode is a first control mode in which a driver controls the steering wheel, or wherein the control mode is a second control mode in which the driver does not control the steering wheel.

[0005] The invention is based on the object of improving a method and a device for detecting a hands-off state on a steering wheel.

[0006] This object is achieved according to the invention by a method having the features of the steering wheel. Furthermore, the invention relates to a steering system with such a device and to a vehicle with such a steering system and / or with such a device. Advantageous embodiments of the invention are set out in the subclaims.

[0007] In particular, a method for detecting a hands-off state on a steering wheel is provided, wherein a steering variable on the steering wheel is detected, wherein the detected steering variable is fed to at least two trained machine learning models as input data, wherein the machine learning models are trained to detect a hands-off state based on at least the detected steering variable and to output it as output data, wherein the output data are combined and provided in a weighted manner to form a hands-off state, and wherein values ​​of the weightings are determined based on a current context.

[0008] Furthermore, in particular, a device for detecting a hands-off state on a steering wheel is created, comprising a steering variable sensor configured to detect a steering variable on the steering wheel, a control device, wherein the control device is configured to receive the detected steering variable, provide at least two trained machine learning models, wherein the machine learning models are trained to detect a hands-off state based on at least the detected steering variable and to output it as output data, feed the detected steering variable to the trained machine learning models as input data, further summarize and provide the weighted output data to form a hands-off state, and determine values ​​of the weightings based on a current context.

[0009] The method and the device enable improved detection of a hands-off state. In particular, the method and the device enable improved detection of the hands-off state in different contexts. For this purpose, it is provided that the detected steering variable is fed to at least two trained machine learning models as input data, wherein the machine learning models are trained to detect a hands-off state based on at least the detected steering variable and to output it as output data. The at least two machine learning models are in particular different from one another, i.e., they have a different structure and / or were trained with different training data. At least some of the machine learning models comprise, in particular, machine learning models specialized (for specific contexts).The output data is weighted and summarized to form a hands-off state and provided. The weighting values ​​are determined based on a current context. The weightings are redefined for each time step. Therefore, a current context is captured and / or recognized during application, and the weightings are determined based on the captured and / or recognized current context.

[0010] A particular advantage of the method and device is that a hands-off state can be detected more effectively, since using multiple machine learning models allows several different domains or a larger domain overall to be covered. Furthermore, one or more of the at least two machine learning models can be specialized for specific contexts, thus improving the robustness and / or accuracy of detecting the hands-off state.

[0011] A steering variable is, in particular, a variable that represents and / or describes a current state of the steering wheel. A steering variable is, in particular, a torque, which is detected in particular by means of a torque sensor on the steering wheel. In principle, however, a steering variable can also be another variable detected directly or indirectly on the steering wheel. For example, it can be provided to detect a current at an electrical machine on the steering wheel and use it as a steering variable. The hands-off state can be detected exclusively on the basis of the steering variable detected on the steering wheel, in particular a detected torque. However, it is also possible, in particular, for the machine learning models to be provided with further (steering) variables that are detected on the steering wheel (e.g., a steering wheel angle and / or a steering wheel angular velocity, etc.).) and the trained machine learning models recognize the hands-off state taking this additional variable into account. Furthermore, variables that are not recorded at the steering wheel can also be taken into account, such as vehicle speed, lateral acceleration, yaw rate, wheel ticks, damper information, and / or other driving dynamics variables, etc. In particular, however, no capacitive sensor is provided on the steering wheel.

[0012] A hands-off state is, in particular, a state in which the driver is not touching the steering wheel. In particular, none of the driver's fingers are in contact with the steering wheel. Detecting the hands-off state can, in particular, include providing a hands-off state signal. This includes, for example, a hands-off probability or coded signals for the states "hands-off detected" and "hands-off not detected."

[0013] A context refers to or includes, in particular, properties of a situation in which the hands-off state is to be recognized. Examples of properties that can determine a context are: an outside temperature, an inside temperature, a steering wheel vibration, a load and / or weight of the vehicle, the presence of a trailer, characteristics (e.g., identity, gender, age, weight) of the driver, etc. The current context is recognized and / or determined, in particular, based on recorded sensor data. For example, it may be provided to query such sensor data via a vehicle's CAN bus and / or to receive it from sensors and / or a vehicle control system of the vehicle.

[0014] A machine learning model is designed, in particular, as a neural network and comprises, in particular, several inner layers. The machine learning model is, in particular, an artificial recurrent neural network that processes the input data X at each time point t. t processed and a hands-off probability y t in [0,1] outputs: y t = p(x t | x 0:t-1). In particular, the neural network has a so-called memory h, in which information from previous time steps is stored and which can be used for the output at the current time step. The output is further processed, for example, by filtering before the decreasing functions (e.g., a lateral guidance assistant) process it. In particular, it can be provided that, based on a comparison of the hands-off probability with a predetermined threshold value, a binary hands-off signal is provided (with the two states "hands-off detected" and "hands-off not detected").

[0015] During a training phase, the machine learning model is or was trained in particular using training data comprising pairs in which data of the steering variable, in particular torque data, are each paired with a hands-off state. The data of the steering variable, in particular the torque data, are in particular time series of the steering variable detected at the steering wheel, in particular time series of torques detected at the steering wheel. The training data is obtained in particular with the aid of test drives and / or in simulators. In principle, the provision of training data can be carried out in particular according to the method described in DE 10 2019 211 016 A1. Training is carried out in a manner known per se, in particular by means of supervised learning.

[0016] Parts of the device, in particular the control unit, can be implemented individually or collectively as a combination of hardware and software, for example as program code executed on a microcontroller or microprocessor. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA).

[0017] In particular, it is intended that the results of at least two machine learning models be summarized in the form of a weighted sum. This can be expressed mathematically as: HODt=∑iθit∗yit∑iθit where HOD t is the summarized value for the hands-off state estimated at time t; θ it is the weight of the machine learning model i at time t and y itis the value estimated by the machine learning models for the hands-off state estimated at time t (e.g., a hands-off probability).

[0018] In one embodiment, it is provided that the values ​​of the weightings are changed continuously within a predefined transition time when the context changes. This results in a smooth transition of the weightings, so that the respective influence of the machine learning models on a final result does not change abruptly, but is continuously adjusted from a previous value to the new value within the predefined transition time. This can increase recognition quality and driving comfort. The predefined transition time can, for example, be one to several seconds. It can be provided that the transition time is predefined and / or determined taking the current context into account. This allows the speed at which the weightings are changed to be flexibly adapted to different contexts or situations. This can enable a change in the weightings in selected situations (e.g.in dangerous situations) can be carried out more quickly.

[0019] In one embodiment, it is provided that one of the at least two trained machine learning models is a general standard model and at least one other of the at least two trained machine learning models is a machine learning model trained for a specific context. This makes it possible to provide a machine learning model for each context. In particular, it can be provided that the general standard model receives a greater weighting if no specialized machine learning model is available for a current context. The general standard model is, in particular, a machine learning model trained in a multitude of possible contexts. This ensures reliable detection of the hands-off state, even if no machine learning model specialized for a current context is available.In other words, the general standard model forms a kind of fallback model, which is given greater weight whenever no specialized machine learning model is available.

[0020] In one embodiment, at least some of the weightings are determined based on at least one parameter of the current context using an assignment table (which can also be referred to as a look-up table). This allows the weightings to be determined without significant computational effort. The assignment table includes, in particular, an assignment of at least one parameter of a context to at least some of the weightings, i.e., a value or value range of the at least one parameter is assigned in the assignment table to precisely one combination of weightings that is used for precisely this value or value range.

[0021] In one embodiment, it is provided that at least one of the at least two machine learning models is assigned at least one operating parameter and / or operating parameter range, wherein the weighting of at least one machine learning model is determined taking into account a distance of the operating parameter and / or operating parameter range from a corresponding parameter of the current context. This allows an ideal operating range of the machine learning model under consideration to be taken into account. In other words, it is provided that the result of the machine learning model is weighted more heavily the closer the current context is to the at least one operating parameter and / or operating parameter range. For example, it can be provided that the machine learning model has the highest quality at a predetermined temperature or within a predetermined temperature range, e.g.in the range above 0°C or in a range around 20°C, etc. The weighting is then set, for example, inversely proportional to the distance from the specified temperature or temperature range, with the weighting being greatest when the specified temperature or temperature range is present. Other examples of operating parameters are: vehicle speed, curve radius, lateral acceleration, or other driving dynamics variables, etc.

[0022] In one embodiment, at least one additional machine learning model, trained to recognize a hands-off state based on at least the detected steering variable and to output it as output data, is activated and weighted, or deactivated depending on the context. This allows a machine learning model to be completely deactivated, so that an associated result is not considered at all. In particular, it is provided that deactivation includes the deactivated machine learning model not being executed at all or the associated computing operations not being executed. This allows computing power and storage space to be saved.

[0023] In one embodiment, the weighting of at least one of the machine learning models is determined based on the current context using a trained third machine learning model. This allows the weightings to be determined or estimated for each context. The third machine learning model was trained, in particular, during a training phase using training data consisting of pairs in which at least one parameter of a context is paired with a value for the weighting. If the weightings of several machine learning models are estimated, the pairs comprise the respective combinations of the weightings. Training can be carried out in a manner known per se, for example by means of supervised learning.

[0024] Further features of the device design will become apparent from the description of embodiments of the method. The advantages of the device are the same as those of the embodiments of the method.

[0025] Furthermore, a steering system is also provided, comprising a device according to one of the described embodiments.

[0026] Furthermore, in particular, a vehicle is also created, comprising a steering system and / or a device according to one of the described embodiments.

[0027] The invention will be explained in more detail below using preferred embodiments with reference to the figures. Fig. 1 is a schematic diagram illustrating an embodiment of the device for detecting a hands-off state on a steering wheel; Fig. 2 a schematic representation to illustrate an embodiment; Fig. 3 a schematic representation to illustrate an embodiment; Fig. 4 a schematic representation to illustrate an embodiment; Fig. 5 a schematic flow diagram to illustrate an embodiment of the method.

[0028] The Fig. 1 shows a schematic representation of an embodiment of the device 1 for detecting a hands-off state 6 on a steering wheel 51. The device 1 is arranged in particular in a vehicle 50. The method described in this disclosure is illustrated and explained in more detail below using the device 1.

[0029] The device 1 comprises a steering variable sensor 2 and a control device 3. The steering variable sensor 2 is configured to detect a steering variable 4 at the steering wheel 51 of the vehicle 50. The steering variable sensor 2 is, for example, a torque sensor, and the steering variable 4 is a torque.

[0030] The control device 3 comprises a computing device 3-1 and a memory 3-2. The computing device 3-1 is configured to perform the computing operations necessary for implementing measures of the method and can access data stored in the memory 3-2 for this purpose.

[0031] The control device 3 is configured to receive the detected steering variable 4, at least two trained machine learning models 5-x ( Fig. 2), wherein the machine learning models 5-x are trained to recognize a hands-off state 6 based on at least the detected steering variable 4 and to output it as output data 20-x, to supply the detected steering variable 4 to the trained machine learning models 5-x as input data, further to summarize and provide the weighted output data 20-x to a hands-off state 6, and to calculate values ​​of the weightings wx based on a current context 7-x ( Fig. 1) to be determined.

[0032] The hands-off state 6 is fed, for example, as a state signal to a control unit 52 of the vehicle 50 for further processing. The control unit 52 can be, for example, a lateral guidance assistant or another assistance system. The state signal can, for example, be a hands-off probability or a binary state value with the two states "hands-off detected" and "hands-off not detected."

[0033] For example, it can be provided that the weightings wx are determined based on values ​​of at least one parameter of the detected and / or recognized context 7-x. This is done in particular by means of the control device 3, which, for example, determines the weightings wx for each context 7-x or for values ​​of the at least one parameter using an assignment table 8 stored in the memory 3-2, which contains an assignment of contexts 7-x to weightings wx.

[0034] It can be provided that the values ​​of the weights wx are changed continuously within a given transition time T when the context changes. This is shown schematically in the Fig. 3. An example is shown in the Fig. 3 shows the temporal development of the weightings wx of two machine learning models 5-x over the time axis t. At the beginning, the context 7-1 is present, for which, for example, one machine learning model 5-1 is specialized. The weighting w-1 belonging to this machine learning model 7-1 therefore has the value 1, while the weighting w-2 of the other machine learning model 7-2 has the value zero. Somewhat later, the context 7-2 is then determined, for which, for example, the other machine learning model 5-2 is specialized. Therefore, after the context 7-2 is determined, the weighting w-1 is reduced from 1 to zero within the specified transition time T, and at the same time the weighting w-2 is increased from zero to 1. Even later, the context 7-1 is determined again. Accordingly, after the context 7-1 is determined, the weighting w-2 is reduced from 1 to zero within the transition time T, and the weighting w-1 is increased again from zero to 1.It may be provided that the transition time T is specified and / or determined taking into account the current context 7-x.

[0035] The Fig. Figure 4 shows a schematic diagram illustrating the change in the weights wx for three machine learning models 5-x. The weights wx are, in particular, normalized to the value 1, as shown in the mathematical formula in the general description. In the example shown, the machine learning model 5-3 is considered in both contexts 7-1 and 7-2 with a constant value for the weight w-3.

[0036] It can be provided that one of the at least two trained machine learning models 5-x is a general standard model and at least one other of the at least two trained machine learning models 5-x is a machine learning model 5-x trained for a specific context 7-x. With reference to the Fig. 3, for example, it can be provided that the machine learning model 5-1 is the general standard model. The machine learning model 5-2, on the other hand, is specialized for the context 7-2, for example, if steering wheel vibrations were detected. With reference to the Fig. 4, for example, it can be provided that the machine learning model 5-1 is the general standard model. The machine learning model 5-2, on the other hand, is specialized for the context 7-2, for example, when steering wheel vibrations are detected. The machine learning model 5-3, on the other hand, takes a temperature or a temperature influence into account when detecting the hands-off state 6.

[0037] It can be provided that the determination of at least some of the weightings wx is carried out on the basis of at least one parameter of the current context 7-x by means of an assignment table 8 ( Fig. 1). The allocation table 8 is stored, for example, in the memory 3-2 of the control device 3.

[0038] It can be provided that at least one operating parameter and / or operating parameter range is assigned to at least one of the at least two machine learning models 5-x, wherein the weighting wx of at least one machine learning model 5-x is determined taking into account a distance of the operating parameter and / or operating parameter range from a corresponding parameter of the current context 7-x. For example, it can be provided that the operating parameter and / or the operating parameter range relates to a temperature or a vehicle speed. The weighting wx can then, for example, be selected to be inversely proportional to the distance, i.e., the weighting wx has, in particular, the value of 1 when the distance is equal to zero, wherein the value gradually decreases to zero (or another value) the greater the distance becomes.

[0039] It can be provided that at least one further machine learning model 5-x, which is trained to recognize a hands-off state 6 based on at least the detected steering variable 4 and to output it as output data 20-x, is activated and weighted or deactivated depending on the context. Deactivation includes, in particular, that no further computing operations are performed for the deactivated machine learning model 5-x. It can also be provided that memory areas occupied by the deactivated machine learning model are released. In particular, it can be provided to use a plurality of trained machine learning models 5-x, which are activated or deactivated depending on the current context 7-x.

[0040] It can be provided that the weighting wx of at least one of the machine learning models 5-x is determined based on the current context 7-x using a trained third machine learning model 9. For this purpose, the control device 3, in particular, provides the third machine learning model 9. It can be provided, in particular, that the weightings wx of all machine learning models 5-x are determined based on the current context 7-x using the trained third machine learning model 9.

[0041] The Fig. 5 shows a schematic flow diagram to illustrate an embodiment of the method for detecting a hands-off state on a steering wheel.

[0042] In a measure 100, a steering variable, for example a torque, is detected on a steering wheel by means of a steering variable sensor, for example a torque sensor.

[0043] In a measure 102, the detected steering variable is fed to at least two trained machine learning models as input data, wherein the machine learning models are trained to recognize a hands-off state based on at least the detected steering variable and to output it as output data, wherein the output data are combined in a weighted manner to form a hands-off state and provided.

[0044] In an upstream measure 101, values ​​of the weightings are determined based on a current context.

[0045] Subsequently, measures 100 to 102 are repeated in particular. List of reference symbols 1 device 2 steering size sensor 3 Control device 3-1 Calculation device 3-2 Memory 4 Steering size 5-x machine learning model 6 Hands-off state 7-x Context 8 Assignment table 9 third machine learning model 20-x output data 50 vehicles 51 Steering wheel 52 Control unit 60 Steering system 100-102 Measures of the procedure t Timeline T Transition period wx weighting

Claims

[1] Method for detecting a hands-off state (6) on a steering wheel (51), wherein a steering variable (4) is detected on the steering wheel (51), wherein the detected steering variable (4) is fed to at least two trained machine learning models (5-x) as input data, wherein the machine learning models (5-x) are trained to recognize a hands-off state (6) based on at least the detected steering variable (4) and to output it as output data (20-x), wherein the output data (20-x) are weighted and summarized to a hands-off state (6) and provided, and where values ​​of the weights (wx) are determined based on a current context (7-x). [2] Method according to claim 1, characterized by that the values ​​of the weights (7-x) are changed continuously within a given transition time (T) when the context changes. [3] Method according to one of the preceding claims, characterized bythat one of the at least two trained machine learning models (5-x) is a general standard model and at least one other of the at least two trained machine learning models (5-x) is a machine learning model (5-x) trained for a specific context (7-x). [4] Method according to one of the preceding claims, characterized by that at least some of the weightings (wx) are determined based on at least one parameter of the current context (7-x) by means of an assignment table (8). [5] Method according to one of the preceding claims, characterized bythat at least one operating parameter and / or operating parameter range is assigned to at least one of the at least two machine learning models (5-x), wherein the weighting (wx) of at least one machine learning model (5-x) is determined taking into account a distance of the operating parameter and / or operating parameter range to a parameter of the current context (7-x) corresponding thereto. [6] Method according to one of the preceding claims, characterized by that at least one further machine learning model (5-x), which is trained to recognize a hands-off state (6) based on at least the detected steering variable (4) and to output it as output data (20-x), is activated and weighted or deactivated depending on the context. [7] Method according to one of the preceding claims, characterized bythat the weighting (wx) of at least one of the machine learning models (5-x) is determined based on the current context (7-x) by means of a trained third machine learning model (9). [8] Device (1) for detecting a hands-off state (6) on a steering wheel (51), comprising: a steering variable sensor (2) arranged to detect a steering variable (4) on the steering wheel (51), a control device (3), wherein the control device (3) is configured to receive the detected steering variable (4), to provide at least two trained machine learning models (5-x), wherein the machine learning models (5-x) are trained to recognize a hands-off state (6) based on at least the detected steering variable (4) and to output it as output data (20-x), to feed the detected steering variable (4) to the trained machine learning models (5-x) as input data, to further summarize and provide the weighted output data (20-x) to form a hands-off state (6), and to determine values ​​of the weightings (wx) based on a current context (7-x). [9] Steering system (60) comprising a device (1) according to claim 8. [10] Vehicle (50) comprising a steering system (60) according to claim 9 and / or a device (1) according to claim 8.

Images