Method and system for identifying a grip position on a steering wheel in a vehicle

Abstract

The invention relates to a method for identifying a grip position on a steering wheel (1) in a vehicle. In the method, sensor data are captured by means of at least one sensor device (2) in the steering wheel (1) of the vehicle, and a potential grip position on the steering wheel (1) is determined using the sensor data, wherein, for this purpose, it is determined from the sensor data whether at least one hand is detected in the region of the steering wheel (1). According to the invention, measurement data are captured which represent a radar point cloud having a number of radar points which are obtained on the basis of a radar scan of a three-dimensional region which at least partially surrounds the steering wheel (1). The potential grip position determined from the sensor data is checked, with a potential grip position being determined using the measurement data and being aligned with the potential grip position determined from the sensor data.

Description

[0001] Method and system for detecting a grip position of a steering wheel in a vehicle

[0002] The present invention relates to a method and a system for detecting a steering wheel grip position in a vehicle. In particular, a feigned steering wheel grip position is to be detected.

[0003] Modern vehicles can be equipped with driving systems for autonomous or semi-autonomous driving. Even if the control of the vehicle, including steering, is carried out completely or at least partially automatically by the vehicle, the responsibility, particularly with regard to safety-related aspects, often still lies with the driver. To ensure that the driver is ready to assume control immediately if the system requires it, it may be mandatory to maintain a steering wheel grip position in which the driver grasps the steering wheel with at least one or even both hands. In vehicles, especially with semi-autonomous driving systems, there are various methods for monitoring whether the driver has their hands on the steering wheel.

[0004] There are known systems for detecting a driver's grip on the steering wheel that are based on capacitive sensing or electric field technology. Capacitive sensors are built into the steering wheel and detect whether the driver has their hands on the steering wheel. These sensors use the principle of capacitance in an electric field as their basic principle. Human skin is conductive and changes the capacitance of an electric field when it comes close to or touches a sensor. These sensors are integrated into the steering wheel of a vehicle. They can be distributed around the circumference of the steering wheel to ensure they can detect a touch at any point. Capacitive sensors are very sensitive and can detect even light touches. They are also installed inside the steering wheel, protecting them from dirt and not visible.

[0005] Other "hands-off / on detection systems" are based on a torque sensor. Torque and angle sensors measure the steering wheel position and the torque exerted by the driver on the steering wheel. When the hands are not on the steering wheel, this can be detected by measuring the torque, particularly by detecting a lack of or insufficient torque on the steering wheel. Both of the aforementioned systems offer a simple and cost-effective solution for detecting a grip on the steering wheel. However, both systems can be easily fooled. Systems that use capacitive sensors in the steering wheel, for example, can be fooled simply by placing an object on the steering wheel that causes a capacitive change similar to a hand. This could be as simple as a wet washcloth. Systems that use torque sensors in the steering wheel can also be easily fooled.Here, too, an object can be attached to the steering wheel to generate torque. Any weight, such as a water bottle, is suitable for this purpose. It is simply clamped between the spokes of the steering wheel, thereby exerting torque on the steering wheel.

[0006] The present invention is based on the object of improving the recognition of a steering wheel grip in a vehicle. In particular, it is intended to make deception more difficult.

[0007] This object is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0008] A first aspect of the invention relates to a method, in particular a computer-implemented method, for detecting a grip position of a steering wheel in a vehicle. In the method, sensor data are acquired by means of at least one sensor device in the steering wheel of the vehicle, and a potential grip position of the steering wheel is determined based on the sensor data. For this purpose, it is determined from the sensor data whether at least one hand is detected in the area of ​​the steering wheel. Measurement data are acquired that represent a radar point cloud with a set of radar points, which is obtained based on a radar scan of a spatial area surrounding the steering wheel at least in part. The potential grip position determined from the sensor data is checked. For this purpose, a potential grip position is determined based on the measurement data and compared with the potential grip position determined from the sensor data.

[0009] The aforementioned method according to the first aspect is therefore based in particular on checking or verifying a potential grip position of a steering wheel, which was determined with the aid of a sensor device in the steering wheel. For this purpose, a potential grip position is determined based on measurement data recorded by a radar sensor device in the interior of the vehicle. This checking or verification can improve the recognition of a grip position. In particular, it is possible to make it more difficult to fake a grip position. As mentioned above, sensors such as capacitive sensors can easily be fooled. Using radar scanning, it can then be checked whether the sensor data actually originate from a predetermined grip position or from an illusion.

[0010] The use of radar measurement data to verify the sensor data from a sensor device in the steering wheel ("Hands-On Detection Sensor" or "HOD Sensor") can be easily implemented in many vehicles, especially if interior monitoring using radar sensors is already provided for other purposes. Radar sensors in the vehicle interior are increasingly being used in vehicles to enable functions such as seat belt reminders, seat occupancy, child presence / life detection, etc. The same sensor can also be used to monitor the driver's hands / arms to support hand detection or steering wheel grip detection. If the driver uses fraudulent means to fool the HOD sensor in order to keep their hands free for other activities, this can be detected by the radar sensor in the driver's cabin.

[0011] The term “grip position” used here refers in particular to the hand position of a vehicle’s steering wheel by the driver. The grip position describes in particular at least whether one hand (right hand or left hand) or both hands of the driver are on the steering wheel, i.e. whether the driver is gripping the steering wheel. The grip position can also indicate how and / or where the driver grips the steering wheel with one or both hands. A grip position that may be prescribed for safety reasons in the context of (semi-)autonomous driving can involve gripping the steering wheel with at least one hand, possibly with both hands. Depending on the regulation, the prescribed grip position can also include gripping the steering wheel with both hands, for example with the left hand in the upper left quadrant of the steering wheel and with the right hand in the upper right quadrant.In the sense of the present invention, determining the grip position also includes whether there is no actual grip position, i.e. in particular situations in which the driver completely lets go of the steering wheel.

[0012] The term "radar point cloud," as used herein, refers in particular to a set of points in a vector space obtained by radar scanning of at least one object surface, which has a typically unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points of the radar point cloud can be referred to as "radar points." A (radar) point cloud can be described in particular by the (radar) points it contains. The radar points, in turn, can each be described in particular by their spatial coordinates, which indicate, for each radar point, a location of the reflection of an emitted radar signal on an object surface, measured during the radar scanning. In addition to the radar points, attributes such as measured Doppler velocity or a signal-to-noise ratio (SNR) can also be recorded.

[0013] The term "vehicle" used here refers in particular to a passenger car, including all types of motor vehicles, hybrid and battery-powered electric vehicles, as well as vehicles such as sedans, vans, buses, trucks, delivery vans, and the like. It is understood that the term "steering wheel" can also be understood to include other control devices for guiding, in particular steering, a vehicle that are operated with one or both hands.

[0014] The terms "comprises," "includes," "includes," "has," "has," "with," or any other variation thereof, as used herein, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or has a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or that are inherent in such a method or apparatus.

[0015] Furthermore, unless explicitly stated to the contrary, "or" refers to an inclusive "or" and not an exclusive "or." For example, a condition A or B is satisfied by one of the following conditions: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

[0016] The terms "a" or "an" as used herein are defined as "one or more." The terms "another" and "another," and any other variations thereof, are defined as "at least one other."

[0017] The term "plurality" or "several", as used here, is to be understood in the sense of "two or more". The term "configured" or "set up" to fulfil a specific function (and respective modifications thereof) is to be understood, within the meaning of the invention, that the corresponding device is already in a design or setting in which it can carry out the function, or that it is at least adjustable - i.e. configurable - so that it can carry out the function after being set accordingly. The configuration can be carried out, for example, by appropriately setting parameters of a process sequence or of switches or the like for activating or deactivating functionalities or settings. In particular, the device can have a plurality of predetermined configurations or operating modes, so that the configuration can be carried out by selecting one of these configurations oroperating modes.

[0018] Preferred embodiments of the method are described below, which, unless expressly excluded or technically impossible, can be combined with each other as well as with the other aspects of the invention described.

[0019] In some embodiments, at least one cluster of radar points is formed from the set of radar points, wherein the radar points of the at least one cluster are used to determine the potential grip position from the measurement data. The formation of clusters improves the detection of a grip position, i.e. in particular the detection of hands and arms. A radar point cloud obtained by radar scanning of a vehicle interior can in principle be generated for any desired areas in the vehicle or the body of a vehicle occupant (in particular the driver), wherein the area around the steering wheel is of interest for the detection of a grip position. Cluster formation then allows for better delimitation. For example, points outside clusters can be ignored for further processing.

[0020] In some associated embodiments, the at least one cluster of radar points is classified in order to determine a potential grip position from the measurement data. In particular, the cluster(s) can be classified according to shape, size, arrangement, and the like. For this purpose, a machine learning-based algorithm can be used, in particular with a corresponding machine learning model. The classification can then be used to determine whether and where a hand is gripping the steering wheel or whether, for example, there is another object in the area of ​​the steering wheel. The classification further supports the verification of the grip position determined from the sensor data.

[0021] In some embodiments, moving radar points are determined from the radar point cloud as the measurement data. This allows the amount of radar points to be processed to be reduced. In particular, stationary points can be ignored, as these are more likely to refer to stationary parts, such as parts of the vehicle interior, whereas moving radar points are more likely to indicate a moving object, such as a person. Therefore, processing only moving radar points may be sufficient to detect hands and arms.

[0022] In some embodiments, the check is only performed if a potential grip position is determined from the sensor data that corresponds to a predefined grip position. A predefined grip position is a grip position that may be prescribed for safety reasons. In other words, if a "valid" prescribed grip position cannot be determined from the sensor data, for example, because the driver does not have a hand on the steering wheel, then a check using the radar measurement data is not necessary. If, however, a valid prescribed grip position is determined from the sensor data, a check is performed to determine whether this is actually the case or whether a deceptive maneuver is taking place.

[0023] In some embodiments, an alarm is furthermore issued if the comparison determines that the potential grip position determined from the sensor data and the potential grip position determined from the measurement data do not match at least partially. In particular, a lack of match can indicate an attempt at deception. An at least partial lack of match can be determined in particular if a grip position with only one hand on the steering wheel or no hand on the steering wheel is determined from the measurement data. In particular, if a grip position with two hands is determined from the sensor data, this leads to a contradiction, i.e. verification is not successful. An alarm can be issued, for example, as an acoustic signal and / or as a visual signal, such as a visible display.

[0024] In some embodiments, an alarm is further issued if the comparison determines that the potential grip position determined from the sensor data and the potential grip position determined from the measurement data match, but an object other than a hand is detected in the area of ​​the steering wheel from the measurement data. In this way, it can be detected whether the driver is actually not distracted, even though they supposedly have their hands (or at least one hand) correctly on the steering wheel. For example, it can be detected whether the driver is holding a smartphone in one hand while the other is on the steering wheel. In particular, the classification of clusters described above can be helpful here, for example, to distinguish between a hand that is gripping the steering wheel and a hand that is holding a smartphone.

[0025] In some embodiments, the sensor data is acquired using at least one capacitive sensor device and / or a mechanical sensor device (particularly torque or angle sensors) in the vehicle's steering wheel. These two systems can be easily tricked using the methods described above, for example, by simulating a grip position using an object attached to the steering wheel. By evaluating the radar measurement data, it can then be quickly determined whether a valid grip position with one or both hands on the steering wheel is actually present.

[0026] A second aspect of the invention relates to a system for data processing in a vehicle, comprising at least one processor which is configured to carry out the method according to the first aspect, as well as at least one sensor device in the steering wheel of the vehicle which is configured to determine a grip position of the steering wheel by the driver, as well as at least one radar sensor device which is configured to acquire measurement data which represent a radar point cloud with a set of radar points at least in a spatial region which at least partially surrounds the steering wheel.

[0027] In some embodiments of the system, the sensor device comprises at least one capacitive sensor and / or a mechanical sensor in the vehicle's steering wheel. These can be easily tricked as described above, so that verifying a potential grip position using corresponding radar measurement data is advantageous to make deception more difficult.

[0028] In some embodiments of the system, the radar sensor device is configured to scan an interior of the vehicle in order to obtain measurement data representing an associated radar point cloud, which was or is obtained based on a radar scan of a spatial area at least partially surrounding a seating arrangement. The radar sensor device is thus not only used to check the grip position on the steering wheel, but is also provided for further interior monitoring, for example, for detecting a seat occupancy status. It can also be provided to set up driver monitoring using radar scanning. For example, the driver's line of sight can be monitored, which can be used to detect the driver's attention.

[0029] A third aspect of the invention relates to a computer program comprising instructions which, when executed on a system according to the second aspect, cause the system to carry out the method according to the first aspect.

[0030] The computer program can, in particular, be stored on a non-volatile data carrier. This is preferably a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program as such is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can be present as a file on a data processing unit, in particular on a server, and can be downloadable via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can have a plurality of interacting individual program modules.

[0031] The system according to the second aspect can accordingly comprise a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example, available on one or more servers or other data processing units, via a communication connection, in particular to exchange data with it that is used during the execution of the method or computer program or that represents outputs of the computer program.

[0032] The features and advantages explained with respect to the first aspect of the invention also apply accordingly to the further aspects of the invention.

[0033] Further advantages, features, and possible applications of the present invention will become apparent from the following detailed description taken in conjunction with the drawings.

[0034] Fig. 1 schematically shows a system according to an embodiment of the invention;

[0035] Fig. 2 a radar point cloud, corresponding clusters and an associated scenario on the steering wheel;

[0036] Fig. 3 a radar point cloud, corresponding clusters and an associated scenario on the steering wheel;

[0037] Fig. 4 a radar point cloud, corresponding clusters and a corresponding scenario on the steering wheel; and

[0038] Fig. 5 a radar point cloud, corresponding clusters and an associated scenario on the steering wheel.

[0039] Throughout the figures, the same reference numerals are used for the same or corresponding elements of the invention.

[0040] Fig. 1 shows a system for detecting a grip position of a steering wheel 1. In particular, a grip position determined based on sensor data from a sensor device 2 in the steering wheel 1 is to be verified by measurement data acquired in the form of a radar point cloud by a radar sensor device 3. The sensor device 2 (simplified and schematically indicated in the steering wheel 1 in Fig. 1) can, for example, be a capacitive sensor device that detects contact of the hands with the steering wheel 1. A mechanical sensor device can also be provided, such as a torque sensor or an angle sensor, in order to detect a grip position based on an associated torque exerted on the steering wheel 1.

[0041] It is now provided, in parallel with the detection of a grip position by means of the sensor device 2, that the position of the driver's hands or arms is (permanently) tracked or monitored with the aid of the radar sensor device 3 in the cabin. The radar sensor device in the cabin supplies radar point clouds of various parts of the driver's body, in particular also the arms and hands in the area of ​​the steering wheel 1. As explained below with reference to Figs. 2, 3, 4 and 5, the detected radar points are clustered and classified so that the shape and position of the driver's hands and / or arms can be determined. Clustering, classification and decision-making can be carried out with the aid of machine learning or conventional signal processing. Clusters can be created in particular for moving points in the radar point cloud.

[0042] Fig. 2 shows an example in which the grip position determined from the sensor data can be confirmed by the radar measurement. Based on the clusters 21 of the radar point cloud 20, the grip position can be confirmed in which the driver has both hands 4, 5 on the steering wheel 1. In this case, the driver is behaving correctly, so no alarm signal is issued.

[0043] Fig. 3 illustrates an example in which a torque sensor in the steering wheel 1 is deceived by an object 6 on the steering wheel 1. The torque generated by the weight of the object 6 (for example, a water bottle, an apple, or another object of this size) causes a correct grip position to be assumed based on the sensor data. However, by evaluating the radar point cloud 30, in particular its cluster 31, it can be determined that no hand of the driver is present in the area of ​​the steering wheel 1. After a certain time, an alarm is then issued to alert the driver to grasp the steering wheel 1.

[0044] Fig. 4 illustrates another example in which the driver has one hand 4 on the steering wheel 1, but is nevertheless distracted by the tablet 7 in his other hand. The evaluation of the sensor data would initially detect a correct grip position due to the presence of one hand 4 on the steering wheel. However, the tablet 7 can then be identified based on the clusters 41 of the radar point cloud 40. The driver can also receive a warning in this case. It could also be helpful for this scenario if the radar sensor device could detect the driver's line of sight.

[0045] In the example shown in Fig. 5, a capacitive sensor in the steering wheel 1 is deceived by an object 8 attached to the steering wheel 1. This object can be a wet cloth, for example, to mimic the electrically conductive properties of a human hand. This deception results in a supposedly correct grip position being detected on the basis of the sensor data. However, a check using the radar measurement data, i.e. cluster 51 of radar point cloud 50, shows that the left hand is not on the steering wheel 1. In addition, the driver is holding a tablet 7 in his right hand and is therefore further distracted. Due to this lack of agreement between the grip position determined on the basis of the sensor data and the grip position determined on the basis of the radar measurement data, an alarm is issued to the driver, as in the examples from Fig. 3 and Fig. 4.

[0046] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are only non-limiting examples and are not intended to limit the scope, applicability, or configuration of the devices and methods described herein. Rather, the foregoing description will provide a guide to those skilled in the art for implementing at least one exemplary

[0047] embodiment, it being understood that various changes may be made in the operation and arrangement of the elements described in an exemplary embodiment without departing from the subject matter as defined in the appended claims, as well as their legal equivalents.

[0048] LIST OF REFERENCE SYMBOLS

[0049] 1 steering wheel

[0050] 2 Sensor device

[0051] 3 Radar sensor device 4 Hand

[0052] 5 hands

[0053] 6 Item

[0054] 7 Tablet

[0055] 8 Item

Claims

CLAIMS 1. A method for detecting a grip position of a steering wheel (1) in a vehicle, the method comprising: Acquiring sensor data by means of at least one sensor device (2) in the steering wheel (1) of the vehicle; Determining a potential grip position of the steering wheel (1) based on the sensor data, wherein for this purpose it is determined from the sensor data whether at least one hand is detected in the area of the steering wheel (1); Acquiring measurement data representing a radar point cloud (20; 30; 40; 50) with a set of radar points, which is obtained on the basis of a radar scan of a spatial area surrounding the steering wheel (1) at least in sections; Checking the potential grip posture determined from the sensor data, whereby a potential grip posture is determined from the measurement data and compared with the potential grip posture determined from the sensor data.

2. The method according to claim 1, wherein at least one cluster (21; 31; 41; 51) of radar points is formed from the set of radar points, wherein the radar points of the at least one cluster (21; 31; 41; 51) are used to determine the potential grip position from the measurement data.

3. The method according to claim 2, wherein the at least one cluster (21; 31; 41; 51) of radar points is classified to determine a potential grip posture from the measurement data.

4. Method according to one of the preceding claims, wherein moving radar points from the radar point cloud (20; 30; 40; 50) are determined as the measurement data.

5. Method according to one of the preceding claims, wherein the check is only carried out if a grip position which corresponds to a predetermined grip position is determined from the sensor data as a potential grip position.

6. The method according to any one of the preceding claims, further comprising: Issue an alarm if the comparison determines that the potential grip posture determined from the sensor data and the potential grip posture determined from the measurement data do not match at least partially.

7. The method according to claim 6, wherein an at least partial lack of agreement is determined when a grip position with only one hand on the steering wheel (1) or no hand on the steering wheel (1) is determined from the measurement data.

8. The method according to any one of claims 1 to 5, further comprising: Issuing an alarm if the comparison determines that the potential grip position determined from the sensor data and the potential grip position determined from the measurement data match, but an object (6, 7, 8) other than a hand is detected from the measurement data in the area of the steering wheel (1).

9. Method according to one of the preceding claims, wherein the sensor data are acquired by means of at least one capacitive sensor device and / or a mechanical sensor device in the steering wheel (1) of the vehicle.

10. System for data processing in a vehicle, comprising at least one processor which is configured to carry out the method according to one of the preceding claims, and at least one sensor device (2) in the steering wheel (1) of the vehicle, which is configured to determine a grip position of the steering wheel (1) by the driver, and at least one radar sensor device (3) which is configured to record measurement data which represent a radar point cloud (20; 30; 40; 50) with a set of radar points at least in a spatial region which at least partially surrounds the steering wheel (1).

11. System according to claim 10, wherein the sensor device (2) comprises at least one capacitive sensor and / or a mechanical sensor in the steering wheel (1) of the vehicle.

12. System according to claim 10 or 11, wherein the radar sensor device (3) is configured to scan an interior of the vehicle in order to obtain measurement data representing an associated radar point cloud (20; 30; 40; 50) which was or is obtained on the basis of a radar scan of a spatial region surrounding a seating arrangement at least in sections.

13. A computer program comprising instructions which, when executed on a system according to any one of claims 10 to 12, cause the system to carry out the method according to any one of claims 1 to 9.

Images