Method and system for identifying a grip position on a steering wheel in a vehicle

Abstract

This invention relates to a method for identifying grip positions on a steering wheel (1) in a vehicle. In this method, sensor data is captured by means of at least one sensor device (2) in the steering wheel (1) of the vehicle, and the sensor data is used to determine potential grip positions on the steering wheel (1), wherein, for this purpose, it is determined from the sensor data whether at least one hand is detected in an area of ​​the steering wheel (1). According to the invention, measurement data is captured, representing a radar point cloud with a plurality of radar points obtained based on radar scans at least partially around a three-dimensional area of ​​the steering wheel (1). Potential grip positions determined from the sensor data are examined, wherein a potential grip position is determined using the measurement data and aligned with the potential grip positions determined from the sensor data.

Description

Technical Field

[0001] This invention relates to a method and system for identifying grip positions on a steering wheel in a vehicle. In particular, it is intended to identify illusions of grip positions on the steering wheel. Background Technology

[0002] Modern vehicles can be equipped with driving systems for autonomous or partially autonomous driving. Even when the vehicle (including steering) is fully or at least partially controlled automatically, responsibility, particularly regarding safety, generally remains with the driver. To ensure the driver is ready to take over control immediately, if the system requires, a specific grip position on the steering wheel can be specified, whereby the driver grips the steering wheel with at least one hand or both hands. Therefore, in vehicles, especially in partially autonomous driving systems, various methods exist for monitoring whether the driver's hands are on the steering wheel.

[0003] Systems based on capacitive sensing or electric field technology for identifying grip positions on a steering wheel are known. For this purpose, capacitive sensors are mounted in the steering wheel and detect whether the driver's hands are on it. These sensors use the principle of capacitance in an electric field as their basic principle. Human skin is conductive and changes the capacitance of the electric field when it approaches or touches the sensor. These sensors are integrated into the vehicle's steering wheel. They can be distributed along the circumference of the steering wheel to ensure they can detect contact at any point. Capacitive sensors are very sensitive, even detecting slight contact. Furthermore, they are mounted inside the steering wheel and are therefore protected from contamination and are invisible.

[0004] Other "hands-off / hands-on recognition systems" are based on torque sensors. Torque and angle sensors measure the position of the steering wheel and the torque applied to it by the driver. If the hands are not on the steering wheel, this can be identified by measuring the torque, specifically by detecting a lack of torque or excessively low torque on the steering wheel.

[0005] Both systems described above offer simple and cost-effective solutions for identifying grip positions on the steering wheel. However, both systems are easily fooled. Systems using capacitive sensors in the steering wheel are easily fooled, for example, by attaching an object that causes a capacitance change similar to that of a hand to the steering wheel. This could be as simple as a wet towel. Systems using torque sensors in the steering wheel can also be easily fooled. Here, an object can also be fastened to the steering wheel to generate torque. For this purpose, any desired weight is suitable, such as a water bottle, which is simply clamped between the spokes of the steering wheel, thus applying torque to the steering wheel. Summary of the Invention

[0006] The purpose of this invention is to improve the recognition of grip positions on the steering wheel in a vehicle. In particular, deception will become more difficult.

[0007] This objective is achieved according to the teachings of the independent claims. Various embodiments and modifications of the invention are the subject of the dependent claims.

[0008] A first aspect of the invention relates to a method for identifying a grip position on a steering wheel in a vehicle, particularly a computer-implemented method. In this method, sensor data is captured by at least one sensor device in the vehicle steering wheel, and a potential grip position on the steering wheel is determined based on the sensor data, wherein, for this purpose, the sensor data is used to determine whether at least one hand is detected in the steering wheel area. Measurement data representing a radar point cloud having a radar point set is captured, the radar point cloud being obtained based on radar scanning at least partially around a spatial region of the steering wheel. The potential grip position determined from the sensor data is examined, in which case, for this purpose, the potential grip position is determined based on the measurement data and compared with the potential grip position determined from the sensor data.

[0009] Therefore, the method described above according to the first aspect is specifically based on checking or verifying a potential gripping position on the steering wheel, which is determined by means of a sensor device in the steering wheel. For this purpose, a potential gripping position is determined based on measurement data captured by a radar sensor device inside the vehicle. This check or verification can improve the identification of the gripping position. In particular, it can make it more difficult to spoof the gripping position. As mentioned at the beginning, sensors, such as capacitive sensors, are easily fooled. By means of radar scanning, it can then be checked whether the sensor data actually originates from a predefined gripping position or from deception.

[0010] Using radar measurement data to examine sensor data from sensor units in the steering wheel (“Hand Grip Detection Sensor” or “HOD Sensor”) can be easily implemented in many vehicles, especially if in-cabin monitoring of the radar sensors is already provided for other purposes. In-cabin radar sensors are increasingly used in vehicles for functions such as seatbelt reminders, seat occupancy detection, and child / life detection. The same sensors can also be used to monitor the driver’s hands / arms to assist in hand detection or identification of grip position on the steering wheel. If a driver uses deception to trick the HOD sensor to keep their hands free for other activities, this can be detected by the in-cabin radar sensors.

[0011] The term "grip position" as used herein specifically refers to the position of the driver's hands on the vehicle's steering wheel. In this context, grip position particularly describes whether at least one of the driver's hands (right or left) or both hands are on the steering wheel, i.e., whether he is gripping the steering wheel. Furthermore, grip position may also indicate how and / or where the driver grips the steering wheel with one or both hands. For safety reasons, grip positions defined in the context of (partial) autonomous driving may include gripping the steering wheel with at least one hand, and possibly with both hands. According to the specification, a predetermined grip position may also include a manner of gripping the steering wheel with both hands, for example, with the left hand in the upper left quadrant of the steering wheel and the right hand correspondingly in the upper right quadrant. In the context of this invention, determining a grip position also includes determining whether an actual grip position has not occurred, i.e., in the specific case where the driver has completely released the steering wheel.

[0012] As used herein, the term "radar point cloud" should be understood, in particular, as a set of points in a vector space that is obtained through a radar scan of at least one object surface and has a generally unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points in the radar point cloud can be referred to as "radar points." A (radar) point cloud can be described, in particular, by the (radar) points it contains. Each radar point can, in turn, be described, specifically by its spatial coordinates, which indicate for each radar point the location at the object surface where the emitted radar signal, measured during a radar scan, is reflected. In addition to radar points, properties such as measured Doppler velocity or signal-to-noise ratio (SNR) can also be captured.

[0013] The term "vehicle" as used herein specifically refers to automobiles, including all types of motor vehicles, hybrid and battery-powered electric vehicles, as well as vehicles such as sedans, vans, buses, trucks, delivery vans, etc. Needless to say, the term "steering wheel" can also be understood as any other control device operated with one or two hands for guiding the vehicle, particularly for steering it.

[0014] The terms “comprising,” “containing,” “including,” “displaying,” “having,” “with,” or any other variation thereof, as may be used herein, are intended to cover non-exclusive inclusion. As an example, a method or apparatus that includes or has a list of elements is therefore not necessarily limited to those elements, but may include other elements not expressly listed or inherent in such a method or apparatus.

[0015] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive "or," not an exclusive "or." For example, condition A or B is satisfied by one of the following conditions: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0016] The term “one” or “an” as used herein is defined in the sense of “one or more”. The terms “another” and “another” and any other variations thereof should be understood to mean “at least one other”.

[0017] The terms “multiple” or “multi-multiple” used here should be understood as meaning “two or more”.

[0018] The terms "configuration" or "setting" to perform a particular function (and its respective variations) should be understood in the context of this invention to mean that the corresponding device already exists in a configuration or setting capable of performing that function, or that the device is at least adjustable—i.e., configurable—so that it can perform the function after appropriate adjustments. For example, configuration can be achieved by appropriately setting parameters of a process sequence or by using switches to activate or deactivate a function or setting. In particular, the device may have multiple predetermined configurations or operating modes, such that configuration can be performed by selecting one of these configurations or operating modes.

[0019] Preferred embodiments of the method will now be described. Unless explicitly excluded or technically impossible, each preferred embodiment may be combined with one another as needed and with other aspects of the invention as further described herein.

[0020] In some embodiments, at least one radar point cluster is formed from a set of radar points, wherein the radar points in the at least one cluster are used to determine potential grip positions from measurement data. Clustering improves the identification of grip positions, i.e., particularly the identification of hands and arms. In principle, a radar point cloud obtained through vehicle interior radar scanning can be generated for any desired area of ​​the vehicle occupant's (especially the driver's) body or the vehicle, where the area around the steering wheel is important for identifying grip positions. Further delimitation can then be achieved through clustering. For example, points outside the clusters can thus be ignored for further processing.

[0021] In some relevant embodiments, at least one cluster of radar points is classified to determine potential grip locations from measurement data. Specifically, one or more clusters can be classified based on shape, size, arrangement, etc. For this purpose, machine learning-based algorithms, particularly those with corresponding machine learning models, can be used. Classification then allows it to determine whether and where a hand might be gripping the steering wheel, or whether it is, for example, another object in the steering wheel area. Classification also supports examining the grip locations determined from sensor data.

[0022] In some embodiments, movable radar points are determined from the radar point cloud as measurement data. This reduces the set of radar points to be processed. Specifically, immovable points can be ignored because they are more likely to represent immovable components, such as parts inside a vehicle, while movable radar points are more likely to indicate movable objects, such as people. Therefore, for hand and arm identification, processing only movable radar points is sufficient.

[0023] In some embodiments, a check is performed only if the sensor data determines a grip position corresponding to a predetermined grip position as a potential grip position. A predetermined grip position is a grip position that can be defined for safety reasons. In other words, if a “valid” predetermined grip position cannot be determined from the sensor data, for example because the driver is not placing their hands on the steering wheel, then a check using radar measurement data is not necessary. On the other hand, if a valid predetermined grip position is determined from the sensor data, a check is performed to determine whether this is indeed the case or if deceptive manipulation has occurred.

[0024] In some embodiments, an alarm is also output if a comparison determines that the potential grip position determined from sensor data and the potential grip position determined from measurement data do not at least partially match. Specifically, a missing match can indicate an attempt to spoof. Specifically, a partially missing match can be determined if the measurement data determines a grip position with only one hand on the steering wheel or no hand on the steering wheel. Specifically, this causes a contradiction if the sensor data determines grip positions with both hands, meaning the verification is unsuccessful. The alarm can be implemented, for example, as an acoustic signal and / or as an optical signal (such as a visible display).

[0025] In some embodiments, an alert is also output if a potential grip position determined from sensor data matches a potential grip position determined from measurement data, but an object different from a hand is identified in the area of ​​the steering wheel from the measurement data. In this way, it is possible to identify whether the driver is actually not distracted, even if the driver assumes both hands (or at least one hand) are correctly placed on the steering wheel. For example, it is thus possible to identify whether the driver is holding, for example, a smartphone with one hand while the other hand is on the steering wheel. In particular, the classification of the clustering described above can be helpful here, for example, in distinguishing between the hand gripping the steering wheel and the hand holding the smartphone.

[0026] In some embodiments, sensor data is captured by means of at least one capacitive sensor device and / or a mechanical sensor device (particularly a torque or angle sensor) in the vehicle's steering wheel. Both systems can be easily deceived by faking the grip position (e.g., by means of an object attached to the steering wheel) using the methods described at the beginning. Evaluation of the radar measurement data then allows for a rapid determination of whether an effective grip position with one or both hands actually exists on the steering wheel.

[0027] A second aspect of the invention relates to a data processing system in a vehicle, comprising at least one processor configured to perform the method according to the first aspect; at least one sensor device in a vehicle steering wheel configured to determine the driver's grip position on the steering wheel; and at least one radar sensor device configured to capture measurement data representing a radar point cloud having a set of radar points, the radar point cloud being at least partially in a spatial region surrounding the steering wheel.

[0028] In some embodiments of the system, the sensor array includes at least one capacitive sensor and / or mechanical sensor in the vehicle's steering wheel. As mentioned above, these can be easily spoofed, making it advantageous to examine potential grip positions using corresponding radar measurement data to make spoofing more difficult.

[0029] In some embodiments of this system, the radar sensor device is configured to scan the interior of the vehicle to obtain measurement data representing an assigned radar point cloud, which has been obtained or is based on radar scans of a spatial area at least partially surrounding the seating arrangement. Therefore, the radar sensor device is not only used to check the grip position on the steering wheel but also configured for further interior space monitoring, such as identifying seat occupancy status. Driver monitoring can also be established using radar scanning. Thus, for example, the driver's direction of gaze can be monitored, which can be used to identify the driver's attention.

[0030] A third aspect of the invention relates to a computer program having instructions that, when executed on a system according to the second aspect, cause the system to perform the method according to the first aspect.

[0031] Computer programs can be stored, in particular, on non-volatile data carriers. This is preferably in the form of optical data carriers or flash memory modules. This can be advantageous if such a computer program is intended to be processed independently of the processor platform on which one or more programs are executed. In another embodiment, the computer program can exist as a file on a data processing unit, particularly a server, and can be downloaded via a data connection (e.g., the Internet or a dedicated data connection, such as a proprietary or local area network). Furthermore, the computer program can have multiple separate, interacting program modules.

[0032] The system according to the second aspect may accordingly have a program memory in which the computer program is stored. Alternatively, the system may also be configured to access externally available computer programs via a communication connection, such as on one or more servers or other data processing units, particularly for exchanging data used during the operation of the method or the computer program, or for the output constituting the computer program.

[0033] The features and advantages explained with respect to the first aspect of the invention also apply accordingly to the other aspects of the invention. Attached Figure Description

[0034] Other advantages, features and possible applications of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0035] In the attached diagram:

[0036] Figure 1 A system according to an embodiment of the present invention is illustrated schematically;

[0037] Figure 2 The radar point cloud on the steering wheel, the corresponding clusters, and the associated scenes are shown.

[0038] Figure 3 The radar point cloud on the steering wheel, the corresponding clusters, and the associated scenes are shown.

[0039] Figure 4 This shows the radar point cloud on the steering wheel, the corresponding clusters, and the associated scene; and

[0040] Figure 5 The diagram shows the radar point cloud on the steering wheel, the corresponding clusters, and the associated scenes.

[0041] In the accompanying drawings, the same reference numerals are always used for the same or corresponding elements of the invention. Detailed Implementation

[0042] Figure 1 A system for identifying grip positions on a steering wheel 1 is shown. Specifically, the grip position determined based on sensor data from a sensor device 2 in the steering wheel 1 is intended to be verified by means of measurement data captured in the form of a radar point cloud by a radar sensor device 3. Sensor device 2 (in...) Figure 1 The steering wheel 1 (shown in a simplified and schematic form) can be, for example, a capacitive sensor device that identifies the contact between the hand and the steering wheel 1. Mechanical sensor devices, such as torque sensors or angle sensors, can also be provided to identify the grip position based on the torque applied to and associated with the steering wheel 1.

[0043] In parallel with identifying the grip position via sensor device 2, it is now stipulated that the position of the driver's hand or arm be (permanently) tracked or monitored by a radar sensor device 3 located in the driver's cabin. The radar sensor device in the cabin provides radar point clouds of different body parts of the driver, particularly radar point clouds of the arm and hand in the area of ​​steering wheel 1. See below for reference. Figure 2 , 3 As explained in points 4 and 5, the identified radar points are clustered and classified, enabling the determination of the shape and position of the driver's hand and / or arm. Clustering, classification, and decision-making can be performed using machine learning or conventional signal processing. In particular, clusters can be created for moving points within the radar point cloud.

[0044] Figure 2 An example is shown where grip positions determined based on sensor data can be confirmed using radar measurements. Based on cluster 21 of radar point cloud 20, the grip positions of the driver's hands 4, 5 on steering wheel 1 can be confirmed. In this case, the driver's behavior is correct, therefore no warning signal is output.

[0045] Figure 3 An example is shown where the torque sensor in steering wheel 1 is tricked by an object 6 on steering wheel 1. The torque generated by the weight of object 6 (e.g., a water bottle, an apple, or other objects of that order of magnitude) has the effect of assuming the correct grip position based on sensor data. However, by evaluating the radar point cloud 30, particularly its clusters 31, it is possible to identify that the driver's hand is not present in the area of ​​steering wheel 1. After a period of time, an alarm is then output, designed to warn the driver to grip steering wheel 1.

[0046] Figure 4 Another example is shown where, although one of the driver's hands 4 is on the steering wheel 1, he is still distracted by the tablet computer 7 in his other hand. The evaluation of sensor data will first be based on the correct grip position of the hand 4 on the steering wheel. However, the tablet computer 7 can then be identified based on clusters 41 of the radar point cloud 40. In this case, the driver can also receive a warning. For this scenario, it may also be helpful if the driver's direction of observation can be identified using a radar sensor device.

[0047] exist Figure 5In the example shown, the capacitive sensor in steering wheel 1 is tricked by an object 8 attached to steering wheel 1. This could be, for example, a damp cloth to mimic the conductivity of a human hand. As a result of this trickery, the supposedly correct grip position is identified based on sensor data. However, inspection using radar measurement data (i.e., cluster 51 of radar point cloud 50) shows that the left hand is not on steering wheel 1. Furthermore, the driver is holding tablet 7 in his right hand, further distracting him. Due to the lack of a match between the grip position determined based on sensor data and the grip position determined based on radar measurement data, an alert is issued to the driver, such as... Figure 3 and Figure 4 As shown in the example.

[0048] While at least one exemplary embodiment has been described above, it should be noted that numerous variations exist in this regard. It should also be noted that the exemplary embodiments described are merely non-limiting examples and are not intended to limit the scope, applicability, or configuration of the apparatus and methods described herein. Rather, the foregoing description will provide guidance to those skilled in the art in implementing at least one exemplary embodiment, in which it will be understood that various changes can be made to the operational methods and arrangements of elements described in the exemplary embodiments without departing from the subject matter defined in the appended claims and their legal equivalents.

[0049] List of reference numerals

[0050] 1. Steering wheel

[0051] 2 Sensor Devices

[0052] 3. Radar sensor device

[0053] 4 hands

[0054] 5 hands

[0055] 6 objects

[0056] 7 tablets

[0057] 8 objects

Claims

1. A method for identifying the grip position on a steering wheel (1) in a vehicle, wherein, The method includes: - Sensor data is captured by means of at least one sensor device (2) in the steering wheel (1) of the vehicle; - Based on the sensor data, determine the potential grip position on the steering wheel (1), wherein, for this purpose, the sensor data is used to determine whether at least one hand is detected in the area of ​​the steering wheel (1); - Capture measurement data representing radar point clouds (20, 30, 40, 50) with radar point sets, the radar point clouds being obtained based on radar scans of at least a portion of the spatial region surrounding the steering wheel (1); - Examine the potential gripping position determined from the sensor data, in which case, for this purpose, the potential gripping position is determined based on measurement data, and the potential gripping position determined based on measurement data is compared with the potential gripping position determined from the sensor data.

2. The method of claim 1, wherein, At least one cluster (21, 31, 41, 51) of radar points is formed from the set of radar points, wherein the radar points in the at least one cluster (21, 31, 41, 51) are used to determine the potential gripping position based on the measurement data.

3. The method of claim 2, wherein, The at least one cluster (21, 31, 41, 51) of radar points is classified in order to determine potential gripping locations based on the measurement data.

4. The method according to any one of the preceding claims, wherein, Movable radar points are determined from the radar point cloud (20; 30; 40; 50) as the measurement data.

5. The method according to any one of the preceding claims, wherein, The check is performed only when a gripping position corresponding to a predetermined gripping position is determined from the sensor data as a potential gripping position.

6. The method according to any one of the preceding claims further comprises: - If the comparison determines that the potential gripping position determined from the sensor data and the potential gripping position determined from the measurement data do not match at least partially, an alarm is output.

7. The method of claim 6, wherein, If the measurement data determines that there is only one hand on the steering wheel (1) or no hand on the steering wheel (1), then at least a partially missing match is determined.

8. The method according to any one of claims 1 to 5, further comprising: - If the comparison determines that the potential grip position determined from the sensor data matches the potential grip position determined from the measurement data, but the measurement data identifies an object (6, 7, 8) that is different from a hand in the area of ​​the steering wheel (1), an alarm is output.

9. The method according to any of the preceding claims, wherein, The sensor data is captured by means of at least one capacitive sensor device and / or mechanical sensor device in the steering wheel (1) of the vehicle.

10. A data processing system in a vehicle, the data processing system comprising: At least one processor, the at least one processor being configured to perform the method according to any one of the preceding claims; and at least one sensor device (2) in the steering wheel (1) of the vehicle, the at least one sensor device being configured to determine the driver’s grip position on the steering wheel (1); And at least one radar sensor device (3), the at least one radar sensor device being configured to capture measurement data representing a radar point cloud (20; 30; 40; 50) having a radar point set, at least in a spatial region at least partially surrounding the steering wheel (1).

11. The system of claim 10, wherein, The sensor device (2) includes at least one capacitive sensor and / or mechanical sensor in the steering wheel (1) of the vehicle.

12. The system of claim 10 or 11, wherein, The radar sensor device (3) is configured to scan the interior of the vehicle to obtain measurement data representing an assigned radar point cloud (20, 30, 40, 50), which has been obtained or is based on radar scanning of a spatial area at least partially surrounding the seating arrangement.

13. A computer program having instructions that, when executed on a system as claimed in any one of claims 10 to 12, cause the system to perform the method as claimed in any one of claims 1 to 9.

Images