Method for providing a dead man's switch function by means of a touchscreen of an operating device, operating device and system

Abstract

Method for providing a dead man's switch function by means of a touchscreen (23) of an operating device (12), wherein the operating device (12) - at least one predetermined operating gesture (24) which is performed by means of an operating object (22) touching the touchscreen (23) is recognized and - depending on the operating gesture (24) detected, a control signal (17) is sent to a device (11) controlled by the operating device (12) and - the control signal (17) remains unchanged or is completely absent if the control object (22) on the touchscreen (23) is stationary, wherein the control object (22) is classified as stationary if the control object (22) changes its touch position from an initial touch position (XY) by less than a predetermined maximum change (36), wherein When the operating object (36) is stationary, a predetermined deadman condition is checked and if the deadman condition is not met, a predetermined abort signal (31) is output to the device (11), characterized by the fact that The dead man condition includes that a touchpad of the touchscreen (23) continues to generate a time-varying detection signal (25) which describes the touch position, but the dead man condition requires that the detection signal (25) which describes the touch position changes by more than zero but less than the maximum change (36) within a predetermined time interval.

Description

[0001] The invention relates to a method for providing a dead man's switch function by means of a touchpad (touch-sensitive operating surface) of a control device. A device, in particular a motor vehicle, can be operated by means of the control device. The invention also includes the control device and a system comprising the control device and the device operated by it.

[0002] Operating a device using a control device may require a technical measure to ensure that the user maintains their attention throughout operation. For example, if a motor vehicle is controlled by the device, the vehicle must be stopped if it is detected that the user has, for instance, removed their hand from the control device, because it must be assumed that the user is distracted. Such a function for detecting the presence of a user at a control device is called a dead man's switch function.

[0003] In connection with the remote control of a motor vehicle by means of a control device, it is known from DE 10 2013 012 394 A1 that a user must perform a continuous movement, for example a circular movement, with their finger on a touchscreen when operating the control device in order for a dead man's switch condition to be met. Continuously pressing on the control surface during operation is not sufficient.

[0004] Remote control of a motor vehicle to perform a parking maneuver is also possible using a smartphone or a tablet PC, as is known from DE 10 2014 108 482 A1.

[0005] A gesture recognition system for a touchscreen of an operating device is known from DE 10 2009 019 910 A1. This gesture recognition system distinguishes between a finger that is stationary on the touchscreen and a finger that is moved during a swipe gesture. For this purpose, the distance traveled by the finger from a starting point is measured.

[0006] Remote control of a vehicle with a trailer using a smartphone requires the specific functionality of assigning a steering angle, direction of travel, and speed to each touchpoint on the touchscreen. Positioning a fingertip on the touchscreen then controls the vehicle's longitudinal and lateral steering. Therefore, a continuous swiping or circular motion cannot be used as a dead man's switch function, as this would also affect the vehicle's longitudinal and lateral steering.

[0007] German patent DE 10 2014 011 864 A1 describes a method for monitoring a semi-autonomous driving process of a motor vehicle, wherein the deadman condition for terminating the driving process is a monitoring of an activity of the operator that is independent of conscious control of the driving process. This activity is, for example, the operator blinking or moving a remote control, the former being detected by means of a camera and the latter by means of a gyroscope or position sensor.

[0008] From US patent 2016 / 0188113 A1, a method is known for assisting a user in operating a touchscreen by smoothing the drawing trace made by the user with their finger on the touchscreen using a filter in order to smooth out a shaky motion, such as can occur when driving a motor vehicle.

[0009] The invention is based on the objective of using the touch-sensitive operating surface of an operating device for a touchscreen to receive operating gestures and providing a dead man's switch function even when the finger is stationary.

[0010] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are described by the dependent patent claims, the following description, and the figures.

[0011] The invention provides a method for providing a dead man's switch function by means of a touchpad in a control device. A touchpad is understood to be a control surface with touch sensitivity or a spatially resolved tactile field. The touchpad is a component of a touchscreen (touch-sensitive screen or touch display). The touch sensitivity can be provided, for example, in a known manner based on a matrix of capacitive proximity sensors. In other words, the touch detection is pixel-based or spatially resolved. The touchpad generates a detection signal that, for example, describes the coordinates of a touch point of a control object touching the touchpad or includes the signals of the individual proximity sensors. Such a control object can be, for example, a stylus or a finger. The detection signal can also be, for example,The detection signal describes the pressure exerted by the control object on the touchpad. It can also describe the area of ​​contact between the control object and the touchpad. Therefore, the detection signal can describe more than just the actual touch position. In the following, the relationship between the control object and the touchpad, as described by the detection signal, is referred to as the operating position.

[0012] The control device recognizes, in a known manner, at least one predetermined control gesture performed by a control object touching the touchpad. This gesture can be, for example, a swipe gesture or swipe movement—that is, a character trace or movement trajectory executed by the control object on the touchpad while maintaining continuous contact. Depending on the recognized control gesture, a control signal is sent to a device controlled by the control device. Thus, each change in the touch position on the touchpad modifies the control signal, as this executes a control gesture. Conversely, the control signal remains unchanged or is absent entirely if the control object is stationary on the touchpad.If, for example, a driving direction, speed, and steering angle are set on a motor vehicle using the operating gesture as described above, the current setting is retained if the control object on the touchpad is held still or stationary. In another case, the control signal can be sent whenever the operating gesture has been successfully recognized. Sending a control signal can be omitted during periods between two recognized operating gestures.

[0013] The point at which a control object is stationary is defined as follows for the control device. A control object is classified as stationary if, starting from an initial touch position, its position has changed by less than a predetermined maximum. For example, the user must move the control object a predetermined minimum distance from its initial touch position before it is recognized as moving. Until then, it is classified as stationary. The initial touch position can be the first touch position after a recognized operating gesture. The touch position can be described, for example, by coordinates. Thus, if a user keeps their finger still on the touchpad, this is classified as a stationary control object, and the control signal is either held unchanged or completely suspended.

[0014] To provide a dead man's switch function in this situation, the invention provides for checking a predetermined dead man's condition when the control object is stationary and outputting a predetermined abort signal to the device if the dead man's condition is not met, is unmet, or is violated. The dead man's condition thus determines when the device may continue to be operated. It requires that the contact position of the control object (contact coordinates and / or contact force and / or contact area) changes from the initial contact position by more than zero, but less than the maximum change, within a predetermined time interval. In other words, the dead man's condition requires that the touchpad continue to generate a time-varying detection signal that describes the contact position.This change in the detection signal must be so small that the described or signaled contact position changes by less than the maximum change. The maximum change can therefore also define a threshold value for the contact force and / or the contact area.

[0015] The invention offers the advantage that a user can hold the control object, such as their finger, motionless on the touchpad and still reliably distinguish between the touchpad being touched by the control object and a released or untouched touchpad. When the touchpad is untouched, the detection signal changes by time to zero, while when the control object is placed on or touching the touchpad, the touchpad always signals a change in position. The monitored deadman condition nevertheless allows any arbitrary movement of the control object to be used to manipulate the device's control signal. No special swipe gestures need to be reserved for the deadman's switch function.

[0016] The invention also includes optional further developments whose features provide additional advantages.

[0017] As previously described, the term "touch position" can encompass several aspects of touching the touchpad. The touchpad's detection signal can describe or indicate the point or position of the control object on the touchpad. In other words, the detection signal indicates the touch coordinates of the control object on the touchpad. Additionally or alternatively, the detection signal can also describe or indicate the pressure or force applied by the control object to the touchpad. The maximum change then describes a change in force or pressure. Additionally or alternatively, the detection signal can describe the area of ​​a touch surface simultaneously touched by the control object on the touchpad. For example, it can indicate the number of pixels or touch sensors covered or touched by the control object.The maximum change then describes a change in the area size. The described types of touch position (touch point, contact force, area size) are quantities that a user unconsciously or involuntarily alters through physiological tremor, even when holding a control object still on the touchpad. The underlying principle is that this tremor can be detected by the touchpad's sensors. Additionally or alternatively, sensor noise can occur when the described quantities are detected by the touchpad, which can also be used as an indication of a touch on a stationary control object.

[0018] In cases where the touch position includes or describes the touch position of the control object on the touchpad, it is preferably provided that the maximum change is defined as a position change within a range of 30 pixels to 2 pixels. The dimensions of a touch sensor correspond to one pixel. In other words, a movement tolerance of up to 30 pixels is permitted, within which the user can move the control object on the touchpad without altering the control signal or triggering the recognition of a user gesture. Alternatively, it can be provided that the maximum change is defined as a position change of one pixel on the touchpad. In other words, a standstill of the control object is only detected in the case of a sub-pixel movement, i.e., a movement of less than one pixel.Such sub-pixel resolution of a touch point can be achieved, for example, when the touchpad weights the signals from multiple touch sensors, thus enabling a coordinate specification with decimal places. This embodiment has the advantage that the operating device reacts very sensitively to a change in the touch position and adjusts the control signal accordingly.

[0019] To detect a touch on a stationary control object that is held so still that its touch coordinates on the touchpad do not change, an embodiment is provided in which noise is detected in the touchpad's detection signal. This noise is present in the detection signal even when the control object's touch coordinates remain constant. This noise can arise, for example, from simultaneous detection using multiple touch sensors whose sensor signals change over time and are combined to determine the touch coordinates. The deadman condition requires that the detection signal indicates the continued presence of the touch coordinates, while the noise exhibits a predetermined minimum power level, which is greater than zero. A minimum power level of zero would, conversely, indicate that the control object has detached from or been removed from the touchpad.

[0020] As previously explained, the object being operated can be a rod. However, the preferred method is to detect the user's finger, as also described, for example, the fingertip. This ensures that even when the object being operated is stationary—that is, when the finger is held still or stationary—the described change in touch position is reliably reflected in the touchpad's detection signal, since the physiological tremor of even a healthy user is reliably transmitted to the touchpad.

[0021] The described operating gesture can be recognized as a press or, preferably, a swipe gesture. Particularly in connection with the recognition of a swipe gesture, the advantage arises that any form of movement of the control object along the touchpad, i.e., any swipe gesture, can be used to generate or vary the control signal, since only a stationary control object is required for the dead man's switch function.

[0022] As previously explained, the deadman condition does not need to be continuously met; rather, the touch position must change by more than zero but less than the maximum change within a predetermined time interval. This time interval is preferably less than 0.5 seconds. This ensures the safe operation of, for example, a motor vehicle, as it receives the abort signal after this time. Alternatively, the time interval can be set to a range of 0.5 to 2 seconds. This allows the user to reposition their grip or change the position of the control object on the touchpad without triggering the abort signal.

[0023] With regard to the abort signal, it is intended that this will preferably bring the device to a complete stop. This prevents damage to the device. "Stopping" specifically refers to braking, coming to a standstill, or a complete halt of the device or a moving part of the device relative to its surroundings.

[0024] This method is particularly advantageous when the device is positioned at a distance from the control device and the control signal is transmitted to the device via a radio or optical connection. In this case, the control device is a remote control. In other words, the device operates independently while the user is at a distance from it with the control device. The device does not continue to operate independently when the user releases the control device from the touchpad.

[0025] The device can therefore be used, in particular, to remotely control a motor vehicle. In this context, it is especially preferred that the described control signal controls the longitudinal and / or lateral guidance of the motor vehicle during a parking maneuver. In this case, any operating gesture, especially any change of position on the touchpad, can be used by the control object to adjust the longitudinal and / or lateral guidance of the motor vehicle. Thus, by holding the control object still on the touchpad, a driving speed and / or a steering angle can be set and then maintained permanently. Even when holding the control object still on the touchpad, the dead man's switch function can be provided by the method according to the invention.

[0026] The invention also provides an operating device for controlling a device. The device has an output device for sending a control signal to the device. In particular, the operating device controls a motor vehicle. The output device can, for example, comprise a radio module that sends the control signal as a radio signal. The radio module can, for example, be a Bluetooth radio module, a WLAN radio module, or an infrared transmitter module. The operating device according to the invention also has a processor unit configured to carry out an embodiment of the method according to the invention. For this purpose, the processor unit can comprise at least one microcontroller and / or at least one microprocessor.Furthermore, the processor may contain program code configured to execute the embodiment of the method according to the invention when executed by the processor. The program code may be stored in a data memory of the processor.

[0027] The control device is preferably designed as a mobile device. A mobile device can be, for example, a smartphone, a smartwatch, or a tablet PC. Thus, a conventional mobile device can also be used to remotely control, for example, a motor vehicle.

[0028] The combination of the operating device according to the invention and a motor vehicle results in the system according to the invention, wherein the motor vehicle is configured to be remotely controlled from the outside by a control signal from the operating device, in particular with regard to the longitudinal and / or lateral guidance of the motor vehicle. The motor vehicle can thus, for example, perform a parking maneuver independently, with the control signal for the longitudinal and lateral guidance being received externally from the operating device.

[0029] An embodiment of the invention is described below. The following is shown: Fig. 1 a schematic representation of an embodiment of the system according to the invention with a motor vehicle and an operating device; Fig. 2 a schematic representation of a touchscreen of the operating device of Fig. 1; and Fig. 3. A diagram illustrating a touchscreen detection signal from Fig. 2.

[0030] The embodiment described below is a preferred embodiment of the invention. In this embodiment, the described components each represent individual features of the invention that can be considered independently of one another. Each of these features further develops the invention independently and can therefore be considered part of the invention individually or in a combination other than that shown. Furthermore, the described embodiment can also be supplemented by other features of the invention already described.

[0031] In the figures, functionally identical elements are each provided with the same reference symbols.

[0032] Fig. Figure 1 shows a system 10 with a motor vehicle 11 and an operating device 12. The vehicle 11 is a device that is operated by means of the operating device 12. In the illustrated example, the motor vehicle 11 can be a car, for example a passenger car or truck, with an attached trailer 13. A user 14 of the motor vehicle 11 operates the motor vehicle 11 from outside the vehicle by means of the operating device 12. The operating device 12 can be, for example, a mobile device such as a smartphone. The operating device 12 has an output device 15 by means of which, for example, a control signal 17 can be transmitted via a radio connection 16 to a receiving device 18 of the motor vehicle 11. The control signal 17 received by the receiving device 18 from the operating device 12 can, for example, be output to a driver assistance system 19 of the motor vehicle 11.The driver assistance system 19 can, for example, perform longitudinal and / or lateral guidance of the motor vehicle 11 depending on the control signal 17, and thereby, for example, carry out a parking maneuver 20, such as parking and / or exiting a parking space 21. For this purpose, a steering angle and / or a direction of travel with a corresponding speed can be set using the control device 12. The user 14 can perform a corresponding operating gesture 24 with a finger 22 on a touchscreen 23. The finger 22 represents a control object within the meaning of the invention. The touchscreen 23 represents a touchpad. A detection signal 25 generated by the touchscreen 23, which is in . Fig. As illustrated in Figure 3 and explained in more detail later, the control signal 17 can be received by a processor unit 26 of the operating device 12 and processed as a basis for generating the control signal 17.

[0033] For this purpose, the processor unit 26 can do this based on Fig. Perform the 2 explained procedures.

[0034] In Fig. Figure 2 shows the touchscreen 23 again. Depending on a touch position, i.e., a signaled position of a touch point 27 of the finger 22 on the touchscreen 23, a steering angle 28 (e.g., X-position) and a direction of travel 29 (Y-coordinate) are set, as well as a driving speed V0...V3 for the respective driving direction 29, where in the example V3 > V2 > V1 > V0 = 0. By positioning the finger 22 on the touch point 27, the user thus sets the steering angle 28, driving direction 29, and driving speed 30 for the motor vehicle 11. A change or displacement of the touch point 27 also means a change in the control signal 17. This thus results in an operating gesture 24 for changing the control signal 17.If the motor vehicle is to continue driving with a constant steering angle and constant speed and direction 29, the user must keep finger 22 unchanged on a specific contact point 27, i.e., keep finger 22 still.

[0035] The control device 12 has a dead man's switch function that ensures the user continuously keeps their finger 22 pressed or positioned on the touchscreen 23. If the user 14 releases their finger 22 from the touchscreen 23, the control device 12 sends a stop signal 31 to the vehicle 11. The stop signal 31 is designed to stop the vehicle 11. This ensures that releasing the finger 22 from the touchscreen 23 does not cause the vehicle 11 to continue driving on its own, but rather brings it to a stop.

[0036] In order to reliably detect the touch on the touchscreen 23 even when the finger 22 is stationary or held still, the detection signal 25 is evaluated, for example, as follows.

[0037] Fig. Figure 3 illustrates the touchscreen 23 touch sensors S, of which only two are labeled with a reference symbol for clarity. Due to the size of the contact area 32 of the finger 22 on the touchscreen 23 at the contact point 27, several touch sensors S can generate a respective sensor signal 33. The detection signal 25 thus describes the contact point 27 based on several sensor signals 33. For example, an average value of the sensor signals 33 can define the coordinates for the contact point 27. Even when the finger 22 is held still on the touchscreen 23, a change in position or movement 34 of the finger 22 occurs, and thus a change in the contact area 32, due to, for example, a physiological tremor, such as that which a healthy user 14 may exhibit. Accordingly, there is a signal change or signal fluctuation 35 of the sensor signals 33.This change can be so small that the coordinates of the contact point 27 effectively do not change. The signal fluctuations 35 thus represent noise, since they are contained in the detection signal 25 when the contact coordinates of the contact point 27 remain unchanged. However, a "flickering" or alternating between several coordinate values, a so-called pixel flicker, can also occur. In order to prevent this from generating a corresponding fluctuation in the control signal 17, the finger 22 is classified as stationary or at rest if the detection signal 25 changes by less than a predetermined maximum change 36, e.g., 10 pixels.

[0038] In other words, for example, a change in the intensity value 37 of the sensor signals 33 due to the signal fluctuation 35 can be tolerated by ensuring that the tolerance according to the maximum change 36 still results in a detection signal 25 that signals constant coordinates for the point of contact 27 or coordinates within the maximum change 36. The value of the maximum change 36 can be graphically visualized for the user 14 on the touchscreen 23, as shown in Fig. 2 is illustrated by the icon as an example.

[0039] To detect the existence of the touch, the signal fluctuation 35 in the detection signal 25, i.e., the change in intensity or signal fluctuation 35 of the intensity values ​​37 of the individual sensor signals 33, can be used to verify or detect that the finger 22 is actually still touching the touchscreen 23. If, on the other hand, the signal fluctuation 35 is zero or less than a minimum value, it can be assumed that the detection signal 25 is no longer being generated by a finger 22, i.e., the user has released the finger 22 from the touchscreen 23. Then the termination signal 31 is generated.

[0040] Thus, the user 14 can specify the articulation angle or steering angle 28 and the speed 30 using the control device 12. The user 14 is monitored by a dead man's switch device or dead man's switch function. As soon as the touchscreen 23 is touched, the dead man's switch is considered activated; that is, the dead man's switch function triggers the abort signal 31 if a dead man's condition is violated. The dead man's condition stipulates that the user 14 must release their finger 22 from the touchscreen 23, so that the dead man's switch is considered deactivated. Such a release of a touchscreen cannot always be reliably detected. This is where the control device comes into play with a method in which the operation of the touchscreen by the user 14 always generates slight movements 34 on the touchscreen 23, even if the finger 22 only presses on one and the same touch point 27.Such fine movements 34 and / or the applied pressure force or intensity 37 are evaluated by the operating device 12. When the touchscreen 23 is activated, it is constantly monitored whether the touchscreen also makes fine movements 34 that result in a change of the contact point 27 of less than the maximum change 36. If these movements 34 do not occur within a defined time window, the dead man's switch is considered not activated, i.e., the abort signal 31 is triggered.

[0041] Therefore, with the operating device 12, it is not necessary for the user to perform an explicit movement gesture on the touchscreen 23 in order to meet the dead man condition.

[0042] This can be used, for example, to secure automatic parking applications or remote controls of motor vehicles.

[0043] Overall, this example shows how the invention can be used to evaluate a touchscreen for a dead man's switch function. Reference symbol list 10 System 11 vehicles 11 Motor vehicle 12 Operating device 13 followers 14 users 15 Output device 16 Radio connection 17 Control signal 18 Receiving equipment 19 Driver assistance equipment 20 parking maneuvers 21 parking spaces 22 fingers 23 Touchscreen 24 operating gestures 25 Detection signal 26 Processor setup 27 Point of contact 27 Points of contact 28 Steering angle 29 Direction of travel 30 driving speed 31 Abort signal 32 Contact surface 33 Sensor signal 34 movements 35 Signal fluctuation 36 Maximum change 37 intensity value S touch sensor

Claims

Method for providing a dead man's switch function by means of a touchscreen (23) of an operating device (12), wherein the operating device (12) recognizes at least one predetermined operating gesture (24) performed by means of an operating object (22) touching the touchscreen (23) and, depending on the respective recognized operating gesture (24), sends a control signal (17) to a device (11) controlled by the operating device (12) and the control signal (17) remains unchanged or is completely absent if the operating object (22) is stationary on the touchscreen (23), wherein the operating object (22) is classified as stationary if, starting from an initial touch position (XY), the operating object (22) changes its touch position by less than a predetermined maximum change (36).wherein a predetermined deadman condition is checked when the operating object (36) is stationary and a predetermined abort signal (31) is output to the device (11) if the deadman condition is not met, characterized in that the deadman condition includes a touchpad of the touchscreen (23) continuing to generate a time-varying detection signal (25) that describes the touch position, wherein the deadman condition requires that the detection signal (25) describing the touch position changes by more than zero but less than the maximum change (36) within a predetermined time interval. Method according to claim 1, wherein the touchscreen (23) describes the detection signal (25) as a touch position - the touch position (XY) of the control object on the touchscreen (23) and / or - a contact force of the control object (22) on the touchscreen (23) and / or - an area size of a touch surface (32) simultaneously touched by the control object (22) on the touchscreen (22). Method according to claim 2, wherein the touch position comprises the touch position (XY) of the control object (22) on the touchscreen (23) and the maximum change (36) is in a range from 30 pixels to 2 pixels (31) or at one pixel (31) of the touchscreen (23). Method according to one of the preceding claims, wherein a noise (35) is detected in the detection signal (25) which is also contained in the detection signal (25) when the touch position (XY) of the control object (22) on the touchscreen (23) is maintained, and the dead man condition comprises that the detection signal (25) signals the maintenance of the touch position (XY) and the noise (35) has a predetermined minimum power that is greater than zero. Method according to one of the preceding claims, wherein the operating object (22) is a finger (22) of a user. Method according to one of the preceding claims, wherein a swipe gesture is captured as the operating gesture (24). Method according to one of the preceding claims, wherein the duration of the time interval is less than 0.5 seconds or is in a range of 0.5 seconds to 2 seconds. Method according to one of the preceding claims, wherein the device (11) is stopped by the abort signal (31). Method according to one of the preceding claims, wherein the device (11) is arranged at a distance from the operating device (12) and the control signal (17) is transmitted to the device (11) via a radio link (16) and / or an optical link. Method according to one of the preceding claims, wherein the device (11) is a motor vehicle remotely controlled. Method according to claim 10, wherein the control signal (17) controls longitudinal and / or lateral guidance of the motor vehicle during a parking maneuver (20). Operating device (12) for operating a device (11), with an output device (15) for sending a control signal (17) to the device (11), in particular to a motor vehicle, characterized in that a processor device (26) is provided and is configured to carry out a method according to one of the preceding claims. Operating device (12) according to claim 12, wherein the operating device (12) is designed as a mobile terminal. System (10) comprising an operating device (12) according to claim 12 or 13 and a motor vehicle which is equipped to be remotely controlled from the outside with respect to longitudinal guidance and / or lateral guidance by a control signal (17) of the operating device (12).

Images