Non-contact operating device for a motor vehicle, as well as motor vehicle and operating procedure for the operating device

Abstract

Control device (17) for a motor vehicle (10), comprising: - a control element (18) which is configured to trigger a predetermined operating function (F1, F2, F3) when activated, - a recognition device (REC) configured to recognize, without contact, a predetermined, freely executed operating gesture (29) by which a user selects the control element (18) over a distance (20), and upon recognition of the operating gesture (29) to select the control element (18) for activation and / or to activate the control element (18), wherein a feedback device (23) is provided and configured to transmit a stimulus signal (31) to at least one finger (19) without contact upon recognition of the operating gesture (29) and thereby generate a feedback stimulus on the at least one finger (19) signaling the recognition of the operating gesture (29), characterized in that the recognition device (REC) comprises a laser vibrometer and / or a radar as a respective sensor (22) and is configured toto detect a relative movement of at least one finger (19) with respect to the control element (18) by means of each sensor (22).

Description

[0001] The invention relates to an operating device for a motor vehicle. A user can thus select an operating element contactlessly using an operating gesture, thereby either marking the operating element for activation and / or immediately activating the operating element. The invention also includes the operating device and a motor vehicle with the operating device according to the invention.

[0002] In a motor vehicle, a control device for the driver and / or front passenger is generally designed for a respective seating position of the user, which the seat assumes during a journey in which the motor vehicle is controlled by a driver. In the event that the motor vehicle is equipped with an autopilot, a driver does not necessarily have to leave their seat in the driving position during piloted travel. Rather, an open seating position (resting position) can be provided, away from the steering wheel and control device, in which the seat is shifted at a greater distance from the control device. However, this means that mechanical and touch-sensitive controls on the steering wheel, dashboard and center console are out of reach of the user.

[0003] DE 10 2015 201 728 A1 discloses a contactless selection of a control element in a motor vehicle. This involves the user aiming at a control element to be selected with their eye and fingertip. This operating gesture is captured by a camera. A subsequent twitch of the finger activates the control element.

[0004] DE 10 2012 215 407 A1 discloses highlighting an individual control element to indicate that it can be activated by a trigger gesture. This highlighting can provide for the control element to be backlit by an LED.

[0005] From DE 10 2009 046 376 A1 it is known that a user can operate a head-up display (HUD) without contact using a gesture that can be recorded by a camera.

[0006] The disadvantage of touchless operation of a control element is that the user has to check whether they have performed the intended operating gesture correctly. This requires, for example, a quick glance at the control device. In a motor vehicle, this distracts the user from the traffic. Even locating the control element can be difficult with touchless operation, as the user has to look at the control device, which also distracts from the traffic.

[0007] WO 2013 / 021 385 A2 discloses an operating device for a device that uses an image sensor, for example, a camera or an ultrasonic transducer, to detect an operating gesture of a hand and then controls the device. To signal successful recognition of a gesture, a confirmation signal can be generated, which can be an ultrasonic signal. The operating device can comprise numerous other sensors, such as a pressure sensor, a light sensor, a temperature sensor, or a magnetic sensor.

[0008] DE 10 2015 009 031 A1 discloses an operating device for a motor vehicle in which function selection is possible using gesture control. To provide feedback upon successful recognition of a gesture, a puff of air is generated, which the operator can feel with their finger.

[0009] US 2015 / 0 175 172 A1 discloses a motor vehicle that generates haptic feedback in response to a recognized operating gesture. Gesture detection can be achieved using a camera, infrared, RFID, or electromagnetic means.

[0010] The invention is based on the object of enabling contactless operation of an operating element of an operating device in a motor vehicle.

[0011] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figure.

[0012] The invention provides an operating device for a motor vehicle. The operating device comprises an operating element configured to trigger a predetermined operating function upon activation. Such an operating element can be, for example, a mechanical button or a mechanical rotary control, or an operating surface (button / icon) on a touchscreen.

[0013] In order to be able to select such an operating element without contact, a recognition device is provided which is designed to recognize, without contact, a predetermined operating gesture carried out freely in space, by which a user selects the operating element from a distance or across a distance. For example, the user can point at the operating element with at least one finger. The operating element can also be selected, for example, via gaze recognition (in addition to or alternatively to the hand / arm / finger gesture), i.e. the operating gesture includes looking towards the object. Upon recognition of the operating gesture, the recognition device selects the operating element for activation, for example by lighting it up, and / or activates the operating element so that the associated operating function is triggered.If the control element is only selected by the control gesture, a separate second control gesture can be provided, which then ultimately activates the selected control element, i.e., triggers the control function. Such a control gesture could, for example, be a pressing motion.

[0014] Due to the described contactless selection by means of an operating gesture, the user usually lacks any feedback confirming which control element he has selected and / or activated by means of the operating gesture.

[0015] According to the invention, a feedback device is therefore provided that is configured to generate feedback upon recognition of the operating gesture. According to the invention, this feedback is tactile with at least one finger yet contactless. For this purpose, the feedback device is configured to transmit a contactless stimulus signal to at least one finger upon recognition of the operating gesture.

[0016] This generates a feedback stimulus on at least one finger, signaling successful recognition of the operating gesture. The user can thus sense on at least one finger whether they have selected and / or activated the control element. For example, they can move their finger through the air or gaze over the control element, thereby (virtually) stroking the control device. Using the feedback stimulus generated by the stimulus signal, they can sense on at least one finger whether they have, for example, reached an edge of the control element.

[0017] The invention thus has the advantage that the user can sense tactilely or haptically, i.e., generally with a feeling on at least one finger, whether they have reached, selected, and / or activated the control element. This provides the user with feedback even when performing a contactless operating gesture, regardless of the distance to the control element.

[0018] In order to recognize the operating gesture, the invention provides that the recognition device has at least one of the following as a respective sensor: a laser vibrometer and / or a radar. In addition, the recognition device can comprise a capacitive proximity sensor and / or a camera. The recognition device is configured to use each sensor to detect a relative movement of the at least one finger with respect to the operating element. By "total" we mean that if there are multiple sensors, their sensor signals can be combined, instead of, for example, detecting the relative movement individually using each sensor. The sensor can therefore detect, for example, the said rotary movement and / or pressing movement with respect to the operating element. A laser vibrometer has the particular advantage that a change in position of the at least one finger in the range of less than 1 mm can be detected.Depending on this, in particular the stimulus signal for the feedback stimulus can also be controlled, so that, for example, a (virtual) pressing of the control element from a distance can be conveyed by the feedback stimulus tactilely and / or haptically and / or by the temperature change, in that a change in the feedback signal is set when a movement of the at least one finger of less than 5 mm, in particular less than 1 mm, is detected.

[0019] The invention also includes further developments which result in additional advantages.

[0020] Several further developments concern the question of how the feedback stimulus can be generated contactlessly on at least one finger.

[0021] A further development provides that the feedback device for this purpose has an ultrasound source that is configured to transmit the stimulus signal at least partially as an ultrasound wave, through which the feedback stimulus then comprises a tactile stimulus. The ultrasound wave is based on the principle also used for so-called ultrasound audio loudspeakers. When reflected at a media transition, ultrasound can generate a subharmonic oscillation (undertone) whose frequency is lower than the frequency of the ultrasound wave itself, whereby the subharmonic oscillation on the skin of a finger represents a vibration signal (structure-borne sound signal). Thus, a vibration is generated in at least one finger in a frequency range that can be perceived by a person as a tactile stimulus. An ultrasound source of the type described can, for example, be formed on the basis of a piezo ultrasound transducer.

[0022] Additionally or alternatively, the feedback device can comprise an infrared source configured to transmit the stimulus signal at least partially as an infrared beam, through which the feedback stimulus includes a heat stimulus. Successful recognition of the operating gesture thus leads to a temperature difference on at least one finger compared to a non-recognized operating gesture. The infrared source can be, for example, an infrared laser.

[0023] Additionally or alternatively, the feedback device can have at least one air nozzle configured to transmit the stimulus signal at least partially as an air flow, through which the feedback stimulus comprises an air stream. Thus, for example, an air vortex or a blast of air is generated at the at least one finger, which can brush over or press in the skin of the at least one finger. The flow strength and / or flow direction can be adjusted depending on the recognition of the operating gesture.

[0024] As already explained, the operating gesture can serve to simply select the operating element for activation, i.e., to prepare the activation or to define an operating focus. In this context, it can then be provided that a subsequent second operating gesture is to be executed, i.e., a gesture that is then also recognized by the recognition device and by which the operating element is subsequently activated. In order to signal to a user that the operating element has been selected, a visual and / or acoustic differentiation of the operating element between the selected state and a non-selected state can be provided. For example, the lighting of the operating element can be adjusted depending on the state.As an acoustic distinction, for example, a signal tone can be set or controlled in terms of its frequency and / or volume depending on the state (selected / not selected) or a voice announcement can be issued which announces the selected operating function.

[0025] The operating gesture can also provide for the actual activation of the operating element. In this case, the operating gesture for activating the operating element can comprise a freely spatially executed rotating movement and / or pressing movement of at least one finger. “Freely spatially” means that the operating gesture is contact-free, i.e., there is no touching of the operating element. By means of a rotating movement, a value of a parameter can be set continuously or in steps by the operating function, e.g., a temperature value for an air conditioning function. By means of a pressing movement, a mechanical button or a touchscreen button can be controlled. The detection of a sliding movement for a slider is also possible.

[0026] The operating gesture can also include both processes—the selection and subsequent activation—where the feedback signal can be modified during the transition from selection to activation, for example, by being amplified. The operating gesture can therefore include a selection part and a triggering part.

[0027] However, the detection range of a laser vibrometer and / or a capacitive proximity sensor can be spatially limited in such a way that the location of at least one finger must first be determined using another sensor. A further problem can arise if a reliable distinction is to be made between a random hand movement on the one hand and the operating gesture on the other. For this purpose, at least one secondary sensor is preferably provided. The recognition device can comprise a camera and / or an eye-tracking device (gaze direction detection) as a respective secondary sensor. The recognition device is then configured to verify the recognized operating gesture on the basis of a plausibility criterion using the respective secondary sensor and to mark the operating element for activation and / or to activate the operating element only if the plausibility criterion is met. For example,As a plausibility criterion, it can be checked whether the user's gaze is directed toward the control element. Additionally or alternatively, based on the respective secondary sensor, the recognition device for recognizing the operating gesture can be aligned to the at least one finger. For example, a detection beam of a laser vibrometer can be aligned to the at least one finger. For this purpose, an electric motor can be controlled, for example.

[0028] In order to enable the user to virtually feel the operating element from a distance, it is preferably provided that the operating element has a knurling and the operating gesture comprises a sliding movement across the operating element. The stimulus signal is then configured to simulate the structure of the knurling. In other words, a temporal progression of the intensity and / or the point of impact of the stimulus signal correlates with the times at which the at least one finger touches individual grooves or knurls of the knurling, as would occur with an actual touching sliding movement on the knurled operating element. The user therefore feels a roughness or knurling of the operating element with the feedback stimulus. This allows the user to identify the operating element based on the knurling. For example, a rotating roller or a rotary actuator with knurling can be provided as the operating element.

[0029] So far, the operating device has only been described with a single operating element. The operating device can also comprise a plurality of operating elements of the type described. In addition to the operating element already described, at least one further operating element is therefore provided. The total of several operating elements can now be virtually swiped over by the user with at least one finger. The operating gesture can therefore comprise a selection movement across the operating elements. The feedback device is then correspondingly configured to signal a transition from one of the operating elements to the next of the operating elements as a transition by means of the stimulus signal. The user therefore feels a gap or an edge or a joint through the feedback stimulus, such as can arise between two operating elements.A temporal progression of the intensity and / or the point of impact of the stimulus signal thus correlates with the times at which at least one finger touches, for example, a joint or an edge between two control elements. The user thus feels the transition between two of the control elements with the feedback stimulus. This allows the user, for example, to count which control element from a series of control elements they have just selected. This allows selection without eye contact.

[0030] The invention also includes a motor vehicle. The motor vehicle according to the invention has an embodiment of the operating device according to the invention. The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck.

[0031] The invention also includes a method for operating an operating device of a motor vehicle. The invention is based on the fact that an operating element triggers a predetermined operating function when activated. Such an operating function can, for example, be the activation or control of an air conditioning system, a radio, a media playback device, or an internet browser. In order to enable this user-side operation from a distance, a recognition device contactlessly recognizes an operating gesture performed freely in space, by which the user selects the operating element from a distance or across a distance. Upon recognition of the operating gesture, the recognition device marks the operating element for activation and / or even activates the operating element. This depends, as described, on the predetermined operating gesture.

[0032] In order to provide the user with a tactile feedback stimulus despite the operating gesture being carried out freely in space, the invention provides that the described feedback device transmits a stimulus signal to at least one finger without contact upon recognition of the operating gesture and thereby generates the feedback stimulus on the at least one finger, which signals the successful recognition of the operating gesture. In other words, the user is signaled by means of the stimulus signal that the operating gesture has been recognized. Otherwise, the stimulus signal is not generated or a different stimulus signal is generated. Thus, in the manner described, for example, swiping over several operating elements can be signaled by signaling the transitions between the operating elements and / or swiping over a relief of the operating element and / or pressing an operating element and / or rotating an operating element.

[0033] The invention also includes further developments of the method according to the invention that have features already described in connection with the further developments of the operating device according to the invention. For this reason, the corresponding further developments of the method according to the invention are not described again here.

[0034] Exemplary embodiments of the invention are described below. The single figure (Fig.) shows a schematic representation of an embodiment of the motor vehicle according to the invention.

[0035] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual, independently considered features of the invention, which also further develop the invention independently of one another and are thus also to be considered as components of the invention, either individually or in a combination other than that shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0036] The figure shows a bird's-eye view of a motor vehicle 10, which may be, for example, a motor vehicle, in particular a passenger car or a truck. The motor vehicle 10 may have a steering handle 11, for example, a steering wheel. Furthermore, a driver's seat 12 is shown, in which a user can sit, a hand 13 of which is shown in the figure. In the situation shown in the figure, the motor vehicle 10 may, for example, be guided by an autopilot 14. Therefore, it is unnecessary for the user to operate the steering handle 11. They may, for example, have moved their driver's seat 12 from a driving position 15 to a relaxed or resting position 16, through which the driver's seat 12 is further away from the steering handle 14 than in the driving position 15.

[0037] The motor vehicle 10 can have an operating device 17 with operating elements 18, each of which the driver can activate to trigger a predetermined operating function F1, F2, F3, for example, switching on an air conditioning system and / or setting a parameter value, for example an air conditioning temperature. This can be a mechanical operating element 18 (respectively a button, slide control, and / or rotary control) or a graphical operating element 18 on a touchscreen. Since the driver is in the rest position 16 with the driver's seat 12, the increased distance to the operating elements 18 means that they can no longer be reached with their hand 13.

[0038] The operating device 17 nevertheless offers the driver the possibility of activating the operating elements 18 contactlessly, for example, with a finger 19 over a distance 20. For this purpose, the operating device 17 can have a control device 21, a sensor 22, a feedback device 23, and at least one secondary sensor 24, 25. The distance 20 is in particular greater than 5 cm, preferably greater than 10 cm.

[0039] By means of the at least one secondary sensor 24, 25, the user's operating intention can be verified, for example, based on a viewing direction and / or a posture and / or position of the hand 13. For this purpose, the secondary sensor 24 can be, for example, a camera whose detection range 26 is directed into an interior 27 of the motor vehicle 10, so that a relative position of the hand 13 with respect to the operating elements 18 can be detected. The secondary sensor 25 can be, for example, an eye-tracking device. Such an eye-tracking device can detect a user's viewing direction. Eye-tracking devices are known per se from the prior art. A detection range 27 of the eye-tracking device 25 can, for example, be directed towards the driver's seat 12.

[0040] The control device 21 can receive respective control signals from the sensor 22 and the at least one secondary sensor 24, 25. The control device 21 can comprise a processor device, which can, for example, comprise at least one microcontroller and / or at least one microprocessor. The processor device can execute program code, which can, for example, be stored in a data memory of the processor device. The control device 21, together with the sensors 22, 24, 25, can provide a detection device REC.

[0041] The detection device REC can be activated, for example, when the autopilot 14 is activated. The control device 21 can check a plausibility criterion 28, which specifies at least one condition for a respective sensor signal from the at least one secondary sensor 24, 25. For example, the plausibility criterion 28 can state that the user's line of sight must be directed toward a predetermined area in which the control elements 18 are located. One condition can state that the hand 13 must be held in a predetermined spatial area to enable operation of the control elements 18.

[0042] By means of the sensor 22, an operating gesture 29, for example of the finger 19, can be detected. The operating gesture 29 can, for example, comprise pivoting the finger or pointing the finger at an operating element 18. The operating gesture 29 can also comprise a pressing or rotating movement, by which a selected operating element 18 can be activated, thus triggering the associated operating function F1, F2, F3.

[0043] The sensor 22 can, for example, be a laser vibrometer, by which a relative distance 30 of the finger 29 to the sensor 22 can be determined. One of the previously described secondary sensors 24, 25 or a combination of several of the sensors 22, 24, and 25 can also be used to detect the operating gesture 29.

[0044] If the operating gesture 29 is recognized by the control device 21, the feedback device 23 can be controlled by the control device 21 to confirm the recognized operating gesture 29. The feedback device 23 can emit a stimulus signal 31 to the fingers 19. The stimulus signal 31 transmits energy from the feedback device 23 to the finger 19 without contact. For example, the feedback device 23 can have an ultrasound source through which an ultrasound signal or an ultrasound wave 32 is emitted through the air at the fingers 19. At the finger 19, the ultrasound wave 32 interacts with the skin of the finger 19. This causes a vibration in the skin whose frequency is lower than the frequency of the ultrasound wave 32. In particular, a vibration with a vibration frequency of less than 1 kHz, in particular less than 500 Hz, results.The user thereby feels a trembling or vibration on finger 19, which he may perceive as a tactile stimulus.

[0045] The feedback device 23 may also comprise an infrared source for generating a heat stimulus and / or an air nozzle for generating an air flow on the finger 19.

[0046] The following embodiment is particularly preferred.

[0047] This results in eye-gesture-haptic coordination at the control device 17 when operated across a distance 20. The control elements 18 are accessible and operable without restriction by the user, for example, the driver and / or front passenger, even in the rest position 16 of the respective seat.

[0048] The pointing direction of finger 19 toward one of the operating elements 18 is detected by the gesture recognition of the recognition device REC, specifically as an operating gesture 29. The gesture recognition can detect a change in the relative distance 30 of less than 1 mm with high precision within a sensing range using a relative measuring laser vibrometer if the absolute distance of finger 19 to the laser vibrometer is less than, for example, 1 m. Using a second measuring beam, for example, eye tracking or camera detection, the operating intention can be verified using the plausibility criterion 28.

[0049] The control element 18 targeted by the operating gesture 29 can be illuminated accordingly depending on the flowing selection movement, resulting in a light signal 33 at the currently selected control element 18. The light signal 33 thus represents a visual distinction between a selected state and a non-selected state. In the selected state, for example, the light signal 33 is active.

[0050] Furthermore, the user can feel haptic feedback or a response stimulus from the user interface of the control elements 18, including transitions 34 therebetween, e.g., joints, and even ribbing on the surface of the control elements 18, with the finger 19. This haptic or tactile feedback can be generated, for example, by ultrasound.

[0051] The selected control element is clearly highlighted visually, for example by pulsing the light signal 33. A rotational movement or pressing movement of the control gesture 29 is also reported back haptically, for example by emitting a stronger ultrasonic pulse in the fingertip of the finger 19.

[0052] Overall, the examples demonstrate how the invention can be used in a motor vehicle to provide operation through gestures and eye tracking with visual and haptic feedback. Non-driving tasks can thus be performed remotely during piloted driving. Eye-gesture-haptic coordination is envisioned.

Claims

[1] Operating device (17) for a motor vehicle (10), comprising: - an operating element (18) which is designed to trigger a predetermined operating function (F1, F2, F3) when activated, - a recognition device (REC) which is configured to detect, without contact, a predetermined operating gesture (29) carried out freely in space, by which a user selects the operating element (18) over a distance (20), and, upon detection of the operating gesture (29), to select the operating element (18) for activation and / or to activate the operating element (18), wherein a feedback device (23) is provided and configured to transmit, without contact, a stimulus signal (31) to at least one finger (19) upon detection of the operating gesture (29) and thereby to generate a feedback stimulus on the at least one finger (19) signaling the detection of the operating gesture (29), characterized byin that the detection device (REC) comprises a laser vibrometer and / or a radar as a respective sensor (22) and is designed to detect, by means of each sensor (22), a relative movement of the at least one finger (19) with respect to the operating element (18). [2] Operating device (17) according to claim 1, wherein the feedback device (23) comprises an ultrasound source which is designed to transmit the stimulus signal (31) at least partially as an ultrasound wave (32), by means of which the feedback stimulus comprises a tactile stimulus. [3] Operating device (17) according to one of the preceding claims, wherein the feedback device (23) comprises an infrared source which is designed to transmit the stimulus signal (31) at least partially as an infrared beam, by means of which the feedback stimulus comprises a heat stimulus. [4] Operating device (17) according to one of the preceding claims, wherein the feedback device (23) has at least one air nozzle which is designed to transmit the stimulus signal (31) at least partially as an air flow, through which the feedback stimulus comprises an air flow. [5] Operating device (17) according to one of the preceding claims, wherein the selection of the operating element (18) provides an optical and / or acoustic differentiation (33) of the operating element (18) between a selected state and a non-selected state. [6] Operating device (17) according to one of the preceding claims, wherein the operating gesture (29) for activating the operating element (18) comprises a freely spatially executed rotational movement and / or pressing movement and / or sliding movement of the at least one finger (19). [7] Operating device (17) according to one of the preceding claims, wherein the detection device (REC) further comprises a capacitive proximity sensor and / or a camera as a respective sensor (22) and is configured to detect, by means of each sensor (22), a relative movement of the at least one finger (19) with respect to the operating element (18). [8] Operating device (17) according to one of the preceding claims, wherein the recognition device (REC) comprises a camera and / or an eye-tracking device as a respective secondary sensor (24, 25) and is configured to use the respective secondary sensor (24, 25) to a) align the recognition device (REC) to detect the operating gesture (29) with the at least one finger (19) and / or b) verify the recognized operating gesture (29) on the basis of a plausibility criterion (28) and to mark the operating element (18) for activation and / or to activate the operating element (18) only if the plausibility criterion (28) is met. [9] Operating device (17) according to one of the preceding claims, wherein the operating element (18) has a ribbing and the operating gesture (29) comprises a sliding movement over the operating element (18) and the stimulus signal (31) is designed to simulate a structure of the ribbing. [10] Operating device (17) according to one of the preceding claims, wherein in addition to the operating element (18) at least one further operating element (18) is provided and the operating gesture (29) comprises a selection movement across the operating elements (18) and the feedback device (23) is configured to signal a transition from one of the operating elements (18) to a next of the operating elements (18) by means of the stimulus signal (31) as a transition (34). [11] Motor vehicle (10) with an operating device (17) according to one of the preceding claims. [12] Method for operating an operating device (17) of a motor vehicle (10), wherein - an operating element (18) triggers a predetermined operating function (F1, F2, F3) when activated, - a recognition device (REC) detects, without contact, an operating gesture (29) carried out freely in space, by means of which a user selects the operating element (18) over a distance (20), and upon detection of the operating gesture (29), selects the operating element (18) for activation and / or activates the operating element (18), wherein - a feedback device (23) upon recognition of the operating gesture (23) transmits a contactless stimulus signal (31) to at least one finger (19) and thereby generates a feedback stimulus on the at least one finger (19) signalling the recognition of the operating gesture (29), characterized by that the detection device (REC) comprises a laser vibrometer and / or a radar as a respective sensor (22) and by means of each sensor (22) detects a relative movement of the at least one finger (19) with respect to the operating element (18).

Images