Method and device for operating an input device

Abstract

Method for operating an input device (10) in a vehicle, wherein the input device (10) comprises at least one operating element (20) which is operable by means of a predetermined operating actuation which comprises a predetermined mechanical force exerted by means of a predetermined actuating element (30), and in which, depending on at least one captured image of an image sequence which represents a movement and / or position of the predetermined actuating element (30) in a predetermined environment of the operating element (20), a prediction of the operating actuation of the operating element (20) is determined, and in which, depending on the prediction - and depending on a measurement signal generated in response to an actuation of the operating element (20), an actuation signal (B) is generated which is representative of the fact that the predetermined actuation of the operating element (20) has been carried out and - a graphic representation on a given display area is changed.

Description

[0001] The invention relates to a method and a device for operating an input device for a vehicle. Furthermore, the invention relates to an operating system for a vehicle.

[0002] Modern motor vehicles feature a multitude of electronic devices that do not function autonomously, but can or must be operated by the driver or other vehicle occupants. A key requirement in the automotive sector is that the controls are as simple as possible to operate, in order to minimize distraction for the vehicle user.

[0003] However, especially when using a control element in a motor vehicle, it is also important that the operator receives feedback about the operation, even without looking at the control element. Acoustic and / or haptic feedback, which does not require visual perception by the driver, is particularly suitable for this. With touchscreens and gesture-based recognition systems, it is often difficult for the operator to determine whether the control element has realized and implemented the desired operation.

[0004] Control elements such as pushbuttons, buttons, or rotary controls therefore continue to be popular in the automotive sector. These control elements make it easy for the operator to determine whether the switch or button has been operated correctly. This occurs, for example, when a switch clicks into place or when a mechanical pressure point is exceeded, providing the operator with immediate tactile or acoustic feedback regarding the actuation of the control element.

[0005] US 2008 / 0255725 A1 relates to a system for managing an interior component of a vehicle capable of carrying a driver and at least one passenger. The system includes a camera for monitoring the at least one passenger, a vehicle speed detector, and a controller in communication with the camera, the vehicle speed detector, and the interior component. The controller is configured to deactivate at least one function of the interior component if it detects that the speed is greater than a preset speed threshold and to activate the at least one deactivated function if the detected speed is greater than the speed threshold, if the passenger attempts to activate the at least one function.DE 102011 054 848 A1 relates, on the one hand, to a control device for controlling a vehicle movement of a vehicle, comprising a control command input device for inputting manual control commands for controlling the vehicle movement of the vehicle, and a control signal unit configured to generate control signals for controlling the vehicle movement of the vehicle depending on the manual control commands. DE 102011 054 848 A1 also relates to a monitoring device for monitoring a touch-dependent control element for controlling a vehicle movement of a vehicle, as well as a detection device for detecting a touch state of at least one touch-dependent control element for controlling a vehicle movement of a vehicle by a driver.DE 10 2006 037 156 A1 relates to a method for operating an interactive operating device, in particular in a motor vehicle, with a display device and information displayed and / or displayable on the display device, which includes operating elements, comprising the following steps: a) determining an operating intention for at least one of the operating elements displayed on the display device, b) adapting the information displayed on the display device depending on the determined operating intention, so that the at least one operating element is displayed in an optimized manner for activating the at least one operating element.

[0006] The object underlying the invention is to provide a method and a device for operating an input device as well as an operating system which contribute to simplifying the operation of the input device and / or enabling an operation of the input device to be detected more reliably.

[0007] The problem is solved by the features of the independent patent claims. Advantageous developments of the invention are characterized in the subclaims.

[0008] According to a first and second aspect, the invention is characterized by a method and a corresponding device for operating an input device in a vehicle, wherein the input device comprises at least one operating element that can be operated by means of a predetermined operating actuation, which comprises a predetermined mechanical force exerted by means of a predetermined actuating element. A prediction of the operating actuation of the operating element is determined depending on at least one captured image of an image sequence that represents a movement and / or position of a predetermined actuating element in a predetermined environment of the operating element. Depending on the prediction, a graphic representation on a predetermined display area is changed.In addition, depending on the prediction and a measurement signal generated in response to an actuation of the operating element, an actuation signal is generated that is representative of the fact that the specified actuation of the operating element has been carried out.

[0009] Depending on the actuation signal, for example, a vehicle function and / or a function in connection with the vehicle that is assigned to the specified actuation of the control element is executed.

[0010] Prediction can be used to advantage to more reliably detect the actuation of the control element and / or to simplify its operation. This can be particularly advantageous for control elements that require a predetermined mechanical force to be exerted by the actuating element on the control element. Such control elements are also referred to as mechanical control elements. Mechanical control elements include, for example, switches, buttons, rotary controls, slide controls, tilt controls, and so on. Such control elements have the advantage of being very simple and intuitive to use. Furthermore, such control elements can very easily be equipped with means for providing haptic feedback to the vehicle user or operator. The haptic feedback is preferably provided via the actuating element.

[0011] The prediction can advantageously be used to preview operating information. The at least one operating element can be designed as a multiple operating element. Depending on the system state or vehicle state, different functions of the vehicle and / or functions associated with the vehicle can be assigned to the specified actuation of the operating element. The prediction makes it possible to provide a vehicle user with specific operating information as soon as an approach of the actuating element to the operating element is detected and / or an intended actuation of the operating element by the vehicle user is detected. This has the advantage that the number of functions assigned to the at least one operating element can be increased, while operating the operating element remains easy for the vehicle user.

[0012] Prediction advantageously enables improved, and in particular more reliable, evaluation of control element activations. Thus, greater recognition reliability can be achieved.

[0013] The prediction can be used, on the one hand, to verify whether an actuation of the control element should be classified as a valid actuation of the control element and / or whether the actuation of the control element should be considered the specified actuation of the control element. The evaluation of the image data makes it possible to reduce the requirements for the precision of executing a specified actuation by the vehicle user to operate the control element. Variations in the execution of the specified actuation of the control element can be taken into account when determining the prediction. Capacitive switching elements for increasing the detection reliability of the control element are not required. Furthermore, greater mechanical wear and / or mechanical tolerances of the control element can be permitted.The at least one control element of the input device can be manufactured simply and thus cost-effectively. Electronic control and / or evaluation of the at least one control element can be carried out very easily.

[0014] Furthermore, prediction can be used to distinguish between accidental and intentional actuations of the control element. For example, if the measurement signal is generated, but the prediction shows that the actuation was not performed using the specified actuation element, it can be concluded that the actuation of the control element was accidental. This can occur, for example, if the control element is accidentally actuated when setting down an object. The analysis of the image data can be advantageously used here. Depending on the image data, it is easy to determine whether the actuation was performed using the specified actuation element and / or whether the actuation follows a typical movement pattern.

[0015] The evaluation of the image data of at least one image enables the prediction to be determined very precisely and reliably. Image processing algorithms for evaluating the image data can be implemented depending on the prediction requirements. Adapting a control element and / or additional hardware devices to detect the approach of the specified actuating element is not necessary. If the prediction requirements change, the image processing algorithms can also be easily adapted subsequently.

[0016] In an advantageous embodiment of the first and second aspects, the actuating element comprises at least a predetermined body part of a vehicle user. This enables simple operation by the vehicle user.

[0017] In a further advantageous embodiment of the first and second aspects, a vehicle function and / or a function in connection with the vehicle that is associated with the predefined operation of the control element is executed depending on an execution condition. Advantageously, even if the control element has a defect and therefore no desired measurement signal is generated, the vehicle function and / or the function in connection with the vehicle can be executed if the execution condition is deemed to be met. This has the advantage that, despite the defect, the execution of the vehicle function and / or the function in connection with the vehicle can be easily initiated by the vehicle user. The execution condition can, for example, comprise a predefined voice input. Alternatively or additionally, the execution condition can comprise a predefined rule.

[0018] In a further advantageous embodiment of the first and second aspects, a control signal representative of the functionality of the control element is determined depending on the actuation signal. This makes it easy to inform the vehicle user that the control element is defective.

[0019] In a further advantageous embodiment of the first and second aspects, the operating element comprises an adjustment element. The adjustment element has the advantage of enabling quick and precise operation.

[0020] According to a third aspect, the invention is characterized by an operating system for a vehicle. The operating system comprises a device according to the second aspect and an image capture device. The image capture device is configured to capture an image or image sequence representing a movement and / or position of a predetermined actuating element in a predetermined environment of the operating element, and to forward image data of the captured image or image sequence to the device.

[0021] Advantageous embodiments of the first and second aspects also apply to the third aspect.

[0022] Embodiments of the invention are explained below with reference to the schematic drawings.

[0023] They show: Fig. 1 a block diagram for an embodiment of an operating system for a vehicle and Fig. 2 an image section representing a movement and / or position of a given operating element in a given environment of the operating element and Fig. 3 a block diagram for a further embodiment of the operating system with a display unit.

[0024] Elements of the same construction or function are provided with the same reference symbols throughout the figures.

[0025] Fig. 1 shows an operating system 45 for a vehicle. The vehicle is preferably designed as a motor vehicle. The operating system 45 comprises an input device 10 with at least one operating element 20 and a control device 60.

[0026] The at least one operating element 20 is designed such that it can be operated by means of a predetermined operating actuation which comprises a predetermined mechanical force exerted by means of a predetermined actuating element 30.

[0027] The control device 60 can also be referred to as a device for operating the input device 10. The control device 60 comprises, for example, a computing unit and a program memory. The control device 60 can, for example, comprise a central processing unit in the vehicle.

[0028] The operating system 45 further includes an image capture device 50. The image capture device 50 is preferably embodied as a camera, for example, a video camera. The image capture device 50 is preferably arranged in the vehicle. The image capture device 50 has a capture area that encompasses at least a predetermined area surrounding the operating element 20.

[0029] Alternatively, it is possible for the control device 60 to comprise the image capture device 50.

[0030] The image capture device 50 is designed to capture an image or an image sequence that represents a movement and / or position of a predetermined actuating element 30 in the predetermined environment of the operating element 20.

[0031] The image capture device 50 is signal-coupled to the control device 60 and is further configured to forward image data of the captured image or image sequence to the control device 60. The image capture device 50 can detect an approach of the actuating element 30, preferably a body part of a vehicle user, to the operating element 20. The body part comprises, for example, one or more finger parts and / or one or more fingers and / or a hand and / or an arm.

[0032] The control device 60 is configured to determine a prediction of the operating operation of the operating element 20 based on the at least one captured image of the image sequence, which represents a movement and / or position of the predefined actuating element 30 in the predefined environment of the operating element 20. Furthermore, the control device 60 is configured to generate an actuation signal B based on the prediction and a measurement signal generated in response to an actuation of the operating element 20, which signal is representative of the fact that the predefined actuation of the operating element 20 has been executed.

[0033] Depending on the actuation signal B, for example, a vehicle function and / or a function in connection with the vehicle, which is assigned to the predetermined actuation of the operating element 20, is executed.

[0034] Based on the detected approach of the actuating element 30 to the operating element 20, an evaluation can be carried out and it can be estimated and / or predicted that the vehicle user intends to actuate the at least one operating element 20.

[0035] Depending on the actual actuation of the control element 20, a measurement signal is recorded. The measurement signal can, for example, comprise a binary signal with two states, such as in the case of a switch. Alternatively or additionally, the measurement signal can comprise a continuous or discrete signal waveform, such as in the case of a rotary or sliding actuator.

[0036] If the control element 20 exhibits a fault, such as a loose connection, the measurement signal will not be generated as desired. For example, the signal level of the measurement signal will not change or will not change in a predetermined manner, despite the vehicle user operating the control element 20.

[0037] The control device 60 is designed, for example, to determine a control signal representative of the functionality of the control element 20 as a function of the actuation signal B. This control signal can be signaled to the vehicle user, in particular visually and / or acoustically and / or haptically, if the measurement signal was not generated as expected.

[0038] Furthermore, the control device 60 can be configured to execute the vehicle function and / or a function associated with the vehicle, which is associated with the specified operating operation of the control element 20, depending on an execution condition. The execution condition can, for example, include a specified voice input. Alternatively or additionally, the execution condition can include a specified rule.

[0039] The control device 60 is configured, for example, to output an acoustic query. The acoustic query can, for example, represent a request to the vehicle user as to whether the vehicle function associated with the specified operating action should still be executed.

[0040] For example, if the specified operation of the control element 20 is assigned an entertainment function, the rule may include, for example, that if more than one attempt is made to operate the at least one control element 20, the entertainment function is executed even though the measurement signal was not generated. The rules may, in particular, include safety aspects.

[0041] Fig. 2 shows an exemplary image section of an image captured by the image capture device 50. In the Fig. In the example shown in Figure 2, the control element 20 comprises an adjustment element. In this case, the adjustment element comprises a rotary control. Alternatively or additionally, the respective control element 20 can be designed as a button, switch, tilt control, and so on. Adjustment elements, in particular rotary controls, are used in many places in vehicles, such as passenger cars or trucks, for operating vehicle functions or functions of a so-called infotainment system. In infotainment systems in particular, the adjustment elements are often used as primary input devices for operation. In addition, the adjustment elements are also used, for example, to regulate the volume of audio playback systems and / or for temperature regulation and / or climate control and / or light regulation.In the area of infotainment systems, the adjustment elements are particularly suitable for operating long lists, as they enable quick and precise operation.

[0042] As in Fig. As shown in Figure 2, a hand, in particular a thumb and index finger, moves toward the control element 20. The approach of the hand to the control element 20 is detected, for example, by the video camera arranged in the vehicle.

[0043] An evaluation of the image data makes it possible, for example by means of a pattern comparison, to compare a movement pattern and / or a position of the actuating element 30 with a desired movement pattern and / or a desired position. The captured image(s) can be evaluated using an image processing program. For example, the movement of the hand and / or another specified body part can be determined using a pattern comparison and / or motion estimation algorithm.

[0044] Fig. 3 shows the operating system 45 with a display unit 70. In this exemplary embodiment, a display unit 70 is assigned to the input device 10. Alternatively, the input device 10 can include the display unit.

[0045] In this case, the control device 60 is configured to determine a prediction of the operating operation of the operating element 20 based on at least the captured image of the image sequence, which represents a movement and / or position of a predetermined actuating element 30 in a predetermined environment of the operating element 20. Furthermore, the control device 60 is configured to change a graphical representation on a predetermined display area based on the prediction. Additionally, the control device 60 can be configured to generate the actuation signal B, representative of the fact that the predetermined operating operation of the operating element 20 has been executed, based on the prediction and the measurement signal generated in response to the actuation of the operating element 20.

[0046] Depending on the prediction, a display is signaled by the display unit a predetermined period of time before the vehicle user actuates the control element 20. The display comprises at least one display element 80, 80b, to which, for example, a function of the vehicle and / or a function associated with the vehicle is assigned. The function of the vehicle and / or the function associated with the vehicle is executed when the respective predetermined actuation of the control element 20 is detected and / or the actuation signal B is generated.

[0047] The Fig.The display of the display unit shown in Figure 3 comprises a plurality of display elements 80, 80b. A first display element 80 can represent a currently executed function, while the other display elements 80b can represent the functions that are executed when the operating element 20 is actuated in a predetermined manner, for example, when a rotary element of the operating element 20 is rotated by a predetermined angle in a first or second direction. The display elements can, for example, each represent a list entry.

[0048] Alternatively, it is possible for the first display element to represent the function that is executed as a result of the specified operating operation by the vehicle user, and for the additional display elements 80b to represent functions that are each executed upon a different specified operating operation of the operating element 20. This has the advantage that the number of functions assigned to the at least one operating element 20 can be increased, while the vehicle user can still easily operate the operating element 20. List of reference symbols 10 Input device 20 Control element 30 Actuating element 45 Operating system 50 image capture device 60 Control device 70 display unit 80 first display element 80b additional display element B Actuation signal

Claims

[1] Method for operating an input device (10) in a vehicle, wherein the input device (10) comprises at least one operating element (20) which is operable by means of a predetermined operating actuation which comprises a predetermined mechanical force exerted by means of a predetermined actuating element (30), and in which, depending on at least one captured image of an image sequence which represents a movement and / or position of the predetermined actuating element (30) in a predetermined environment of the operating element (20), a prediction of the operating actuation of the operating element (20) is determined, and in which, depending on the prediction - and depending on a measurement signal generated in response to an actuation of the operating element (20), an actuation signal (B) is generated which is representative of the fact that the predetermined actuation of the operating element (20) has been carried out and - a graphic representation on a given display area is changed. [2] Method according to claim 1, wherein the actuating element (30) comprises at least one predetermined body part of a vehicle user. [3] Method according to claim 1 or 2, wherein, depending on an execution condition, a vehicle function and / or a function in connection with the vehicle, which is associated with the predetermined operating operation of the operating element (20), is executed. [4] Method according to one of the preceding claims, in which a control signal is determined as a function of the actuation signal (B), which is representative of a functionality of the operating element (20). [5] Method according to one of the preceding claims, wherein the operating element (20) comprises an adjusting element. [6] Device for operating an input device (10) in a vehicle, wherein the input device (10) comprises at least one operating element (20) which can be operated by means of a predetermined operating actuation which comprises at least touching the operating actuation with a predetermined actuating element (30), wherein the device is designed to carry out a method according to one of claims 1 to 5. [7] Operating system (45) for a vehicle with a device according to claim 6 and an image capture device (50) which is designed - to capture an image or an image sequence which represents a movement and / or position of a given actuating element (30) in a given environment of the operating element (20) and - To forward image data of the captured image or image sequence to the device.

Images