Method and system for generating feedback for a user input in a vehicle

Abstract

The invention relates to a method for generating feedback for user inputs in a vehicle, there being provision for a detection device (4) which is configured to detect user inputs in a contactless manner as gestures in a three-dimensional spatial region in the interior (1) of the vehicle. A hand (11) of a user (10) in the interior (1) of the vehicle is detected by means of the detection device (4), this involving determination of a position of the hand (11) in the three-dimensional spatial region. A relative positional relationship between the position of the hand (11) and the windscreen (2) of the vehicle is determined and visual feedback is generated on the basis of the relative positional relationship, the feedback being output by means of a feedback device (3), in particular in the form of illuminating LEDs, which extends around the windscreen (2) of the vehicle, this involving activation of a region (6) of the feedback device (3) which depends on the relative positional relationship.

Description

[0001] METHOD AND SYSTEM FOR GENERATING FEEDBACK FOR USER INPUT IN A VEHICLE

[0002] The present invention relates to a method and a system for generating feedback for user input in a vehicle. In particular, visual feedback for gesture control can be generated, especially by means of a feedback device at the edge of the vehicle's windshield.

[0003] Modern vehicles often boast a wide range of functions, typically encompassing various vehicle and comfort features such as navigation system settings, climate control, seat adjustments, lighting settings, and the like, or infotainment system functions such as playing music, making phone calls, and so forth. A display may be provided for viewing and controlling these functions, for example, centrally located in the dashboard. This display can show the individual functions, menus, and the like, allowing them to be controlled, for instance, by tapping.

[0004] It is also known to control certain functions in a vehicle contactlessly using gestures, which is particularly advantageous when a user interface is displayed on a head-up display (HUD) in the windshield. For this to work, a user performs specific predefined gestures within the vehicle interior, such as in a defined area, for example, above the center console, which are then detected by a suitable sensor, such as a camera or a 3D sensor.

[0005] However, users often receive no feedback on executed gestures. Therefore, performing or even learning gestures for reliable control of a user interface can be difficult. Methods are known in which a successfully executed gesture is confirmed by feedback, such as a light pattern on the display, which also indicates the user interface. Other methods can display the current hand position using a cursor on the display, the position of which is determined, for example, by the direction the hand is pointing. However, such a small marker can often be difficult to see, especially without taking one's eyes off the road, thus potentially distracting the driver. The present invention aims to provide an improved solution for generating feedback for user input.In particular, feedback should be generated for gesture control in a vehicle.

[0006] The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0007] A first aspect of the invention relates to a method, particularly a computer-implemented one, for generating feedback for user input in a vehicle, wherein a detection device is provided which is configured to capture user input contactlessly as gestures in a three-dimensional spatial area in the interior of the vehicle. In the method, a user's hand is detected in the interior of the vehicle by means of the detection device, whereby the position of the hand in the three-dimensional spatial area is determined. A relative positional relationship between the position of the hand and the windshield of the vehicle is determined, and visual feedback is generated depending on this relative positional relationship.The feedback is output (displayed) by means of a feedback device that extends around the windshield of the vehicle, whereby an area of ​​the feedback device is activated which depends on the relative position relationship.

[0008] The aforementioned method, as described in the first aspect, is therefore based primarily on the ability to generate feedback for gestures. This feedback is output via a feedback device that runs around the windshield. Thus, feedback can be subtly perceived in the user's peripheral vision without requiring them to take their eyes off the road. The attention of both the user and the driver is therefore not impaired, which in turn contributes to driving safety, while simultaneously providing appropriate feedback for hand movements or gestures. These gestures can, for example, be used to capture user input for controlling vehicle functions via a user interface displayed on the windshield using a head-up display.

[0009] The procedure described in the first aspect thus allows for a balance between providing valuable information and maintaining the driver's concentration in the vehicle. This is particularly advantageous for the use of advanced technology integration in the form of touchless interaction and gesture control. It provides a novel and subtle approach to user feedback, avoiding complications and the mixing of different types of feedback resulting from various control mechanisms.

[0010] In summary, the system offers a subtle yet consistent way to provide the driver with feedback without distracting them from the road. The feedback device, for example in the form of dimmable LEDs, surrounds the vehicle's windshield and is in constant interaction with the detection device, such as a 3D camera that monitors the vehicle's interior. A detected hand is then dynamically indicated by illuminated LEDs, which also show the hand's movement and, where applicable, correctly executed and recognized gestures, as will be described later.

[0011] The term "feedback" used here refers specifically to a system response to a user action. Specifically, feedback is given for hand movements, including gestures. In principle, feedback can encompass any type of perceptible stimuli, such as acoustic or optical (visual) feedback in the form of sounds, displays, and the like. However, feedback can also be "visual feedback," which refers specifically to feedback that a user can see. The "feedback device" as used in the invention can therefore also be referred to as a "display device." The term "feedback" can also be used synonymously with "feedback."

[0012] The term "detection device" used here refers specifically to a device that can detect objects in three-dimensional space without physical contact and determine their position. In particular, the detection device can detect a user's hand. For example, optical methods can be used to detect a user's hand in space. The detection device can consist of one or more parts, depending on the desired detection area. For example, a (2D) camera or a 3D sensor may be used. The detection area is defined as the area within which events or changes can be perceived by the detection device; in the context of this disclosure, this refers specifically to the area (or more precisely, the three-dimensional space) in which a hand can be detected.This can essentially encompass the entire vehicle interior or at least an area around the driver / passenger. In the case of cameras or other optical detection devices or sensors, this can also be referred to as the "field of view." The detection device can be installed in a vehicle, particularly in the vehicle's interior.

[0013] The term "three-dimensional space" used here refers specifically to a space that can be described by three-dimensional coordinates. A position within this three-dimensional space has unique three-dimensional coordinates. A hand position or gesture, in particular, can be captured in three-dimensional coordinates.

[0014] The term "user interface" or "graphical user interface" used here refers in particular to a graphical representation of controls that are linked to a specific function and allow a user to control that function. The user interface (UI or GUI) can contain controls such as input fields, buttons, icons, sliders, toolbars, selection menus, and the like, which a user can operate, particularly in the context of the present invention, without touching them. The (graphical) user interface can also be referred to as the (graphical) user interface. The GUI can be displayed, in particular, on a display device such as a head-up display or any other display, screen, monitor, and the like.

[0015] The term "user input" as used here refers specifically to a user's interaction with the graphical user interface. This can include simply moving a pointer (also called a "cursor") on the graphical user interface, or controlling a function, such as selecting and activating a control (especially by "clicking" or "double-clicking"), navigating through the user interface (e.g., "scrolling"), changing the appearance of objects or controls, including moving objects (especially "drag and drop").

[0016] The term "vehicle" used here refers in particular to a passenger car, including all types of motor vehicles, hybrid and battery-powered electric vehicles, as well as vehicles such as sedans, vans, buses, trucks, delivery vans and the like.

[0017] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.

[0018] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive or and not an exclusive "or". For example, a condition A or B is satisfied by one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0019] The terms "ein" or "eine," as used here, are defined as "one or more." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0020] The term “plural” or “several”, as it may be used here, is to be understood in the sense of “two or more”.

[0021] The terms "configured" or "set up" to perform a specific function (and their respective variations) are understood within the meaning of the invention to mean that the corresponding device already exists in a configuration or setting in which it can perform the function, or at least that it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predetermined configurations or operating modes, so that configuration can be carried out by selecting one of these configurations or operating modes.Preferred embodiments of the method are described below, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the invention as desired.

[0022] In some embodiments, the hand's position is determined and processed in real time to generate real-time feedback. This allows the feedback to be generated in such a way that it is perceived by the user as instantaneous and fluid. In particular, the time between the detection of the hand and the output of the corresponding feedback ("latency") should be as short as possible, preferably a maximum of 10 ms (milliseconds). "In real time" also means, in particular, that a constant data stream of the detected hand's position data is processed for generating the feedback without being stored.

[0023] In some embodiments, a target point on the windshield is further determined by perpendicularly projecting the hand's position onto the windshield, or by determining the hand's pointing direction, with the target point on the windshield then being defined as the intersection of the pointing direction with the windshield. An activation point is then determined on the feedback device, the point on the feedback device being the one with the shortest distance to the target point. At least one area of ​​the display device, encompassing the activation point, is activated. Thus, a target point (with two-dimensional coordinates) on the windshield is first calculated from the three-dimensional coordinates of the hand.To control the feedback device, which runs along the edge of the windshield, a corresponding activation point is calculated—namely, the point on the feedback device that is closest to the target point on the windshield. Feedback is then provided at this point. This form of feedback intuitively reflects the hand's position in space, allowing the user to better orient themselves when performing gestures. A perpendicular projection onto the windshield can be calculated with minimal effort, while determining the pointing direction is more complex, but this can potentially improve the user experience.

[0024] In some designs, the brightness of the feedback signal is determined based on the distance between the activation point and the target point, with the brightness increasing as the distance decreases. In other words, if the user's hand (or the target point) is closer to the edge or pointing at a point nearer the edge, the feedback can be brighter than if the user's hand is more centrally located on the windshield. This allows for more precise feedback regarding the hand's current position.

[0025] In some embodiments, the windshield is divided into at least two zones. When generating feedback, data processing is limited to the zone of the windshield containing the target point. This increases the efficiency of the process by reducing the computational effort. The zones can be, for example, halves of the windshield, such as the left and right halves or the upper and lower halves, or, in the case of four zones, the four quadrants (upper left, upper right, lower left, lower right). Other zone divisions, such as thirds, are of course also conceivable.

[0026] In some embodiments, the feedback device's activated area is defined as a section extending along a display length on both sides of the activation point along an edge of the windshield. Displaying a "bar" instead of just a dot significantly improves visibility, especially since it is located in the user's peripheral vision. The bar's length can be adjusted to individual needs. A longer bar may be more easily perceived, but it provides a less accurate indication of hand position compared to a shorter bar. The bar can extend along one side of the windshield or, depending on the hand's position, wrap around two edges.

[0027] In some embodiments, a hand gesture is detected, and a confirming feedback signal is issued via the feedback device if the detected gesture can be matched to a valid gesture. It is therefore possible to provide not only dynamic feedback about the current hand position but also to indicate successful execution of a gesture. For example, the confirming feedback could be an animation, such as a dot or bar that travels completely or partially around the windshield, perhaps in a clockwise direction. The activation point described above could serve as the starting point. The duration of the confirming feedback could be, for example, approximately 2 to 3 seconds, such as 2.5 seconds.It goes without saying that any feedback can be considered as confirmation of a successful execution of a valid gesture, such as different lighting patterns, flashing, circular or symmetrical designs, colors, etc.

[0028] In some embodiments, a color is used for feedback, with the color depending on the user whose hand is detected by the sensor. This is useful, for example, when a driver and a passenger want to control a user interface. Different colors can then be used for feedback, depending on who is performing the gesture. This can be determined based on the hand's position in space. For example, a detected hand on the driver's side can be assigned to the driver, and a detected hand on the passenger's side to the passenger.

[0029] In some implementations, the speed at which the detected hand moves is determined, and the hand's position is only determined and feedback generated if the determined speed is below a predefined threshold. Conversely, no feedback (and potentially no determination of the hand's position) occurs if the speed exceeds the predefined threshold. This "speed filter" prevents unintended hand movements from generating feedback, as such movements (e.g., "gesturing") are typically faster than deliberate gestures used to control a user interface. This should be avoided to prevent unnecessary distractions and confusion. In particular, this "false positive" error must be avoided.A “false negative” error, i.e., the failure to recognize an intended gesture, should also be avoided, but is usually less critical with regard to potential distractions.

[0030] In some embodiments, the hand for which the feedback position is determined is one of a user's hands, or the hands of several users, whose position is closest to the windshield. This allows for a reliable determination of which hand of the driver (or passenger) is most likely the hand currently performing a gesture. A hand farther away from the windshield is less likely to be the performing hand. This also allows for the reliable ignoring of other hand movements in the background, i.e., at a greater distance from the windshield, while the hand is being detected.

[0031] In some embodiments, a user's hand is detected within a detection area, which is a limited three-dimensional space within the vehicle's interior. While it is theoretically possible to cover and monitor the entire vehicle interior with the detection device, allowing a gesture to be performed anywhere inside, it can be advantageous to detect a hand only within a limited area. This makes it easy to exclude unintentional hand movements from detection and reduces computational effort. For example, a portion of the steering wheel might be excluded from detection, as the driver's hands will likely remain in this area most of the time without making any gestures.It may also be possible to define the detection area as an area in front of a user (driver and / or passenger), around a user, or a central area above the armrest or center console. It may also be possible to adjust the detection area to individual needs.

[0032] A second aspect of the invention relates to a data processing system comprising at least one processor configured to perform the method according to one of the preceding claims, and at least one detection device configured to capture user inputs contactlessly as gestures in a three-dimensional spatial area in the interior of the vehicle, and a feedback device extending around the windshield of the vehicle and configured to output the feedback.

[0033] In some embodiments of the system, the feedback device comprises an array of light sources, in particular light-emitting diodes (LEDs), for outputting the (visual) feedback. These are preferably dimmable and / or multicolored, so that the brightness and / or color of the feedback can be adjusted as explained above. The array of LEDs can extend as a "band" around the windshield to display the feedback at any point along the edge of the windshield. It is understood that various configurations are possible, for example, as a directly visible band or as an indirectly illuminated light edge. In particular, it can be provided that the display device is solely designed for outputting the feedback and serves no other purpose, especially not as a display showing other content.

[0034] In some embodiments of the system, the detection device comprises at least one image detection device, in particular a camera and / or a 3D sensor. A camera can be used to easily determine the position of a user's hand in three-dimensional space. One or more cameras may be used. The camera can be an infrared camera. Advantageously, the at least one camera is a time-of-flight (ToF) camera, also called a time-of-flight sensor. By using such a 3D sensor device, the position of the hand in three-dimensional space and its movement or gestures can be directly detected. 2D sensors can also be combined to detect the position of the hand in three-dimensional space.

[0035] A third aspect of the invention relates to a computer program with instructions which, when executed on a system according to the second aspect, cause the system to execute the method according to the first aspect.

[0036] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example, the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules.

[0037] The system according to the second aspect can accordingly include a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the method or computer program or represents outputs of the computer program. The features and advantages described with regard to the first aspect of the invention also apply accordingly to the further aspects of the invention.

[0038] Further advantages, features and possible applications of the present invention will become apparent from the following detailed description in conjunction with the drawings.

[0039] This shows:

[0040] Fig. 1 schematically shows a system for generating feedback from the perspective of the vehicle interior;

[0041] Fig. 2 schematically shows a system for generating feedback in side view;

[0042] Fig. 3 shows an example of feedback for a first hand position;

[0043] Fig. 4 shows an example of feedback for a second hand position; and

[0044] Fig. 5 shows a status diagram for a method for generating feedback.

[0045] The same reference numerals are used throughout the figures for the same or corresponding elements of the invention.

[0046] A common challenge with touchless gesture control is the lack of immediate and direct feedback for the user. This problem is further complicated when such interaction technology is implemented in a vehicle, where safety in potentially dangerous traffic scenarios is paramount. In such contexts, it is crucial to minimize distractions caused by feedback. Another complicating factor is the lack of physical contact with a surface, which renders conventional haptic feedback methods ineffective. This separation between user and system leads to uncertainty. The user is unsure whether the system recognizes their gestures or whether they have executed them correctly. To address these challenges, the present invention provides a subtle yet easily understandable form of feedback for the user.A corresponding system for generating feedback for user input in the form of gestures is schematically illustrated in Figures 1 and 2. The gestures are performed by a user 10 with a hand 11 freely within the interior 1 of a vehicle. The position of the hand 11 is detected by a sensing device 4, in particular in the form of a 3D sensor, such as a ToF camera, which can, for example, be located in the base of the rearview mirror 5 to monitor the interior 1. The monitoring can be limited to a detection area 14, which can simply be the field of view of the sensing device 4 or, if necessary, further limited to improve recognition. The position data of the hand 11 is processed in real time, in particular by a processor 9, to output the feedback with the lowest possible latency (e.g., less than or equal to 10 ms).This creates a smooth experience for user 10. User 10 can be, in particular, the driver of the vehicle, but also a passenger.

[0047] A feedback device 3 is provided for output, in particular for displaying feedback. Specifically, a series of multicolored, dimmable light sources, such as LEDs, surround the windshield 2, which illuminate accordingly to symbolize the relative position of the hand 11 with respect to the windshield 2. In particular, this can support the control of a user interface, which is displayed on the windshield 2 by means of a head-up display (not shown).

[0048] The system calculates a target point 7 on the windshield 2 (in 2D coordinates) from the position of hand 11 (3D coordinates), as will be described in more detail below. The feedback is time-sensitive and therefore displays the movement of the detected hand 11 in real time. This allows the user 10 to always know if and where their hand 11 is located within the system, leading to a feeling of control, connection, and trust in the system. When the user 10 performs a gesture that is recognized by the system, the LEDs light up clockwise, starting from the currently displayed position of the hand, until they reach their initial position and indicate a correctly recognized gesture. This animation can last, for example, approximately 2.5 seconds. Afterward, the position of hand 11 is simply displayed again.

[0049] The area 6 of the feedback device 3 to be activated, i.e., in particular those LEDs that are to light up depending on the position of a detected hand 1 1, is determined as explained below. As mentioned, the interior 11 is continuously monitored by means of the detection device 4. It is checked, in particular using special software, whether hands are visible and whether a predefined gesture is being performed by these hands. If a hand 11 is detected, the 3D position of the hand 1 1 is first converted into a 2D point on the windshield 2, here called target point 7. This allows the shortest distance between the hand 11 and the windshield 2 to be determined, or the intersection of a pointing direction 12 of the hand 11 with the windshield 2.

[0050] It can now be further checked in which predefined zone 2-1, 2-2, 2-3, 2-4 of the windshield 2 the target point 7 is located. These zones 2-1, 2-2, 2-3, 2-4 can improve the efficiency of the method, especially if only LEDs adjacent to the corresponding zone (zone 2-2 in the example shown in Fig. 1) are considered for the feedback display. The system now identifies the LEDs closest to the hand position, or more precisely, to the target point 7, preferably in the corresponding zone. For this purpose, an activation point 8 is determined as the point of the feedback device 3 with the smallest distance 13 to the target point 7. An area 6 containing the activation point 8 is then activated.In area 6, which can extend along a display length (also called "offset") of approximately 5 cm in both directions from activation point 8, for example, the LEDs light up to signal the position of the user's hand 11 to the user 10. A single LED would not be clearly visible to the user 10, therefore activating a slightly longer area is necessary.

[0051] 6. Advantageous for visibility and user-friendliness.

[0052] To further improve the position display, the distance of 13 from the target point is increased.

[0053] The distance from target point 7 to activation point 8 (i.e., essentially the distance from target point 7 to the edge of the windshield 2) is indicated by the brightness. Specifically, the LEDs shine brighter the closer target point 7 is to the edge. To achieve this, zones 2-1, 2-2, 2-3, and 2-4 can be further subdivided into ten-percent segments. Within each of these zones, one segment represents the distance to the top (or bottom) edge, and another represents the distance to the side edge. The dimmest segment, for example, at only 10% brightness, is furthest from the edge, while the brightest segment, at 100% brightness, is closest to the edge. This improved dynamic range makes the feedback even more vivid and intuitive.

[0054] Examples of visual feedback of a hand position are shown in Figs. 3 and 4. In Fig. 3, the hand 11 is located near the lower edge of the windshield 2. The LEDs in a corresponding area 6 of the feedback device 3, which is determined as explained above, illuminate. If the user 10 now moves their hand upwards and to the left, as shown in Fig. 4, the LEDs illuminate accordingly in a different area 6. As can be seen in Fig. 4, the area 6 can also extend (consecutively or separately) along two sides of the windshield 2. For this purpose, a second activation point can be calculated, for example, over the next smaller distance 15 (see Fig. 1), corresponding to the activation point 8. It is understood that, in principle, other areas 6 can also be activated.For example, depending on individual user preferences, an area of ​​the feedback device 3 on the opposite side of the windshield 2 with respect to the activated area 6 can also be activated.

[0055] In general, there can be various types of settings that the user can change to tailor the feedback to their specific needs and further improve human-machine interaction. These settings can include, for example, the color of the LEDs, which can be set for both the driver and the passenger. This is helpful for specific user needs and for differentiating between the display of hands belonging to different people. It can also be possible to adjust the length of the display area (display length), for example, shorter or longer than the aforementioned 5 cm before and after the activation point (8), to accommodate the different visibility needs of various users, minimize distractions, and improve visibility.

[0056] Figure 5 summarizes the processes described above in a status diagram 100. Initially, the system is in standby mode (101). The interior space 1 is monitored. As long as no hand is detected, no LEDs are illuminated. When a hand 1 1 is detected (102), the system is activated (block 103) and the LEDs in area 6 illuminate as described above (104). With further hand detection (105), and especially with movement of the hand 1 1, area 6 changes accordingly in real time to indicate the current hand position (106). This continues with further detection of the hand 1 1 (107). As soon as no hand is detected (108), the LEDs switch off (109) and the system returns to standby mode (110).

[0057] The described procedure and system can be used in the following exemplary scenario: A user is driving a car. He wants to use gesture control to answer an incoming call. He lifts one hand from the steering wheel. As soon as the hand is detected by the time-of-flight camera, the lights around the windshield illuminate according to the hand's position. The user now knows that his hand has been correctly detected and also where it is located relative to the system, without taking his eyes off the road, as the visual feedback is perceived in his peripheral vision at the edge of the windshield. He then performs the gesture to answer the call. The lights move around the windshield and the call is answered. The user knows immediately that he has performed the gesture correctly, again without taking his eyes off the road.He also doesn't have to invest time in a complex visual interface or search for or press any buttons.

[0058] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents.

[0059] REFERENCE MARK LIST

[0060] 1 Vehicle interior

[0061] 2 Windscreen

[0062] 2-1, ..., 2-4 zones

[0063] 3 Feedback device

[0064] 4. Recording device

[0065] 5 Interior mirrors

[0066] 6. Activated area for displaying feedback

[0067] 7 Destination

[0068] 8 Activation point

[0069] 9 Control unit (processor)

[0070] 10 users

[0071] 11 Hand

[0072] 12 Pointing direction

[0073] 13 (smallest) distance to the edge

[0074] 14 Detection area

[0075] 15 mm distance to the edge

[0076] 100 Status diagram (Procedure for generating feedback)

[0077] 101, .. 110 Status or status transition in the status diagram 100

Claims

REQUIREMENTS 1. Method for generating feedback for user inputs in a vehicle, wherein a detection device (4) is provided which is configured to detect user inputs contactlessly as gestures in a three-dimensional spatial area in the interior (1) of the vehicle, wherein the method comprises: Detecting a hand (11 ) of a user (10) in the interior (1 ) of the vehicle by means of the detection device (4), whereby a position of the hand (11 ) in the three-dimensional spatial area is determined; Determining a relative positional relationship between the position of the hand (11) and the windshield (2) of the vehicle; and Generating visual feedback depending on the relative position relationship, wherein the feedback is output by means of a feedback device (3) which extends around the windshield (2) of the vehicle, whereby an area (6) of the feedback device (3) is activated which depends on the relative position relationship.

2. Method according to claim 1, wherein the position of the hand (1 1 ) is determined and processed in real time to generate the feedback in real time.

3. The method of claim 1 or 2, further comprising: Determining a target point (7) on the windscreen (2) by perpendicular projection of the position of the hand (11) onto the windscreen (2) or by determining a pointing direction (12) of the hand (11), wherein the target point (7) on the windscreen (2) is then determined as an intersection of the pointing direction (12) with the windscreen (2); and Determining an activation point (8) on the feedback device (3), wherein the activation point (8) is the point on the feedback device (3) which has the shortest distance (13) to the target point (7); wherein at least one area (6) of the display device (3) is activated which includes the activation point (8).

4. Method according to claim 3, wherein a brightness is determined depending on the distance (13) between the activation point (8) and the target point (7), with which gives the feedback, with the brightness increasing as the distance (13) decreases.

5. Method according to claim 3 or 4, wherein the windshield (2) is divided into at least two zones (2-1 , 2-2, 2-3, 2-4), wherein, when generating the feedback, data processing is limited to the zone (2-1 , 2-2, 2-3, 2-4) of the windshield (2) in which the target point (7) is located.

6. Method according to one of claims 3 to 5, wherein the area (6) of the feedback device which is activated for feedback is defined as an area (6) of the feedback device which extends along a display length on both sides of the activation point (8) along an edge of the windshield (2).

7. A method according to any one of the preceding claims, further comprising: Determining a gesture performed by hand (11); and issuing a confirming feedback via the feedback device (3) if the determined gesture can be matched to a valid gesture.

8. Method according to one of the preceding claims, wherein a color is determined with which the feedback is output, wherein the color is determined depending on a user (10) whose hand (1 1 ) is detected by means of the detection device (4).

9. Method according to one of the preceding claims, wherein a speed is determined at which the detected hand (11) moves, wherein the position of the hand (11) is determined and feedback is generated accordingly only if the determined speed of the hand (11) is below a predetermined threshold value.

10. Method according to one of the preceding claims, wherein the hand (11) for which the position for outputting the feedback is determined is one of the hands of a user or of the hands of several users whose position has the smallest distance to the windshield (2).

11. Method according to one of the preceding claims, wherein the detection of a hand (1 1 ) of a user (10) takes place in a detection area (14) which is a limited three-dimensional spatial area in the interior (1 ) of the vehicle.

12. Data processing system comprising at least one processor (9) configured to perform the method according to one of the preceding claims, and at least one detection device (4) configured to detect user inputs contactlessly as gestures in a three-dimensional spatial area in the interior (1) of the vehicle, and a feedback device (3) extending around the windshield (2) of the vehicle and configured to output the feedback.

13. System according to claim 12, wherein the feedback device (3) comprises a field with a plurality of light sources.

14. System according to claim 12 or 13, wherein the detection device (4) comprises at least one camera and / or a 3D sensor.

15. Computer program with instructions which, when executed on a system according to any one of claims 12 to 14, cause the system to execute the method according to any one of claims 1 to 11.

Images