Method and system for capturing user inputs
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- GESTIGON GMBH
- Filing Date
- 2025-11-26
- Publication Date
- 2026-06-04
Smart Images
Figure EP2025084283_04062026_PF_FP_ABST
Abstract
Description
[0001] gestigon GmbH 26.11.2025
[0002] 1
[0003] METHOD AND SYSTEM FOR CAPTURING USER INPUT
[0004] The present invention relates to a method and a system for capturing user input. In particular, user input can be captured without contact.
[0005] User interfaces, especially those displayed on a screen, screen, or similar device—for example, for interacting with a smart TV, projector, computer desktop, car's onboard computer, or even an aircraft's flight controls—are often designed to display a wealth of content, information, controls, and so on. Therefore, intelligent interaction and design concepts are necessary to avoid overwhelming users with cluttered interfaces. This becomes increasingly difficult the smaller the screen size or the greater the user's distance.
[0006] There are known concepts based on gesture control, which eliminate the need to touch a display. A device can then be used from a greater distance. For gesture control, a user moves their hand within a detection area in front of a device, where the hand is recognized by a detection device such as a camera or a 3D sensor (especially a time-of-flight camera). However, using gesture control directly in front of a device often means that part of the user interface is hidden behind the hand performing the gesture.
[0007] In general, a clean interface is desirable for a fully gesture-controlled application. In particular, various controls, such as those offering interaction options like "Back," "Menu," or "Close," should be located in areas that are not accidentally triggered. If these controls are always visible, however, there is a high probability that the hand will trigger unwanted interactions with them. This leads to flickering of the suspended elements and even to unwanted or accidental interactions with the system.
[0008] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0009] 2
[0010] For example, a clean user interface is desirable for a gesture-controlled game application, free from distracting elements. The user can move their hand within a detection area across the entire width and length of the physical display to activate or interact with different areas of the game. However, by default, options to exit the game or open the menu are also needed. If such controls are always visible and interactive, there is a high probability that they will be activated unintentionally, or at least that the controls, which appear to float above the actual game interface, will flicker.
[0011] Interaction concepts that incorporate the screen's surroundings can prevent parts of the user interface from being obscured and thus overlooked. This is often achieved by hiding some functional elements, such as menu controls, that are not needed during operation, for example, while interacting with a game. It is then also beneficial to expand the actual interaction area by triggering interactions at various points within the device's environment.
[0012] It is therefore known to incorporate a surrounding area of a display into the user interface for controlling certain functions. This involves utilizing the portion of a sensor's field of view, such as a time-of-flight camera, that extends beyond the actual display area. If movement is detected in this surrounding area, functions or controls that are normally hidden can be displayed in that area, enabling appropriate control. However, controlling such normally hidden elements is not always reliable.
[0013] The present invention is based on the objective of improving the capture of user input. In particular, it aims to improve the capture of user input in the area surrounding a display surface.
[0014] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0015] 3. In particular, it should be made more reliable, while also reducing the risk of unwanted interactions.
[0016] The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.
[0017] A first aspect of the invention relates to a method, particularly a computer-implemented one, for capturing user input on a graphical user interface displayed on the screen of a display device. The graphical user interface has at least one control element that is hidden from the screen. A user's hand is detected by means of a detection device, and at least one position of the hand is determined, both within a detection area. The detection area is a three-dimensional spatial region covered by the detection device and includes an interaction area that is associated with the screen.The system determines whether the user's hand is within an activation area. The activation area is a portion of the detection area adjacent to the interaction area and associated with a surrounding area adjacent to the display surface. At least one control element is then displayed on the display surface, so that when fully displayed, it can be selected to control an associated function. This display process only begins after it has been determined that the user's hand has been within the activation area for at least a predefined activation time, and it is only executed if an activation movement of the user's hand within the activation area is detected.
[0018] The aforementioned procedure, as described in the first aspect, is therefore based in particular on the fact that user input is captured not only on a display surface or a corresponding three-dimensional interaction area, for example, the area in front of the display surface, but also in a surrounding area of the display surface or the corresponding associated three-dimensional activation area. User interactions in this
[0019] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0020] 4
[0021] The surrounding area can be used to display at least one control element that is hidden when not in use, so that the entire display area can be used for the actual content, for example, of a game application. In other words, if a user moves their hand from the interaction area (or, in 2D, from the display area) beyond the edge into the activation area (or, in 2D, into the surrounding area), the otherwise hidden control element is displayed.
[0022] Only when the control is fully displayed, i.e., the display process is complete, can the control be selected to control an associated function, such as opening a menu, closing the running application, and so on. The display process is subject to conditions before it even begins and continues. Specifically, the hand must be in the activation area for at least a predefined activation time, for example, one second or less, such as 0.5 seconds. To then continue the display, the user must move their hand within the activation area; an activation movement must be detected.
[0023] In this way, the risk of unwanted interface interactions in gesture-controlled systems can be reduced or even completely avoided. In particular, it is highly likely that a user intends to intentionally display (and then select) a control when they move their hand into the activation area, i.e., beyond the edge of the actual user interface. If the display process does begin unintentionally, there is hardly any risk of the control being activated unintentionally, since it cannot be selected until it is fully displayed. The user therefore has sufficient time to cancel the display, for example, by moving their hand back into the interaction area.
[0024] The term "user interface" or "graphical user interface" used here refers specifically to a graphical representation on a display surface with controls, each of which is linked to a specific function and allows a user to control that function.
[0025] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0026] 5
[0027] A user interface (UI or GUI) can contain controls (UI elements), such as input fields, buttons, icons, sliders, toolbars, selection menus, and the like, which a user can operate, particularly in accordance with the present invention, without touching them, i.e., by gesture control. The (graphical) user interface can also be referred to as the (graphical) user interface. The control element can also be a collection of several controls, such as a taskbar. If a control element is "hidden," this means, in particular, that it is not visible (and therefore not selectable). Conversely, "displayed" means, in particular, that it is visible (on the display area). It can then be selected, in particular, to control a correspondingly assigned function.
[0028] The term "display area" as used here refers specifically to any surface or area on which the user interface can be graphically represented by a display device. This can be a display or a projection surface. The display area can have any shape. In particular, it can be flat, even, and rectangular, as with typical displays, or curved, polygonal, round, etc. However, a point on the display area can be described using two-dimensional coordinates.
[0029] The term "detection device" used here refers specifically to a device that can detect objects in three-dimensional space without physical contact and determine their position. In particular, the detection device can detect and locate a user's hand. For example, optical methods can be used to detect a user's hand in space. The detection device can consist of one or more parts, depending on the desired detection area. For example, it may use one or more cameras. A 3D sensor, such as a time-of-flight (ToF) camera (also called a ToF sensor), can be advantageously used as a detection device.
[0030] The term "three-dimensional space" used here refers in particular to a region defined by three-dimensional coordinates.
[0031] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0032] 6. A position in the three-dimensional space has unique three-dimensional coordinates. The coordinate system can be chosen arbitrarily. For example, a coordinate system of the detection device can be chosen, or a coordinate system with respect to the display surface. In particular, there is a relationship between the detection area and the display surface in order to assign the position of the hand to corresponding points on the display surface.
[0033] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.
[0034] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive or and not an exclusive "or". For example, a condition A or B is satisfied by any of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).
[0035] The terms "ein" or "eine," as used here, are defined as "one or more." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."
[0036] The term "plural", as used here, is to be understood in the sense of "two or more".
[0037] The terms “configured” or “set up” to fulfill a specific function (and respective variations thereof) are to be understood within the meaning of the invention as meaning that the corresponding device is already in a specific configuration or
[0038] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0039] 7
[0040] A setting exists in which it can perform the function, or at least it is configurable so that it can perform the function after appropriate adjustment. This configuration can be achieved, for example, by setting parameters of a process flow, or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predefined configurations or operating modes, allowing configuration by selecting one of these configurations or operating modes.
[0041] Preferred embodiments of the method are described below, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the invention as desired.
[0042] In some implementations, the control remains visible and selectable on the display area for a predetermined time after it appears. This predetermined time can be, for example, a few seconds, such as 1 to 5 seconds, and in particular 2 seconds. Within this time, a user must interact with the now-displayed control, for example, by selecting it (i.e., pressing it) to control the associated function, or at least by pointing at it. In the latter case, the control can then remain visible. If no interaction occurs within the predetermined time, or if the user releases it, the control disappears again and can no longer be selected.
[0043] In some designs, the activation time begins when the user's hand moves beyond a boundary between the interaction area and the activation area. The user can therefore initiate the display by moving their hand out of the interaction area (or the display surface). Since, as mentioned, the display only begins after the activation time has elapsed, accidental movements beyond the edge do not result in any unintended activation of the control.
[0044] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0045] 8
[0046] In some embodiments, the control is displayed in a portion of the display area that corresponds to a specific part of the activation area where the hand's position is detected. This means, for example, that a control is displayed at or within a portion of the display's edge where the user's hand has moved from the display area (or interaction area) into the surrounding area (or activation area). For instance, a control can be displayed at or from the bottom of the display area when the user's hand moves downwards out of the interaction area and is detected in the lower part of the activation area.
[0047] In some embodiments, an activation movement is recognized as a movement in which the hand's position within the activation area moves away from the interaction area. Thus, movements parallel to or towards the edge of the display surface are not recognized as activation movements. It is understood that this could alternatively be the case in other embodiments. However, it has proven advantageous and intuitive to recognize an activation movement when the hand's movement outwards from the display surface continues in the surrounding area, i.e., in a direction away from the display surface, for example, as if the user were pulling the edge outwards.
[0048] In some embodiments, the display of the control element during the display phase depends on the hand's position within the activation area. For example, in the case of an outward activation movement, as described above, the display of the control element can change as the hand's distance from the interaction area increases. The display can then be designed as an animation.
[0049] In some implementations, the progress of the fade-in process is indicated by the control's display. This provides the user with feedback on how far the fade-in has progressed. Conversely, this can also show how far or how long the hand has been moved.
[0050] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0051] 9 still needs to be moved to complete the fade-in process. The progress can be represented in various ways, such as a bar (e.g., around a circular control), the size of the control until it reaches a final size, the color of the control, or any other visual effect.
[0052] In some embodiments, the width of the surrounding area is at least 5%, preferably at least 10%, of a corresponding dimension of the display area. For a rectangular display area, the width of the surrounding area can therefore constitute a portion of the height or width of the rectangle. The wider the surrounding area, the more diverse the design possibilities for gestures that can be performed there, for example, in the form of the activation movement described. A wider surrounding area may also facilitate operation, as more space is available for gestures, allowing them to be performed over a larger area. The width of the surrounding area is preferably essentially the same throughout, but may, under certain circumstances, be limited, for example, by the boundaries of the detection range of the detection device.It can also happen that the surrounding area is wider in some parts than in others, for example due to a conical shape of the detection area.
[0053] In some implementations, the user interface includes an indicator for a hidden control, displayed at the edge of the screen where the hidden control can be made visible. Such an indicator is helpful for users, drawing their attention to the presence of a hidden control or its position, thus improving usability. It could be, for example, a small part of the control itself, a colored marker such as a dot, a stripe, or a glow. Naturally, there are many design possibilities. It is also understood that a clean, uncluttered screen may be preferred. That is, such an indicator does not necessarily have to be displayed but can be designed according to personal preference. For example, experienced users might prefer a simpler, less cluttered screen.
[0054] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0055] 10
[0056] Users often know that there are hidden controls and where they are located.
[0057] In some embodiments, the position of a pointer on the display surface or on the adjacent surrounding area is further determined by perpendicular projection of the hand's position or by the hand's pointing direction onto the display surface or the adjacent surrounding area. The projection can be, in particular, an orthogonal projection, whereby the hand's position in three-dimensional space is mapped onto two-dimensional coordinates of the display surface. The pointer indicates, in particular, a location on the display surface that is determined by the hand's position. Thus, by moving the pointer, a control element on the graphical user interface can be selected by correspondingly moving the hand.
[0058] As mentioned, the pointer's position can be determined by an orthogonal projection of the hand's position from three-dimensional space onto the two-dimensional display surface. The hand's position can be defined by the point on the hand closest to the display surface, as explained below. This could also be a fingertip pointing towards the display surface. Alternatively, instead of an orthogonal projection, the pointing direction of a finger, such as an extended index finger, can be determined, and the pointer's position on the display surface can be calculated by extending this finger in the same direction.
[0059] The user's hand position can be determined as the position of a point on the user's hand that is closest to the display area. To allow for unambiguous determination of the user's hand position, it is advantageous to select a point on the hand that represents the hand's position. While in principle any point on the hand could be chosen as the hand's position, such as the center of the hand, it is advantageous to use the point on the hand that is closest to the display area for determining the position. This simplifies the calculation of the hand's position. Furthermore,
[0060] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0061] 11. The foremost point of the hand, from the user's point of view, is considered particularly relevant for input on the display area.
[0062] A second aspect of the invention relates to a data processing system, in particular for capturing user input on a graphical user interface. The system comprises at least one processor configured to perform the method according to the first aspect of the invention. The system also comprises at least one display device configured to display a graphical user interface on a display surface, and a detection device configured to detect a user's hand in a three-dimensional space and to determine the hand's position within that space.
[0063] In some embodiments of the system, the detection device comprises at least one image detection device, in particular a camera. A camera can be used to easily determine the position of a user's hand in three-dimensional space. One or more cameras may be used. The camera can be an infrared camera. Advantageously, the at least one camera is a time-of-flight camera (ToF camera or ToF sensor), i.e., a 3D sensor device. By using such a 3D sensor device, the position of the hand in three-dimensional space and its movement can be directly detected. Two-dimensional sensors can also be combined to detect the position of the hand in three-dimensional space.
[0064] A third aspect of the invention relates to a computer program with instructions which, when executed on a system according to the second aspect, cause the system to execute the method according to the first aspect.
[0065] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can be stored as a file on a
[0066] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0067] 12
[0068] A computer program must be stored on a data processing unit, particularly on a server, and be downloadable via a data connection, such as the internet or a dedicated data connection, like a proprietary or local network. Furthermore, the computer program may consist of multiple interacting individual program modules.
[0069] The system described in the second aspect may accordingly have a program memory in which the computer program is stored. Alternatively, the system may also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the procedure or computer program or represents outputs of the computer program.
[0070] The features and advantages explained in relation to the first aspect of the invention also apply accordingly to the other aspects of the invention.
[0071] Further advantages, features and possible applications of the present invention will become apparent from the following detailed description in conjunction with the drawings.
[0072] This shows:
[0073] Fig. 1 schematically shows a system for capturing user input according to one embodiment;
[0074] Fig. 2 schematically shows a method for capturing user input according to one embodiment; and
[0075] Fig. 3 shows a control element during the fade-in process.
[0076] The same reference numerals are used throughout the figures for the same or corresponding elements of the invention.
[0077] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0078] 13
[0079] Figure 1 schematically illustrates a system 1 for capturing user input on a graphical user interface 9. A graphical user interface (GUI) 9 with at least one control element 10 is displayed on a display area 5 of a display device 4, such as a display. The control element 10 is initially hidden and can be displayed using the method described below, particularly with reference to Figure 2.
[0080] User input is captured without contact; that is, the system allows user input via gesture control (a "gesture-based system" or "gesture control-based system"). For this purpose, a detection device 2 is provided, which detects a user's hand 11 and determines its position in three-dimensional space. The detection device 2 can, for example, be a 3D sensor such as a ToF camera, or it can comprise optical 2D sensors.
[0081] The field of view of the detection device 2 is referred to as the detection area 3 and is the three-dimensional spatial area covered by the detection device 2 in which a user's hand 11 can be detected in real time. A control unit ("controller"; not shown) may also be provided, which calibrates the system 1. In particular, the controller knows the physical position and size of the display surface 5 (in world coordinates) so that it can geometrically relate the detection data of a hand (e.g., finger positions) to the surface positions. The controller receives the data from the detection device 2 and converts it into suitable inputs for the GUI application.
[0082] System 1 allows the user to control a pointer with their hand 11, which represents a logical position on the display surface 1 or the user interface 4. The pointer may have a visual appearance, but it does not have to. The position of the hand 11, as detected by the sensing device 3, determines the position of the pointer on the display surface 5. This can be determined, for example, by orthogonally projecting the user's hand 11 onto the display surface 5 or an adjacent area 6, or by a pointing direction. It is particularly important to note that the
[0083] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0084] 14
[0085] Finger position is a position in three-dimensional space (detection area 3), while pointer position is a value in two-dimensional space (display area 1 or screen). Therefore, a corresponding three-dimensional interaction area 7 is assigned to the two-dimensional display area 5 (also shown two-dimensionally here due to the schematic representation).
[0086] The present invention describes a gesture-based system 1 for capturing user input, in which the capture device 3 covers more space than is necessary for the actual interaction with the graphical user interface 9 displayed on the screen 5 within the interaction area 7. This means that gestures made by the user's hand 11 are also recognized outside the interaction area 7, thus providing the user with additional possibilities for interacting with the system 1. For this purpose, an ambient area 6 borders the screen 5, or – in three-dimensional space – an activation area 8 borders the interaction area 7.
[0087] Preferably, the surrounding area 6 surrounds the display area 5, i.e., it extends around the edge of the display area 5. The width of the surrounding area 6 is preferably at least 10% of the width or height of the display area 5. The larger the surrounding area 6, the more interaction possibilities can be implemented. However, there may be cases where this cannot be guaranteed everywhere, particularly for technical reasons. For example, some areas, especially corners, may not be covered due to limitations of the detection area 3.
[0088] The surrounding area 6 is included in the calibration of system 1. This can proceed, for example, as follows: First, the exact position of the display area 5 is measured and set relative to the detection device 2. The size and aspect ratio of the display area 5 are communicated to system 1 beforehand, so that system 1 knows where the surrounding area 6 begins. After the data exchange regarding the actual size of the display area 5 is complete, a boundary frame is created through calibration, which defines the surrounding area 6.
[0089] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0090] 15
[0091] The environment area 6 is used in particular to display a control element 10, which is hidden when not in use. Specifically, the user's hand 11 is detected within environment area 6 for this purpose. While displaying a control element 10 can, in principle, be independent of the position of the hand 11 in the activation area 8, or of the point where the hand 11 leaves the interaction area 7, it is advantageous to take the position into account. This allows controls 10 to be displayed more selectively, for example, only those in the lower part of the display area 5, if the hand 11 moves downwards out of the display area 5. System 1 can therefore be configured to detect where the hand 11 leaves the interaction area 7 and enters the activation area 8. This position can then be stored, particularly if the hand 11 leaves the detection area 3.
[0092] It is understood that the surrounding area can have six different configurations. For example, it can encompass individual areas at the edge of display area 5, such as only the area below display area 5, or all four edges. It is also possible to define specific, personalized smaller areas that can then be calibrated accordingly.
[0093] The activation area 8 can have different sizes and shapes, depending on the number and field of view of the sensors of the detection device 2 and on individual needs. Within this area, it is particularly possible to track the precise position, movement, and speed of the hand. This data offers new possibilities for interaction, where the position of the hand can be translated into new interaction concepts.
[0094] However, an unintentional display can impair usability, as it causes flickering. It can also lead to unintentional activation of control 10. To prevent this, activation conditions are defined that must be met in order for control 10 to be fully displayed and then selected. In particular, the user's hand 11 must be in the activation area 6 for at least a predefined activation time before the display begins. The display process will then only continue or be completed if a
[0095] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0096] 16
[0097] Activation movement is performed within the activation area, for example, a movement outwards. The display can be stopped at any time if one of the conditions is no longer met. In particular, it is stipulated that the selection must then be made within a specified time; otherwise, control 10 will be hidden again.
[0098] A corresponding method 100 for capturing user input on a graphical user interface 9 is described with reference to Fig. 2. The arrows represent possible relationships between the individual stages of display, which depend in particular on the position of the user's hand 11. Method 100 begins with a standard view if no hand is detected (101). From this point, control of an application can occur if the hand is detected in the interaction area 7 (or the display area 5) (102). The hand can then be detected further in the activation area 8 (or the surrounding area 6) (103), here at the bottom edge. The display of a control element 10 then begins with a representation 14 if the hand is detected in the activation area 8 for at least one activation time, e.g., 0.5 seconds (104).The display 15 moves further into the display area 5 as the hand continues to move downwards (105) until the display of control 10 is complete (106). This control can then be selected within a predefined time, e.g., 2 seconds, and thus an associated function can be controlled, or at least held, e.g., by pointing at it without selecting it (107). Otherwise, it disappears again (108). The display can also be canceled at any time if the hand moves back from the activation area 6 into the interaction area 7 (109, 110). This prevents unwanted or accidental interactions with control 10.
[0099] An example of the behavior of a control element 10 is illustrated in Fig. 3. Initially, the control element 10 is hidden. Then it can either be completely invisible or—as shown in Fig. 3a—slightly highlighted by a cue 12 to guide the user to the correct area. The downward movement of the hand 11 from the display area 5 over the edge 16 into the surrounding area 6 (or, correspondingly, over a boundary between the
[0100] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0101] 17
[0102] The interaction area 7 and the activation area 8) can be designed to act like pulling a string to reveal (unhide) the hidden control. The user must then keep their hand 11 in the surrounding area 6 for a predefined activation time, e.g., 0.5 seconds, before the unhide process begins. During the actual unhide process, the hand movement is then correlated with an animation, the appearance of which depends on how far the hand moves from the edge 16. This is evident in Figures 3b to 3d from the upward movement of the hand's position. This can also be accompanied by a change in color, a "flash," a change in size, or similar effects. In Figures 3b and 3c, illustrations 14 and 15 also include a progress indicator 13, which shows the user the progress of the unhide process.In 3D, control 10 is finally fully displayed and thus selectable. Specifically, the user has the option to select or at least hold control 10 within a predefined time, e.g., 2 seconds. Otherwise, control 10 is hidden again.
[0103] The following are examples of how activation area 8 and environment area 6 can be used in gesture-controlled systems.
[0104] Displaying controls allows for the activation of additional options. This enables the creation of user interfaces with a clean and uncluttered appearance. No cluttered menu items distract the user from their task, such as 3D object modeling. As soon as the user moves their hand downwards from the user interface, leaving the area in front of the screen (the interaction area), a menu appears. In this menu, the user can perform various actions, such as saving or exporting a modeled 3D object.
[0105] The activation area itself can also be used for control. A user is working with a music app. Some UL elements suggest they move their hand to the right, out of the user interface (i.e., from the interaction area into the activation area). When they do this, a
[0106] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0107] 18
[0108] A slider extends vertically along the edge of the user interface, allowing the user to adjust the music volume. By moving their hand upwards next to the display area within the activation zone, they increase the volume. Once they move their hand back into the interaction zone, the new volume level is set and the slider disappears.
[0109] Special effects can be used to enhance usability. For example, the user's attention can be drawn to a glowing line at the bottom of the user interface. As the user moves their hand toward this line, the glow intensifies, indicating that there is a hidden element within the interface. When the hand extends beyond the edge of the display area, a "close" icon appears. However, the user still needs to complete something in the application, so they move their hand back to the interaction area in front of the display. Finally, when they decide to close the application, they move their hand back to the bottom edge, where they were previously alerted to the hidden control. This can feel like pulling a string.If he continues to move his hand, i.e., if he "continues to pull the string", the "close" symbol for closing the application will appear and he can close the application by selecting it.
[0110] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents.
[0111] RGTH G251154WO12616PT gestigon GmbH November 26, 2025
[0112] 19
[0113] REFERENCE MARK LIST
[0114] 1. System for capturing user input
[0115] 2. Recording device
[0116] 3. Detection area
[0117] 4 Display unit
[0118] 5 Display area
[0119] 6 Surrounding area
[0120] 7 Interaction area
[0121] 8 Activation area
[0122] 9 graphical user interface
[0123] 10 Control element
[0124] 11 Hand
[0125] 12 Note
[0126] 13 progress indicators
[0127] 14. Representation of the control element
[0128] 15. Representation of the control element
[0129] 16 Rand
[0130] 100 methods for capturing user input
[0131] 101 - 110 individual phases of the process 100
[0132] RGTH G251154WO12616PT
Claims
gestigon GmbH 26.11.2025 20 REQUIREMENTS 1. Method (100) for capturing user input on a graphical User interface (9), which is located on a display surface (5) of a Display device (4) is displayed, with the graphic The user interface (9) has at least one control element (10) which is hidden from the display area (5), the method comprising: Detecting a user's hand (11) and determining at least one position of the hand (11) by means of a detection device (2) in a detection area (3), wherein the detection area (3) is a three-dimensional spatial area covered by the detection device (2) and which includes an interaction area (7) which is assigned to the display area (5); Determine whether the user's hand (11) is located in an activation area (8), wherein the activation area (8) is part of the detection area (3) which is adjacent to the interaction area (7) and is associated with an environment area (6) adjacent to the display area (5); and Displaying at least one control element (10) on the display area (5) so that, when fully displayed, it can be selected to control an associated function, wherein i) the display only begins when it is determined that the user's hand (11) is in the activation area (8) for at least a specified activation time, and ii) the display is only executed when an activation movement of the user's hand is detected in the activation area (8).
2. Method according to claim 1, wherein the control element (10) remains displayed and selectable only for a predetermined time after being displayed on the display surface (5). RGTH G251154WO12616PT gestigon GmbH 26.11.2025 21 3. Method according to claim 1 or 2, wherein the activation time begins when the position of the user's hand crosses a boundary between the interaction area (7) and the activation area (8).
4. Method according to one of the preceding claims, wherein the control element (10) is displayed in a part of the display area (5) which is assigned to a corresponding part of the activation area (8) in which the position of the hand (11) is detected.
5. Method according to one of the preceding claims, wherein the activation movement is recognized as a movement in which the position of the hand (11) in the activation area (8) moves away from the interaction area (7).
6. Method according to one of the preceding claims, wherein a representation of the control element (10) during the display depends on the position of the hand (11) in the activation area (8).
7. Method according to claim 6, wherein the progress of the display is indicated by the representation of the control element (10).
8. Method according to one of the preceding claims, wherein a width of the surrounding area (6) is at least 10% of a corresponding dimension of the display area (5).
9. Method according to one of the preceding claims, wherein the user interface (9) has a reference to a hidden control element (10) which is displayed at a location on the edge of the display area (5) where the hidden control element (10) can be displayed.
10. Method according to one of the preceding claims, wherein a position of a pointer on the display surface (5) or on the adjacent surrounding area (6) is determined by perpendicular projection of the position of the hand (11) RGTH G251154WO12616PT gestigon GmbH 26.11.2025 22 or is determined by a pointing direction of the hand (11) towards the display area (5) or towards the adjacent surrounding area (6).
11. System (1) for data processing, comprising at least one processor configured to perform the method according to one of the preceding claims, and at least one display device (4) configured to display a graphical user interface (9) on a display surface (5), and a detection device (2) configured to detect a user's hand (11) in a three-dimensional spatial area and to determine the position of the hand (11) in the three-dimensional spatial area.
12. System according to claim 11, wherein the detection device (2) comprises at least one camera or at least one 3D sensor device.
13. Computer program with instructions which, when executed on a system (1) according to claim 11 or 12, cause the system to execute the method (100) according to any one of claims 1 to 10. RGTH G251154WO12616PT