Performing actions in response to hovering over an input surface
The hover touch interface enhances user interaction by using proximity sensing and depth detection to allow actions on touchscreens through floating inputs, overcoming the limitations of traditional touchscreens that require physical contact.
Patent Information
- Application Number
- DE102017102691
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2016-03-03
- Filing Date
- 2017-02-10
- Publication Date
- 2025-07-17
- Estimated Expiration
- 2037-02-10
AI Technical Summary
Traditional touchscreens require direct physical contact for user interaction, limiting the number of actions that can be performed in response to displayed information.
Implementing a hover touch interface that utilizes proximity sensing and depth detection to allow users to perform actions by floating an input instrument, such as a finger or stylus, above the screen, enabling depth and touch combinations for additional user interface operations without requiring navigation into sub-menus or remote inputs.
Enables enhanced user interaction by allowing multiple actions to be performed without physical contact, improving intuitiveness and efficiency in interacting with display content.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
BACKGROUND
[0001] A touch screen is an input device, typically located on top of an electronic visual display of an information handling device. A user can provide input or control the information handling system through single or multi-touch gestures by touching the screen with a special stylus and / or one or more fingers. The user can use the touch screen to respond to what is displayed and to control how it is displayed, for example, zooming to increase text size. The touch screen allows the user to interact directly with what is displayed, instead of using a mouse, touch pad, or other input device.
[0002] Touch screens are common in devices such as game consoles, personal computers, tablet computers, electronic dialers, and smartphones. They can also be attached to computers or used as network connection devices. Touch screens are also used in other types of information handling devices such as smartphones, personal digital assistants (PDAs), and electronic book readers.
[0003] The popularity of smartphones, tablets, and many types of information devices is driving the demand and adoption of conventional touch screens for wearable and functional electronics. Touchscreens are used in the medical field and heavy industry, as well as for automated teller machines (ATMs) and fully automated points of sale (VARs) such as museum displays or room automation, where other input devices do not enable suitable intuitive, fast, or accurate user interaction with the display content.
[0004] Traditional touch screens require the user to touch the screen to perform an action, which limits the number of actions that can be performed in response to the information displayed.
[0005] EP 2 445 182 A2 discloses a mobile terminal comprising a touchscreen for displaying and receiving information and a controller for controlling the touchscreen to display a lock screen when the mobile terminal enters the lock screen state, and for controlling the touchscreen to display a preview image of each of at least one application that was running before entering the lock screen state when a proximity touch input was received on the displayed lock screen. When a pointer, such as a user's finger, approaches the touchscreen, a proximity sensor located in or near the touchscreen detects the approach and outputs a proximity signal. The proximity sensor may be configured to output a proximity signal corresponding to the distance between the pointer approaching the touchscreen and the touchscreen.
[0006] From EP 3 627 302 A1, published on March 25, 2020, a method for controlling a device is known, comprising determining three-dimensional (3D) location information of a floating input received by the device, selecting at least one of a plurality of tasks associated with an operation of an application executed by the device based on the operation of the application and the 3D location information of the floating input, and executing the at least one selected task by retrieving a class of execution input corresponding to the floating input.
[0007] EP 2 701 056 A2 discloses a device and method for operating a pen function in an electronic device. A pen detection panel detects a touch pen according to a set mode. A control unit collects pen status information. A storage unit stores a pen function table that provides a pen function command corresponding to the collected information according to the set mode. The display panel displays information corresponding to the execution of a function corresponding to the pen function command. SUMMARY
[0008] It is an object of the present invention to enable an improved execution of actions using an input instrument hovering over an input surface.
[0009] This object is solved by the subject matter of main claim 1 and the independent claims 7 and 8, which define the present invention.
[0010] Preferred embodiments of the present invention are the subject of the subclaims. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] This disclosure may be better understood by reference to the accompanying drawings, in which: Fig. 1 is a block diagram of a data processing system capable of carrying out the methods described herein; Fig. 2 an enlargement of the environment of the information handling system shown in Fig. 1, to illustrate that the methods described herein can be carried out on a wide variety of information handling systems operating in a network environment; Fig. 3A is a component diagram showing a user using a hover touch interface with a touch-enabled laptop system; Fig. 3B is a component diagram showing a user using a hover touch interface with a touch-enabled tablet system; Fig. 4 is a flowchart showing steps used in setting hover touch preferences of a system; Fig. 5 is a flowchart showing high-level steps implementing a hover touch interface; Fig. 6 is a flowchart showing steps performed to handle a possible hover touch action; and Fig. Figure 7 is a flowchart showing further steps handling a motion-based hover touch action. Detailed description
[0012] Fig. Figures 1-7 show an approach that provides a way to perform additional user interface operations without navigating submenus or requiring inputs unrelated to the current task. Air gestures can be used for navigation, but cannot combine depth and touch. The approach enables hover touch by proximity sensing over a display or keyboard. Using sensors such as proximity sensing devices or a camera, the system maintains a z-coordinate value for a sensing input. When a user hovers an input device, such as the user's finger or a stylus, over the screen, the user can perform an action that is distinct from a current press / key / click of the screen location. The approach utilizes depth and touch combinations.In one embodiment, the user varies the depth or distance the user hovers over the screen surface to activate different actions. For example, suppose the user has selected some text. If the user clicks again, the cursor position would be moved and the selection would be reset. Using this approach, the user could hover to activate a context-sensitive menu for additional hover-touch selections. For another example, suppose the user has an image displayed on the screen. Hovering close to the screen over the image would zoom it in fully, while hovering farther away from the screen would zoom it in a smaller amount.
[0013] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will further be understood that the terms "comprising" and / or "comprising," when used in this specification, specify the presence of features, integers, steps, acts, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, acts, elements, and / or components, and / or groups thereof.
[0014] The corresponding structures, materials, acts, and equivalents of all means or steps plus functional elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The detailed description has been presented for purposes of illustration, but is not intended to be exhaustive or limiting of the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain the principles of the invention and its practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
[0015] As will be appreciated by one skilled in the art, aspects may be embodied as a system, method, or computer program product. Accordingly, aspects may take the form of a fully hardware implementation, a fully software implementation (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, collectively referred to herein generally as a "circuit," a "module," or a "system." Furthermore, aspects of the present disclosure may take the form of computer program products embodied in one or more computer-readable media having computer-readable program code embodied therein.
[0016] Any combination of one or more computer-readable media may be used. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing.More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction-executing system, by or in connection with an apparatus, or by or in connection with an appliance.
[0017] A computer-readable signal medium may include a propagating data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagating signal may take a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combinations thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that can communicate, propagate, or transport a program for use by or in connection with an instruction-executing system, apparatus, or device. As used herein, a computer-readable storage medium does not include a computer-readable signal medium.
[0018] Computer program code for performing operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++, or the like, and conventionally executable programming languages such as the "C" programming language or similar programming languages. The program code may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote server.In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made through an external computer (e.g., through the Internet using an Internet service provider).
[0019] Aspects of the present disclosure are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus(es), (systems), and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of the blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions.These computer program instructions may be provided by a processor of a general purpose computer, a special purpose computer, or other programmable data processing device to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce means for performing the functions / actions specified in the flowcharts and / or the block or blocks of the block diagram.
[0020] These computer program instructions may also be stored in a computer-readable storage medium that can guide a computer, other programmable data processing apparatus, or other devices to perform functions in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture, including the instructions that perform the functions / acts specified in the flowcharts and / or the block or blocks of the block diagram.
[0021] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable devices, or other devices to produce a computer-executed process, such that the instructions executing on the computer or other programmable devices provide processes for performing the functions / actions specified in the flowchart and / or the block or blocks of the block diagram.
[0022] The following detailed description will generally follow the summary as set forth above, and further explain and expand the definitions of the various aspects and embodiments as necessary. At this point, the detailed description will first describe a computing environment in Fig. 1 suitable for implementing software and / or hardware techniques associated with the disclosure. A network environment is described in Fig. 2 as an extension of the basic computer environment to emphasize that modern computer techniques can be executed across multiple discrete devices.
[0023] Fig. 1 illustrates an information handling system 100, which is a simplified example of a computer system capable of performing the computer operations described herein. The information handling system 100 includes one or more processors 110 coupled to a processor interface bus 112. The processor interface bus 112 connects the processors 110 to a northbridge 115, also known as a memory controller hub (MCH). The northbridge 115 connects to the system memory 120 and provides means for the processor(s) 110 to access the system memory. A graphics controller 125 is also connected to the northbridge 115. In one embodiment, a PCI Express bus 118 connects the northbridge 115 to the graphical controller 125. The graphical controller 125 is connected to a display device 130, such as a computer monitor.
[0024] The northbridge 115 and the southbridge 135 are interconnected using bus 119. In one embodiment, the bus is a direct media interface (DMI) bus that transfers data at high speed in either direction between the northbridge 115 and the southbridge 135. In another embodiment, a peripheral component interconnect (PCI) bus connects the northbridge and the southbridge. The southbridge 135, also known as the I / O controller hub (ICH), is a chip that generally performs capabilities that operate at slower speeds than those provided by the northbridge. The southbridge 135 typically provides various buses used to connect different components.These buses include, for example, PCI and PCI Express buses, an ISA bus, a system management bus (SMBus or SMB), and / or a low pin count (LPC) bus. The LPC bus often connects low-bandwidth devices, such as a boot ROM 196 and legacy I / O devices (using a Super I / O chip). The legacy I / O devices (198) can include, for example, serial and parallel ports, keyboards, mice, and / or a floppy disk controller. The LPC bus also connects the southbridge 135 to a security platform module (TPM, Trusted Platform Module) 195.Other components often included in the southbridge 135 include a direct memory access (DMA) controller, a programmable interrupt controller (PIC), and a storage device controller that connects the southbridge 135 to the non-volatile storage device 185, such as a hard disk drive, using a bus 184.
[0025] An ExpressCard socket 155 is a slot that connects quick-connect devices to the information handling system. The ExpressCard socket 155 supports both PCI Express and USB connectivity, connecting to the southbridge 135 using both the universal serial bus (USB) and the PCI Express bus. The southbridge 135 includes the USB controller 140, which provides USB connectivity to devices connected to the USB. These devices include a webcam (camera) 150, an infrared (IR) receiver 148, a keyboard and trackpad 144, and a Bluetooth device 146, which provides wireless personal area networks (PANs).The USB controller 140 also provides USB connectivity to other USB-connected devices 142, such as a mouse, a removable non-volatile storage device 145, modems, network cards, ISDN connectors, fax machines, printers, USB hubs, and many other types of USB-connected devices. While the removable non-volatile storage device 145 is shown as a USB-connected device, the removable non-volatile storage devices 145 could be connected using a different interface, such as a Firewire interface, and so on.
[0026] A wireless local area network (LAN) device 175 connects to the southbridge 135 via the PCI or PCI Express bus 172. The LAN device 175 typically implements one of the IEEE 802.11 standards of radio modulation techniques, all of which use the same protocol to wirelessly communicate between the information handling system 100 and another computer system or device. An optical storage device 190 connects to the southbridge 135 using a serial ATA (SATA) bus 188. Serial ATA adapters and devices communicate over a high-speed serial connection. The serial ATA bus also connects the southbridge 135 to other forms of storage devices, such as hard disk drives. Audio circuitry 160, such as a sound card, connects the southbridge 135 via bus 158.An audio circuit 160 also provides functionality such as an audio line-in and optical digital audio port 162, an optical digital output and headphone jack 164, internal speakers 166, and an internal microphone 168. An Ethernet controller 170 connects the southbridge 135 using a bus, such as the PCI or PCI Express bus. The Ethernet controller 170 connects the information handling system 100 to a computer network, such as a local area network (LAN), the Internet, and other public and private computer networks.
[0027] While Fig. 1 shows a single information handling system, an information handling system can take many forms. For example, an information handling system can take the form of a desktop, a server, a portable computer, a laptop, a notebook, or other computer or data processing system form factors. In addition, an information handling system can take other form factors such as a digital personal assistant (PDA), a gaming device, an ATM machine, a portable telephone device, a communications device, or other devices that include a processor and memory.
[0028] The security platform module (TPM 195, Trusted Platform Module), which is Fig. 1 and described herein to provide security functions is only one example of a hardware security module (HSM). Therefore, the TPM described and claimed herein encompasses any type of HSM, including but not limited to hardware security devices that comply with the Security Computer Group (TCG) standard and are referred to as "Trusted Platform Module (TPM) Specification Version 1.2." The TPM is a hardware security auxiliary system that may be incorporated into any number of information handling systems, such as those used in Fig. 2 are shown.
[0029] Fig. 2 represents an enlargement of the environment of the information handling system, which is Fig. 1 is provided to illustrate that the methods described herein may be performed on a wide variety of information handling systems operating in a network environment. Types of information handling systems range from small handheld devices, such as a handheld computer / cellular phone 210, to mainframe systems, such as a mainframe computer 270. Examples of handheld computers 210 include personal digital assistants (PDAs), personal entertainment devices, such as MP3 players, portable televisions, and compact disc players. Other examples of information handling systems include pen or tablet computers 220, laptop or notebook computers 230, workstations 240, a personal computer system 250, and servers 260. Other types of information handling systems not individually described in Fig. 2 are represented by the information handling system 280. As shown, the various information handling systems can cooperate over networks using a computer network 200. Types of computer networks that can be used to interconnect the various information handling systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, the public switched telephone network (PSTN), other wireless networks, and any other network topology that can be used to interconnect the information handling systems. Many of the information handling systems include non-volatile data storage, such as fixed drives and / or non-volatile memory. Some of the information handling systems described in Fig. 2 depict separate non-volatile data storage (server 260 uses non-volatile data storage 265, mainframe 270 uses non-volatile data storage 275, and information handling system 280 uses non-volatile data storage 285). The non-volatile data storage may be a component external to the various information handling systems or may be internal to one of the information handling systems. Additionally, a removable non-volatile storage device 145 may be distributed among two or more information handling systems using various techniques, such as connecting the removable non-volatile storage device 145 to a USB port or other connector of the information handling systems.
[0030] Fig. Figure 3A is a component diagram showing a user using a hovering touch interface with a touch-activated laptop system. The touch-activated information handling system 230, such as a laptop computer system, has a touch-activated display screen. The system includes one or more sensors, such as a proximity sensing sensor 330, that detects an input instrument 300, such as a stylus or a human finger, hovering at a distance 320 from the surface of the display screen. Fig. 3A, the distance is shown as "z." The input device hovers over the screen location 310, such as a graphical user interface (GUI) control or other object displayed on the screen. In the example shown, the proximity detection sensor 330 is shown on the keyboard component of the information handling system, however, such a sensor could be located elsewhere, such as on the display component of the system.
[0031] An input surface 350 may be a touch-activated display screen, a touchpad surface, or a non-display surface capable of receiving touch inputs. Fig. 3A and Fig. 3B, an input surface 350 is shown as a touch-activated display screen.
[0032] Fig. Figure 3B is a component diagram showing a user using a hover touch interface with a touch-activated tablet system. The touch-activated information handling system 220, such as a tablet computer system, includes a touch-activated display screen. Similar to the system shown in Fig. 3A, this system also includes one or more sensors, such as a proximity detection sensor 330 that detects an input instrument 300, such as a stylus or a human finger, hovering at a distance 320 away from the surface of the display screen. In Fig. In Figure 3B, this distance away from the display screen area is also shown as "z." The input device hovers over the screen location 310, such as a graphical user interface (GUI) control or other object displayed on the screen. In the example shown, the proximity detection sensor 330 is shown as being included in the display component of the system, such as on the edge of the tablet system.
[0033] Fig. Figure 4 is a flowchart showing steps used in setting hover touch preferences on a system. Processing begins in Fig. 4 at 400 and shows the steps performed by a process that executes the installation processing. At step 410, the process selects the first visible area for hover actions configured by the user. The visible area could be a graphical user interface (GUI) control, an application area, a desktop background area, or the like. At step 420, the process configures the first hover action for the selected visible area. In one embodiment, multiple actions can be configured for a screen location with the action performed when the user hovers it over the screen location based on the distance the input instrument is from the screen.
[0034] At step 425, the process selects an action to be performed when the hover action is detected at the selected screen location. Actions may include copying the screen location to a clipboard, performing a zoom operation (e.g., zooming in / out based on input instrument movement toward / away from the screen), panning the display, opening a properties menu, etc.). At step 430, the process prompts the user for a type of hover gesture to be configured.Types of hover actions include a fixed z-position (distance) from the screen surface over a screen location, a movable z-position, such as moving the input instrument toward or away from the screen surface, for example, to zoom in or out, and a movable x / y control, for example, to move an object shown on the display screen or to select an area defined by a "lasso" drawn by the user from a start point to an end point of the defined area.
[0035] At step 440, the process causes the user to hover the input instrument (e.g., finger, stylus, etc.) over a test symbol displayed on the screen for the z-distance. At step 450, the process records the z-distance at which the user hovers the input instrument in response to the prompt at step 440. At step 460, the process records the configuration data pertaining to this hover action in data store 470. Data store 470 includes various visible screen locations and hover actions pertaining to the various screen locations with the hover actions, including the hover distances, the type of hover, and the response actions performed.
[0036] The process determines whether the user wishes to configure additional hover actions for the selected screen location (decision 480). If the user wishes to configure additional hover actions for the selected screen location, then decision 480 branches to the 'Yes' branch, which loops back to step 420 to configure the next hover action for the selected screen area. This feedback continues until the user no longer wishes to configure additional hover actions for the selected screen location, at which point decision 480 branches to the 'No' branch, exiting the feedback loop.
[0037] The process determines whether the user wishes to configure additional screen locations (decision 490). If the user wishes to configure additional screen locations, decision 490 branches to the 'Yes' branch, which loops back to step 410 to select the next screen location for hover actions. This feedback continues until the user no longer wishes to configure additional screen locations, at which point decision 490 branches to the 'No' branch, exiting the feedback loop. Thereafter, at 495, processing according to Fig. 4.
[0038] Fig. Figure 5 is a flowchart showing high-level steps that implement a hover touch interface. Processing according to Fig. Figure 5 begins at 500 and shows the steps performed by a process that handles the user's use of hover actions. At step 520, the process turns on the device's proximity detector. The device's proximity detector or detectors are one or more sensors that detect the input device hovering over an area of the display screen. At step 540, the process monitors for a hover over the display screen of the input device (e.g., human finger, stylus, etc.). A hover event is detected when the input device is not touching the display screen but is at a relatively constant xyz position distance away from the display screen. More specifically, the input device hovers at a z-distance away from the display screen over a screen location denoted by x,y.
[0039] The process determines whether the indicator is detected hovering over a screen location (decision 560). If the indicator is detected hovering over a screen location, then decision 560 branches to the 'Yes' branch to process the hover event using a predefined process 580, which then loops back to step 540. If a hover event is not detected, then decision 560 will skip the predefined process 580 and loop back to step 540. If a hover event is detected, then, in a predefined process 580, the process continues with the routine for handling a possible hover touch action (see Fig. 6 and corresponding text for processing details). After the event has been handled, processing loops back to step 540.
[0040] Fig. Figure 6 is a flowchart showing steps performed to handle a possible hover touch action. The processing according to Fig. Figure 6 begins at 600 and shows the steps taken by a process handling a possible hover event detected by the information handling system. At step 610, the process retrieves the visible area(s) corresponding to the xy location over which the input device is hovering. For example, the hover event could be detected when the input device is hovering over an icon that is within a container area located within an application being displayed on a desktop.
[0041] At step 625, the process selects the first visible area, or a screen location from the smallest area to the largest area. Using the example above, the smallest area would be the icon, and the largest area would be the desktop. At step 630, the process retrieves any hover action data configured for the selected screen area. From the example above, step 630 could retrieve any hover action data pertaining first to the icon, then the container, then the application, and then the desktop.
[0042] The process determines whether one or more hover actions have been configured for the selected screen location (decision 640). If one or more hover actions have been configured for the selected screen location, then decision 640 branches to the 'Yes' branch to perform steps 650 through 685. If, on the other hand, no action(s) have been configured for the selected area, then decision 640 branches to the 'No' branch to perform steps 690 through 695.
[0043] If one or more hover actions have been configured for the selected screen location, decision 640 branches to the 'Yes' branch to perform steps 650 through 685. The process next determines whether multiple actions have been configured for the selected screen location (decision 650). If multiple actions have been configured for the selected screen location, then decision 650 branches to the 'Yes' branch, whereupon the process selects the hover action with a z-position closest to the current z-position of the pointer at step 680. In one embodiment, the selected action is within a distance range that includes a current distance that the input instrument is from the screen.On the other hand, if only one hover action has been configured for the selected screen location, then decision 650 branches to the 'No' branch, whereupon the process selects the only hover action configured by the selected screen location at step 665.
[0044] The process determines whether the selected action involves movement of the input instrument (decision 670). If the selected action involves movement of the input instrument, decision 670 branches to the 'Yes' branch, after which the process executes the movement-based action routine at predefined process 675 (see Fig. 7 and the corresponding text for processing details). On the other hand, if the selected action does not involve movement of the input instrument, then decision 670 branches to the 'No' branch, whereupon the process performs the selected fixed position action at step 680. The processing according to Fig. 6 then returns to the calling routine (see Fig. 5) back at 685.
[0045] Now returning to decision 640, if no actions have been configured for the selected visible area, the process next determines whether there are multiple (larger) visible areas below the input instrument (decision 690). Using the example from above, if no actions have been configured for the icon, then the process would determine whether any actions have been configured for the container, the application, and finally, the desktop. If there are no more visible areas below the input instrument, then decision 690 branches to the 'Yes' branch, which loops back to step 625 to select the next larger screen area and to determine whether any actions are configured for the newly selected screen area.This feedback continues until there are no more screen areas to select and process, at which point decision 690 branches to the 'No' branch, leaving the feedback loop and processing proceeding to the calling routine (see . Fig. 6) returns to 695 without performing any actions.
[0046] Fig. Figure 7 is a flowchart showing further steps that handle a movement based on a hover touch action. The processing according to Fig. Figure 7 begins at 700 and shows the steps performed by a process that handles motion-based hover actions. At step 705, the process indicates the start of the motion-based action on the screen (such as by blinking or otherwise highlighting a screen area, etc.). At decision 710, the process determines whether the hover action is a z-based hover action (up / down, which moves the input instrument closer to or farther away from the screen surface) or an x,y-based hover action (across the screen). If the hover action is a z-based hover action (up / down), then decision 710 branches to the left (z-based) branch from decision 710 to perform steps 715 through 745.On the other hand, if the hover action is an x,y-based hover action (across the screen), then decision 710 branches to the downward (x,y-based) branch to perform steps 750 through 795.
[0047] If the hover action is a z-based hover action (up / down), then decision 710 branches to the left (z-based) branch from decision 710 to execute steps 715 through 745. At step 715, the process sets the starting z-position (distance) to the current z-position (distance) of the input device from the screen surface. At step 720, the process first receives the x, y, z position of the input device. The process determines whether the input device was moved up or down, changing the distance (z-position) from the screen surface (decision 725). If the input device was moved up or down, then decision 725 branches to the 'yes' branch, after which the process continues the hover action according to the direction of movement at step 730.For example, the hover action could be zooming in when the input tool is moved toward the screen surface and zooming out when the input tool is moved away from the screen surface.
[0048] On the other hand, if the input instrument has not been moved up or down (away from or toward the surface of the screen), then decision 725 branches to the 'No' branch to execute decision 735. At decision 735, the process determines whether the input instrument has been moved away from the original x / y position (decision 735). If the input instrument has been moved away from the original x / y position, then decision 735 branches to the 'Yes' branch, whereupon at step 740 the z-based hover action is terminated and processing returns to the calling routine (see Fig. 6) returns at 745. On the other hand, if the input instrument has not been moved from the original x / y position, then decision 735 branches to the 'no' branch, whereupon processing continues looping back to step 720 to receive the next x,y,z position of the input instrument relative to the screen surface.
[0049] Returning now to decision 710, if the hover action is an x,y-based hover action (above the screen), then decision 710 branches to the downward branch (x,y-based) to perform steps 750 through 755. At step 750, the process sets the starting x,y position to the current x,y position of the input device. At step 755, the process receives the next x,y,z position of the input device. The process first determines whether the input device is moving to a different x / y position above the display screen, with the z position remaining relatively constant (decision 760). If the input device is moving to a different x / y position above the display screen, then decision 760 branches to the 'yes' branch, whereupon at step 765 the process continues the hover action according to the direction of movement of the input device.For example, when the user selects an area on the screen, the change in x / y position can be adjusted to the size of a "lasso" drawn on the screen.
[0050] On the other hand, if the input device is not moved to a different x / y position above the display screen, decision 760 branches to the 'No' branch to execute decision 770. At decision 770, the process determines whether the input device is being moved up / down or toward / away from the display screen (decision 770). If the input device is being moved up / down, then decision 770 branches to the 'Yes' branch, whereupon at step 775 the x / y-based hover action is terminated and processing returns to the calling routine (see Fig.6) returns at 795. On the other hand, if the input instrument is not moved up / down, then decision 770 branches to the 'no' branch, whereupon processing continues looping back to step 755 to receive the next x,y,z position of the input instrument relative to the screen surface.
[0051] While particular embodiments have been shown and described, it will be apparent to those skilled in the art that, based on the teachings herein, changes and modifications may be made without departing from this disclosure and its broader aspects. Therefore, the appended claims are intended to include within their scope all such changes and modifications as fall within the scope of this disclosure. It is further understood that the invention is defined solely by the appended claims.
Claims
[1] Method (600) comprising: Detecting an input instrument hovering at a distance away from an object displayed on an input surface of a touch-activated display of an information handling system, wherein the detecting includes detecting an input location on the input surface over which the input instrument is hovering; Retrieving (610) visible regions of the input surface corresponding to the detected input location, the visible regions having different region sizes ranging from a smallest region to a largest region; Selecting (625) the smallest area of the visible areas; Determining (640) whether one or more actions have been configured for the selected area; if one or more actions have been configured for the selected area, executing (675, 680) one of the one or more actions; and if no actions have been configured for the selected area, selecting (625) the next larger area of the visible areas and determining (640) whether one or more actions have been configured for the selected next larger area. [2] The method (600) of claim 1, wherein the input instrument is selected from a group consisting of a human finger and a stylus. [3] The method (600) of claim 1, wherein the detecting further detects a length of the distance, and wherein the method further comprises: Selecting (660) a first action from a plurality of actions in response to the distance being within a first distance range away from the input surface; and Selecting (660) a second action from the plurality of actions in response to the distance being within a second distance range away from the input surface, wherein the action performed is the selected first or second action. [4] The method (600) of claim 1, further comprising: Execute (675, 680) the action on the object. [5] The method (600) of claim 1, further comprising: Detecting (670) a movement of the floating input instrument from the detected input location to a second input location, wherein the first and second input locations define a surface area enclosing the object, wherein the action is performed based on the surface area. [6] The method (600) of claim 1, wherein the detecting further detects a first length of the distance and the input location is proximate to the object, and wherein the method further comprises: Detecting (670) a movement of the hovering input instrument over the input location, wherein the movement changes the length of the distance the input instrument is hovering over the input location, wherein the action taken is based on the change in length. [7] Information handling system comprising: one or more processors; a memory coupled to at least one of the processors; a touch-activated display accessible by at least one of the processors, the touch-activated display having an input surface; one or more sensors accessible by at least one of the processors that detect an input instrument hovering over the touch-activated display; and a set of instructions stored in the memory and executable by at least one processor for performing a method (600) according to any one of claims 1 to 6. [8] Computer program product comprising: a computer-readable storage medium comprising a set of computer instructions, the computer instructions being operable to perform a method (600) according to any one of claims 1 to 6.
Citation Information
Patent Citations
Mobile terminal and method of controlling a mobile terminal
EP2445182A2
Method for operation of pen function and electronic device supporting the same
EP2701056A2
Method and system for controlling device
EP3627302A1