Touch input conversion
By detecting finger information and predetermined criteria in the touch sensing system and dynamically switching input modes, the problem of unnatural response of touch input systems in the prior art during complex interactions and multi-finger inputs is solved, and the user experience and input flexibility are improved .
Patent Information
- Application Number
- CN202210115838.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2011-07-26
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2031-07-26
AI Technical Summary
Existing touch input systems are difficult to achieve flexible input conversion and more advanced control when handling complex touch interactions and multi-finger inputs, especially when the number of fingers and motion modes are changed. The system response is not natural enough and the user experience is poor. .
By detecting the contact number, movement, lifting and down-touching information of the finger in the touch sensing system, combined with predetermined criteria, dynamically selecting and switching input modes, such as switching from pointing input to scroll input, switching from scroll input to drag The dynamic input allows input conversion when some fingers are touched, and uses touch information and other factors to determine whether to switch the input state.
Achieve more complex touch interactions and more flexible input controls, improve user experience, allows for natural operational flow and system responses when multi-finger input changes, and enhances the flexibility and accuracy of touch inputs.
Smart Images

Figure CN114237486B_ABST
Abstract
Description
[0001] This application is a divisional application of an invention patent application with an application date of July 26, 2011, application number 201710079432.X, and invention name “CONVERSION OF TOUCH INPUT”. Technical Field
[0002] The present invention relates generally to transitioning between touch input selections, and more particularly to selecting contact inputs on or near a touch sensing surface. Background Art
[0003] Nowadays, there are many types of input devices that can be used to perform operations within a computing system, including input devices such as buttons or keys, mice, tracking balls, joysticks, touch sensor panels, touch screens, etc. In particular, touch screens are becoming more and more popular because they are easy to operate, suitable for multi-purpose operations, and are becoming increasingly affordable. A touch screen may include a transparent touch sensor panel or an integrated touch screen positioned in front of a display device such as a liquid crystal display (LCD), in which a touch sensing circuit is partially or completely integrated into the display. A touch screen may allow a user to perform various functions by touching the touch screen using a finger, stylus, or other object at a location indicated by a user interface (UI) displayed by a display device. In general, a touch screen may identify a touch event and the location of the touch event on the touch sensor panel, so that the computing system may interpret the touch event based on the display that appears at the time of the touch event, and then may perform one or more actions based on the touch event.
[0004] For example, a mutual capacitance touch sensor panel may be comprised of an array of drive and sense lines of a generally transparent conductive material, such as indium tin oxide (ITO), typically arranged in horizontal and vertical rows and columns on a generally transparent substrate. A drive signal may be transmitted through the drive lines, which may allow the static mutual capacitance at the intersections or adjacent regions (sensing pixels) of the drive and sense lines to be measured. Based on the sense signals that may be generated in the sense lines due to the drive signals, the static mutual capacitance and any change in static mutual capacitance due to a touch event may be determined. Summary of the invention
[0005] The present invention generally relates to conversion between touch input selections, and more specifically, to selecting a contact input on or near a touch sensing surface. The contact can be tracked to obtain touch information of the contact, such as the number of contacts, the movement of the contact, the moment of lift-off and touchdown, etc. A first gesture can be detected, and a first input corresponding to the first gesture can be selected, the first gesture corresponding to the first touch information of multiple contacts performing an action. A second gesture can be detected, and it can be determined whether to select a second input corresponding to the second gesture, the second gesture corresponding to the second touch information of multiple contacts performing an action. If the third information meets a predetermined criterion, the second input can be selected, and if the third information does not meet the predetermined criterion, the first input can be maintained. For example, the third information can include the number of downward contacts, the number of contacts lifted off, the movement and / or stillness of all contacts or a subset of contacts, the arrangement of contacts, whether to add contacts or remove contacts, and other information, such as the currently selected input, the input to be selected, whether the input is locked, etc. In this way, for example, the touch sensing system can allow more complex interactions and more control over touch-based inputs. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1A-Figure 1D An example mobile phone, an example digital media player, an example personal computer, and an example wireless trackpad are shown, each of which includes functionality according to embodiments of the present disclosure.
[0007] Figure 2 An example computer system including functionality according to an embodiment of the present disclosure is shown.
[0008] Figure 3 An example method of transitioning from an unspecified quiescent state according to an embodiment of the present disclosure is shown.
[0009] Figure 4 An example method of transitioning after a pointing input has been selected (but not locked) is shown according to an embodiment of the present disclosure.
[0010] Figure 5 An example method of transitioning from locked pointing input according to an embodiment of the present disclosure is shown.
[0011] Figure 6 An example method of transitioning from a currently selected unlocked scroll input according to an embodiment of the present disclosure is shown.
[0012] Figure 7 An example method of transitioning from a drag input according to an embodiment of the present disclosure is shown.
[0013] Figure 8 and Fig. 9 An example drag-continuation input according to an embodiment of the present disclosure is shown.
[0014] Fig.10 An example method of transitioning based on lift-off and drop-off finger subsets according to an embodiment of the present disclosure is shown.
[0015] Fig.11 A functional block diagram of a computing system according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0016] In the following description of the embodiments, reference is made to the accompanying drawings, which constitute a part of this description and show by way of illustration specific embodiments of the present disclosure that can be implemented. It should be understood that other embodiments can be used and structural changes can be made without departing from the scope of the disclosed embodiments.
[0017] The disclosed example embodiments relate to selecting inputs of a touch sensing surface. A user may perform touch inputs, for example, by contacting a touch sensing surface of a computing system with one or more fingers. For example, a user may perform certain actions, such as motion, tapping, etc., on the touch sensing surface with a certain number of fingers to move a cursor, click a button, scroll a document, select text by dragging a cursor, etc. The contacts corresponding to the user's fingers may be tracked to obtain touch information of the contacts, such as the number of contacts, the movement of the contacts, the moments of lift-off and touch-down, etc. The touch information may be used to detect gestures corresponding to various inputs.
[0018] For example, a first gesture may be detected, and a first input corresponding to the first gesture may be selected, the first gesture corresponding to the first touch information of multiple contacts performing an action. For example, a user may touch down two fingers and move the two fingers vertically, a two-finger vertical motion gesture may be detected, and a corresponding scroll input may be selected. A second gesture may be detected, the second gesture corresponding to the second touch information of multiple contacts performing an action. For example, while the scroll input is selected, the user may touch down an additional finger. It may be determined whether to select the second input corresponding to the second gesture. If the third information meets the predetermined criteria, the second input may be selected, and if the third information does not meet the predetermined criteria, the first input may be maintained. For example, the third information may include the number of downward contacts, the number of contacts lifted off, the movement and / or stillness of all contacts or a subset of contacts, the arrangement of contacts, whether to add contacts or remove contacts, and other information, such as the currently selected input, the input to be selected, whether the input is locked, etc. In this way, for example, the touch sensing system may allow more complex interactions and more control over touch-based inputs.
[0019] Although the embodiments disclosed herein may be described and illustrated with respect to mutual capacitance touch sensing surfaces, it should be understood that the embodiments are not limited thereto, but may be otherwise applicable to self-capacitive, optical, resistive, or other touch sensing surfaces and technologies that can detect single and / or multiple touches on or near a surface, for example.
[0020] Figure 1A FIG. 1D shows an example system in which embodiments of the present disclosure may be implemented. Figure 1A An example mobile phone 136 having a touch screen 124 is shown. Figure 1B An example digital media player 140 having a touch screen 126 is shown. Figure 1C An example personal computer 144 is shown having a touch screen 128 and a trackpad 130 . Figure 1D An example wireless trackpad 150 is shown that may be wirelessly connected to a personal computer, such as personal computer 144, for example.
[0021] Figure 2 2 is a block diagram of an example computing system 200 illustrating one implementation of an example touch screen 220 according to an embodiment of the present disclosure. The computing system 200 may be included in, for example, a mobile phone 136, a digital media player 140, a personal computer 144, or any mobile or non-mobile computing device that includes a touch screen. A similar computing system having similar touch sensing functionality without the need for a display function may be included in, for example, a trackpad 150. The computing system 200 may include a touch sensing system including one or more touch processors 202, peripherals 204, a touch controller 206, and touch sensing circuitry. The peripherals 204 may include, but are not limited to, random access memory (RAM) or other types of memory or storage devices, a watchdog timer, and the like. The touch controller 206 may include, but is not limited to, one or more sensing channels 208, channel scan logic 210, and driver logic 214. The channel scan logic 210 may access the RAM 212, automatically read data from the sensing channels, and provide control for the sensing channels. Additionally, channel scan logic 210 may control driver logic 214 to generate excitation signals 216 at various frequencies and phases that may be selectively applied to drive regions of touch sensing circuitry of touch screen 220. In some embodiments, touch controller 206, touch processor 202, and peripherals 204 may be integrated into a single application specific integrated circuit (ASIC).
[0022] The touch screen 220 may include touch sensing circuitry that may include a capacitive sensing medium having a plurality of drive lines 222 and a plurality of sense lines 223. The drive lines 222 may be driven according to an excitation signal 216 from the driver logic 214 via a drive interface 224, and the resulting sense signals 217 generated in the sense lines 223 may be transmitted to the sense channels 208 (also referred to as event detection and demodulation circuitry) in the touch controller 206 via a sense interface 225. In this manner, the drive lines and sense lines may be part of a touch sensing circuit that may interact to form capacitive sensing nodes that may be considered touch picture elements (touch pixels), such as touch pixels 226 and 227. This understanding is particularly useful when the touch screen 220 is viewed as capturing an "image" of a touch. In other words, after the touch controller 206 has determined whether a touch is detected at each touch pixel in the touch screen, the pattern of touch pixels in the touch screen where the touch occurs may be considered an "image" of the touch (e.g., a pattern of fingers touching the touch screen).
[0023] The computing system 200 may also include a host processor 228 for receiving output from the touch processor 202 and performing actions based on the output. For example, the host processor 228 may be connected to a program storage device 232 and a display controller, such as an LCD driver 234. The host processor 228 may use the LCD driver 234 to generate images on the touch screen 220, such as images for a user interface (UI), and may use the touch processor 202 and the touch controller 206 to detect touches on or near the touch screen 220, such as touch input to a displayed UI. The touch input may be utilized by a computer program stored in program storage device 232 to perform actions that may include, but are not limited to, moving an object (such as a cursor or pointer), scrolling or panning, adjusting control settings, opening a file or document, navigating a menu, making a selection, executing instructions, operating a peripheral connected to the host device, answering a phone call, placing a phone call, terminating a phone call, changing volume or audio settings, storing information related to telephone communications (such as addresses, frequently dialed numbers, calls received, missed calls), logging into a computer or computer network, allowing authorized individuals to access restricted areas of a computer or computer system, loading a user profile related to a user's preferred computer desktop layout, allowing access to web content, launching a particular program, encrypting or decoding messages, etc. Host processor 228 may also perform additional functions that may not be related to touch processing.
[0024] The computing system 200, and in particular the touch sensing system of the computing system 200, can allow a user to input by one or more touch devices (such as fingers, thumbs, etc.), such as tapping, sliding, etc., on a touch sensing surface (such as the touch screen 220). For example, based on multiple contacts and the movement of contacts on or near the touch sensing surface, a specific input can be selected. For example, one finger placed on the touch sensing surface and the movement can correspond to a pointing input, which can cause a mouse cursor to move in the direction of the one finger movement. Two fingers placed on the touch sensing surface and the movement can correspond to a scrolling input, which can cause a document displayed on the touch screen or display to scroll in the direction of the two fingers movement.
[0025] To switch between selected inputs, some systems may require the user to lift all fingers off the touch screen and then drop the number of fingers required for the new input. In other words, even if the number of fingers changes, some systems may simply maintain the currently selected input until all fingers are lifted off. For example, in order to switch from a two-finger scrolling input to a one-finger pointing input, the system may require the user to lift two fingers and drop back to one finger. In this system, the scrolling input may remain selected even after lifting one of the two fingers. On the other hand, some systems may simply select the current input that matches the number of fingers currently touching down. In these systems, for example, each new finger lift-off / touch-down causes the selected input to switch to the corresponding number of fingers touching down. In other words, both systems may always allow or never allow switching between inputs while some fingers remain touching down.
[0026] Various exemplary embodiments are described below in which a selected input may be switched based on whether information satisfies certain predetermined criteria, such as touch information based on characteristics of a contact corresponding to a finger, or other information, such as a current operating state. In the following examples, unless otherwise specified, a "finger" may include a finger or a thumb.
[0027] The exemplary embodiments are described below using a set of exemplary inputs corresponding to various combinations of contact quantity / arrangement and motion (also referred to herein as "basis gestures") as summarized in Table 1. When the touch sensing system detects a combination of contact quantity / arrangement and motion, a corresponding input may be made to a computing system (such as computing system 200).
[0028] Table 1
[0029]
[0030] In this example, when the touch system does not detect a touch, for example, the user does not touch the touch surface, the touch system can enter a "no current input" state. When in the no input state, if one of the basic gestures is detected, the input corresponding to it can be selected without further evaluation. Therefore, the user can directly select one of the inputs in Table 1 by lifting all fingers, dropping (i.e., touching down) the number of fingers corresponding to the desired input, and performing the corresponding action (e.g., tapping, moving). When the desired input is completed, the user can lift all fingers and return the system to the no input state.
[0031] In addition, the example touch system can allow more complex interactions, including lifting, dropping, stationary and / or moving one or more additional fingers while selecting the current input. Based on touch information determined according to various characteristics of the contact, such as the number of down contacts, the number of contacts lifted off, the movement and / or stationary of all contacts or a subset of contacts, the arrangement of contacts, adding or removing contacts, and other information (such as the currently selected input, the input to be selected, whether the input has been locked), etc., the touch system can decide whether to switch the currently selected input to a new input.
[0032] Figures 3 to 10 An example method for determining when to allow and when to prevent a currently selected input / state from being switched to a new input / state is shown. The example method described below does not necessarily cover all possible switching situations that may occur, but rather provides examples in which various touch information and other information can be compared to predetermined criteria to determine whether to switch inputs.
[0033] Figure 3An example method of converting from an unspecified static state is shown. The unspecified static state can be, for example, a state in which all fingers are static and no input has been selected. For example, the unspecified static state can be entered by dropping one or more fingers on the touch surface while keeping the fingers substantially fixed on the surface. While in the static state, certain conversions to the selected input can be performed, while other conversions can be prevented. Starting from the unspecified static state 301, if a finger movement is detected (303), the number of fingers touching down (i.e., the number of fingers currently touching the touch surface) can be counted (305). If the number of fingers touching down is equal to 2, the unspecified static state can be maintained (307). On the other hand, if the number of fingers touching down is not equal to 2, the pointing input can be selected (309). In other words, starting from the unspecified static state, as long as more than two fingers are currently touching down, the user can start the pointing input by moving one finger. Therefore, determining whether to select a new input can be based on touch information, such as the total number of fingers touching down, and whether the other fingers touching down are in a static state (e.g., substantially fixed). The determination can also be based on other information, such as whether the current state is an unspecified static state.
[0034] If, instead of detecting one finger movement (303), two finger movements are detected (311), the number of down-touched fingers may be determined (313), and if the number of down-touched fingers is equal to 3, the unspecified static state may be maintained (315). On the other hand, if the number of down-touched fingers is greater than 3, a scroll input may be selected (317). In other words, starting from the unspecified static state, as long as at least four fingers are down, the user may move two fingers and initiate a scroll input.
[0035] If five finger movements are detected (319), the pointing input (323) can be selected. On the other hand, if five finger movements are not detected, the unspecified stationary state (321) can be maintained. In other words, starting from the unspecified stationary state, the user can move all five fingers to initiate the pointing input. Note that neither three fingers nor four fingers can initiate input from the unspecified stationary state. That is, if the user moves three or four fingers, the unspecified stationary state is maintained. On the other hand, if the user moves one, two, or five fingers, a new input state can be selected. In this example embodiment, once an input is selected, subsequent stationary movement of the fingers will not enter the unspecified stationary state unless all fingers are lifted off and then touched down in the stationary state. In other words, even if the fingers are subsequently stationary, the currently selected input will remain selected.
[0036] Figure 4An example method for converting after a pointing input has been selected (but not locked) is shown. Locking the pointing input is described in more detail below. Starting from the selected pointing input 401, if the number of fingers touched down is 1 (403), the user can point with a single finger. Other inputs that may occur when the user points with a single finger that has touched down may include dropping an additional finger (405). If no additional finger is dropped, the pointing input (407) can be maintained. On the other hand, if an additional finger is dropped during the process of single finger pointing, further testing can be performed to determine whether to switch the input. If four fingers are dropped (409), the pointing input (407) can be maintained. If not four fingers are dropped, it can be determined whether the additional finger is dropped when the single finger rolling stops (rolling stop), or whether the additional finger is dropped immediately after the single finger touches down. For example, by determining a deceleration measure of the motion, the rolling stop can be determined. For example, the touch system may include a pair of autoregressive filters including a slow filter and a fast filter, the slow filter may provide an indication of the average speed of the contact movement over a longer period of time, and the fast filter may provide an indication of the average speed of the contact movement over a shorter period of time. For example, comparing the outputs of the two filters may provide an indication of whether the contact movement is decelerating. For example, if the output of the fast filter is lower than a predetermined ratio of the slow filter, it may be determined that the scrolling has stopped.
[0037] Returning to determining whether a single finger is in a scroll stop (411), for example, if an additional finger drop occurs within a predetermined time before or after the scroll stop of the single finger motion as determined by comparison of the autoregressive filter, it can be determined that the additional finger drop occurs when the single finger scrolls. To determine whether the additional finger drop occurs immediately after the single finger touches down, for example, if the drop occurs within, for example, 125 milliseconds of the initial touchdown of the single finger, a certain number of additional finger drops can be allowed to select a new input. If the additional finger drops do not meet a criterion, that is, the first finger is in a scroll stop or immediately after the initial touchdown, the pointing input selection can be maintained (407). On the other hand, if the criterion (411) is met, then if three additional fingers drop (413), a four finger input can be selected (415). If two additional fingers drop (417), a three finger input (e.g., swipe or drag) can be selected (419). If a finger tap is detected (421), a finger tap input can be selected (423). If it is detected that a finger is dropped and the dropped finger remains in contact (425), the scroll input (427) may be selected. Otherwise, the pointing input (407) may remain selected.
[0038] In other words, if the user is pointing with one finger down, the user can change the selected input by dropping one, two, or three fingers as long as the one finger down becomes a scroll stop before the additional finger is dropped. Additional finger drops among the four fingers and finger drops that do not occur at a scroll stop or immediately after the initial single finger down do not switch the input from the selected pointing input. Therefore, determining whether to select a new input can be based on touch information, such as the total number of fingers down, the number of additional fingers dropped, and whether the single finger is in a scroll stop.
[0039] If more than one finger is down while a pointing input is selected, one way the user can select another input may be to move more than one finger. However, the touch system may require that certain criteria be met in order for a new input to be selected. If one or zero fingers are moving (429), the pointing input may be maintained (407). On the other hand, if more than one finger is moving (429), it may be determined whether all currently down fingers are in motion (431). If not all down fingers are moving, the pointing input may be locked (433). In other words, if a pointing input is currently selected and more than one but not all currently down fingers are moving, the pointing input may be locked from being selected. Figure 5 Locked pointing input is described in more detail in .
[0040] If it is determined that all the down-touched fingers are in motion (431), it can be determined whether the down-touched finger count is equal to 2 or 5 (435). If the finger count is 2 or 5, the pointing input can be locked (433). On the other hand, if the finger count is not equal to 2 or 5, it can be determined whether the movement of all the down-touched fingers occurs when the scrolling stops (437). If the movement does not occur when the scrolling stops, the pointing input can be maintained (407). On the other hand, if the down-touched fingers are all in scrolling stop (437), then if three fingers are in motion, three finger input (such as swiping or dragging) can be selected (441). If four fingers are detected to move vertically (443), the view all window input (445) can be selected. If four fingers are detected to move horizontally (447), the application switching input (449) can be selected. Otherwise, the pointing input (433) can be locked.
[0041] In other words, if the user is in a stationary pointing input, that is, pointing is currently selected and the user has more than one finger down, the user can continue pointing by moving only one finger. By performing a specific multi-finger motion that meets specific criteria, the user can switch to another input. Specifically, if the user is pointing with one of the three fingers down, the single finger enters a scroll stop, and a three finger motion is initiated, a three finger input can be selected. If the user has four fingers down and moves a single finger during the pointing input, the user can enter one of the four finger inputs by entering a scroll stop with the single finger, and initiating a four finger vertical motion to switch to a view all windows input or initiating a four finger horizontal motion to switch to an application switching input. All other multi-finger motions when pointing is selected can lock the pointing input selection.
[0042] Therefore, based on touch information, such as the total number of fingers touching down, whether all fingers or a subset of fingers moved, and whether the fingers moved at substantially the same time, it can be determined whether a new input is selected. The determination can also be based on other information, such as whether the currently selected input is locked (to be described in more detail).
[0043] Figure 5 An example method of converting from a locked pointing input is shown. Starting from the locked pointing input selection 501, the user can unlock the pointing input selection by lifting and tapping more than one finger (503) or by lifting and touching down more than one finger (505), and select another input. In each of these cases, the pointing selection can be unlocked (507), and the input corresponding to the number of fingers when lifting and tapping or lifting and touching down can be selected (509). Another method in which the user can unlock the pointing selection can be to lift all fingers except one finger (511). If the user lifts all fingers except one finger, the pointing input can be unlocked (513), and as long as the one finger remains in touch, the selection of the pointing input can be kept (515) in an unlocked state. Otherwise, the selected input can remain locked in the pointing input (517). Once the pointing input is locked, the user can point using various finger combinations and movements. This can allow the user to freely point, which can be a common task. While keeping at least one finger down, the user can still change inputs by lifting and tapping or lifting and touching down a subset of the total number of fingers that are down. In addition, the user can simply lift all but one finger to unlock the pointing selection, and thus select other inputs through additional actions once only one finger remains on the surface.
[0044] Figure 6An example method for converting from a currently selected scroll input that is never locked is shown. Starting from the scroll input selection 601, if the number of fingers touched down is equal to 2 (603), some inputs can be selected by dropping additional fingers. If the user drops additional fingers (605), and the number of additional fingers dropped is equal to 3 (607), the scroll input (609) can be maintained. However, if the user drops one or two additional fingers, if the additional finger drops when the two scroll fingers stop scrolling, or if the additional finger drops immediately after the initial two scroll fingers touch down (611), the input can be changed. If the two scroll fingers are not in a scroll stop or are not touched down immediately before the additional fingers drop, the scroll input (609) can be maintained. However, if the two scroll fingers are in a scroll stop, or are touched down immediately before the drop of an additional finger (613), a new input can be selected according to whether the user has set the three finger input to a sweep input or a drag input. If the user has set the three finger input to a sweep input (615), the sweep input (617) can be selected. However, if the user has selected the three finger input as the drag input, the scroll input (609) may be maintained, i.e., the system may prevent switching to the drag input. In other words, while in two finger scrolling, the user may switch to the swipe input by dropping an additional finger when the scrolling stops, however, the user may not switch from two finger scrolling to three finger drag input in the same manner. Thus, determining whether to switch to a new input may depend on information such as the function of the new input, e.g., assuming the same touch information, such as the number of contacts, motion, etc., switching to a new input may depend on, for example, the function selected by the user to correspond to a particular base gesture.
[0045] For example, by allowing switching when the motion associated with the switch and the motion associated with the new input match, and preventing switching when the motion associated with the switch and the motion associated with the new input do not match, this can improve the user experience. In this example, the user can usually use fast motion for sweep input and use slower motion for drag input. Compared with the usually slower drag motion, allowing the user to switch to sweep input when the scrolling of two fingers scrolling stops can better match the typical fast motion used when the two fingers scroll stop. In addition, compared with switching from scroll input to drag input, it is more typical that the user may want to switch from scroll input to sweep input. In this way, for example, using different information, such as touch information and other information to determine whether to allow switching to a new input, can allow the design of the touch sensing system to be more natural and easier to use.
[0046] Back to Figure 6, if the scrolling two fingers are at scroll stop or were touched down just recently (611), and the user drops two additional fingers (619), a four finger input may be selected (621). The four finger input may, for example, depend on a specific direction of motion of the four fingers. In this case, in step 621, additional motion tests may be required to generate a four finger input, such as a view all windows input or an application switching input. In contrast to dropping an additional single finger, dropping two additional fingers may switch to a new input event, regardless of the function of the input event to be switched to.
[0047] Note that, in contrast to the single finger pointing input selection, the scrolling input selection in this example embodiment does not include an additional way to transition to a different input using only the motion of the additional finger without requiring the additional finger to be dropped. For example, a user performing a two finger scroll is more likely to inadvertently move the additional finger than a user performing a one finger pointing.
[0048] Figure 7 An example method for converting from a drag input according to an embodiment of the present disclosure is shown. Starting from the drag input selection 701, the system can determine whether the number of fingers touching down is equal to 3 (703). If the number of fingers touching down is not equal to 3, then if all fingers except one finger are lifted off (705), the input can be switched to a pointing input (707); otherwise, the drag input can be maintained (709). If the number of fingers touching down is equal to 3 (703), then if two fingers are lifted off (711) and the three-finger drag is not in a scroll stop (713), the drag continuation input (715) can be selected. If two fingers are lifted off when the scroll of the three-finger drag stops (713), the input can be switched to a pointing input (717). If the two fingers are not lifted off (711), the drag input (709) can be maintained.
[0049] In other words, if the user is dragging with three fingers, by lifting two of the three fingers when not at a scroll stop, the drag continuation input can be selected. Figure 8 and Fig. 9 Describes drag-continuous input in more detail.
[0050] Figure 8 and Fig. 9 An example drag-and-hold input according to an embodiment of the present disclosure is shown. Figure 8 , obtain the velocity of the two finger subsets lifted off during the three finger drag (801). The original decay rate can be determined and set based on the lift-off velocity (803). Fig. 9An example diagram is shown, which shows three speed ranges, a high speed range 901, a medium speed range 903, and a low speed range 905, which correspond to a first decay rate 907, a second decay rate 909, and a third decay rate 911. In this example, the lift-off speed is within the high speed range 901, so the first decay rate 907 can be selected (803) as the initial decay rate, and the drag input motion can initially continue at the lift-off speed and then decelerate according to the initial decay rate (805). When the speed of the continued drag motion reaches the next slower speed range, the decay rate can be reset to the decay rate of the next slower range (807). In this example, when the continued drag motion is decayed to the medium speed range 903, the second decay rate 909 can be selected, and the continued drag motion can be decelerated based on the second decay rate. The same is true when the continued drag motion decays to the low speed range 905.
[0051] like Fig. 9 As shown in , ranges and associated decay rates can be selected so that motion that is initiated in the high speed range can decay quickly, and the motion can be so fast that the average user's vision may not be able to easily track the motion. In this way, for example, a motion that is too fast for the user to track can be quickly slowed to a more reasonable speed, such as the medium speed range. Once the sustained motion reaches the medium speed range, the decay rate can be set so that the motion decays more slowly. In this way, for example, a relatively fast motion can be maintained at a speed that the user can visually track for a long period of time. Finally, when the sustained motion becomes very slow, such as reaching the slow speed range, the decay rate can be set to quickly decay the motion so that the sustained motion does not remain in slow motion for too long before coming to a stop.
[0052] In some exemplary embodiments, the range and rate of decay can be set so that high-speed continuous motion stops within a predetermined distance or time, regardless of the precise speed of the lift-off speed within the high-speed range. Although the above describes the example motion continuation with reference to continuous dragging motion, those skilled in the art will appreciate that other forms of motion continuation (such as cursor motion continuation, scrolling motion continuation, etc.) are also applicable.
[0053] Fig.10 An example method of converting based on lifting and dropping a subset of fingers according to an embodiment of the present disclosure is shown. In this example, the thumb is considered a finger. However, in some embodiments, when the user drops the thumb within the thumb resting zone portion of the touch surface, the touch system can determine that the contact is the thumb and can ignore the input from the thumb or allow the user to use a specific gesture involving the thumb.
[0054] Fig.10 It is shown that when four or five fingers are stationary, the user is able to switch to the basic gesture input even when the current input is selected, that is, when the four or five fingers become stationary (such as scrolling stops). The process can also be applied to the unspecified stationary state of four or five fingers. With four fingers (1001) or five fingers (1003) stationary, the user can lift and tap one finger (1005) to switch to a one finger tapping input (1007), lift and tap two fingers (1009) to switch to a two finger tapping input (1011), lift, drop and move two fingers (1013) to switch to a locked scrolling input (1015), lift, drop and move three fingers (1017) to switch to a three finger input (1019), or lift, drop and move four fingers (1021) to switch to a four finger input (1023) when the five fingers are stationary, otherwise the currently selected input (1025) can be maintained.
[0055] According to some embodiments, Fig.11 1 shows a functional block diagram of a computing system 1100 configured according to the principles of the present invention described above. The functional modules of the computing system can be implemented by hardware, software, or a combination of hardware and software to implement the principles of the present invention. Those skilled in the art should understand that Fig.11 The functional modules described can be combined or divided into sub-modules to implement the principles of the present invention described above. Therefore, the description herein can support any possible combination, division, or further definition of the functional modules described herein.
[0056] like Fig.11 As shown, the computing system 1100 includes a touch sensing surface 1110. The computing system 1100 further includes: a motion tracker 1120 and an input selection system 1130. The input selection system 1130 includes: a first detection unit 1132, a selection unit 1134, a second detection unit 1136, and a determination unit 1138. The motion tracker 1120 tracks the contact on or near the touch sensing surface to obtain touch information of the contact. The first detection unit 1132 detects a first gesture corresponding to the first touch information of multiple contacts performing an action. The selection unit 1134 selects a first input to the computing system corresponding to the first gesture. The second detection unit 1136 detects a second gesture corresponding to the second touch information of multiple contacts performing an action. The determination unit 1138 determines whether to select a second input to the computing system corresponding to the second gesture, wherein if the third information satisfies a predetermined criterion, the second input is selected, and if the third information does not satisfy the predetermined criterion, the first input is maintained.
[0057] In some embodiments, the number of contacts in the first touch information and the second touch information is different.
[0058] In some embodiments, the number of contacts in the first touch information and the second touch information is the same, and the actions performed by the contacts in the first touch information and the second touch information are different.
[0059] In some embodiments, the third information includes touch information of the contact.
[0060] In some embodiments, the third information includes one of a total number of contacts, a number of contact lift-offs, a time of contact lift-offs, a number of down contacts, a time of down contacts, a number of stationary contacts, and contact motion.
[0061] In some embodiments, the contact movement in the third information includes determining whether one or more contacts within the first touch information are in a scroll stop.
[0062] In some embodiments, the time of the down contact in the third information includes determining whether one or more contacts in the second touch information are down within a predetermined time of a down contact of one or more contacts in the first touch information.
[0063] In some embodiments, the third information includes information about a current operating status.
[0064] In some embodiments, the action includes one of a translational motion, a rotational motion, a zooming motion, a lift-off, and a touch-down.
[0065] In some embodiments, the total number of contacts on or near the touch-sensing surface during the second gesture is greater than the number of contacts that perform an action corresponding to the second gesture.
[0066] According to an embodiment of the present disclosure, a device for selecting an input of a touch sensing surface is provided. The device includes: a device for tracking a contact on or near the touch sensing surface to obtain touch information of the contact; a device for detecting a first gesture corresponding to first touch information of multiple contacts performing an action; a device for selecting a first input corresponding to the first gesture; a device for detecting a second gesture corresponding to second touch information of multiple contacts performing an action; and a device for determining whether to select a second input corresponding to the second gesture, wherein if third information satisfies a predetermined criterion, the second input is selected, and if the third information does not satisfy the predetermined criterion, the first input is maintained.
[0067] Thus, it can be seen from the above that some embodiments of the present disclosure relate to a method of selecting an input of a touch sensing surface, the method comprising: tracking a contact on or near the surface to obtain touch information of the contact; detecting a first gesture corresponding to a first touch information of a plurality of contacts performing an action; selecting a first input corresponding to the first gesture; detecting a second gesture corresponding to a second touch information of a plurality of contacts performing an action; and determining whether to select a second input corresponding to the second gesture, wherein if the third information satisfies a predetermined criterion, the second input is selected, and if the third information does not satisfy the predetermined criterion, the first input is maintained; and wherein the number of contacts in the first information is greater than the number of contacts in the second information. In other embodiments, the actions performed by the contacts in the first touch information and the second touch information are different. In other embodiments, the actions performed by the contacts in the first touch information and the second touch information are the same. In other embodiments, the number of contacts in the first touch information and the second touch information are the same, and the actions performed by the contacts in the first touch information and the second touch information are different. In other embodiments, the third information includes one of the total number of contacts, the number of contacts lifted off, the time of contacts lifted off, the number of down contacts, the time of down contacts, the number of stationary contacts, and contact motion. In other embodiments, the total number of contacts in the third information is equal to the number of contacts in the second information. In other embodiments, the total number of contacts in the third information is greater than the number of contacts in the second information. In other embodiments, the total number of contacts on or near the surface during the second gesture is greater than the number of contacts performing the action corresponding to the second gesture.
[0068] Some embodiments of the present disclosure relate to a computing system including a touch sensing surface, the computing system comprising: a motion tracker for tracking contacts on or near the surface to obtain touch information of the contacts; and an input selection system, the input selection system detecting a first gesture corresponding to first touch information of multiple contacts performing an action, selecting a first input corresponding to the first gesture to the computing system, detecting a second gesture corresponding to second touch information of multiple contacts performing an action, and determining whether to select a second input to the computing system corresponding to the second gesture, wherein if the third information satisfies a predetermined criterion, the second input is selected, and if the third information does not satisfy the predetermined criterion, the first input is maintained. In other embodiments, the third information includes information of a current operating state. In other embodiments, the third information includes one of identification of the first input event, identification of the second input event, a function of the second input event selected by the user, and a lock state of the first input event.
[0069] Some embodiments of the present disclosure relate to a mobile computing device, comprising: a touch sensing system, including: a touch sensing surface, a motion tracker, and an input selection system, the motion tracker tracking contacts on or near the surface to obtain touch information of the contacts, the input selection system detecting a first gesture corresponding to first touch information of multiple contacts performing an action, selecting a first input to the computing system corresponding to the first gesture, detecting a second gesture corresponding to second touch information of multiple contacts performing an action, and determining whether to select a second input to the computing system corresponding to the second gesture, wherein if the third information meets a predetermined criterion, the second input is selected, and if the third information does not meet the predetermined criterion, the first input is maintained.
[0070] Although the disclosed embodiments have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art, and these changes and modifications should be understood to fall within the scope of the disclosed embodiments as defined by the appended claims.
Claims
1. A method comprising: At a computing device having a display and one or more input devices including a touch-sensitive surface: detecting touch information of a plurality of objects stationary on the touch-sensitive surface; while detecting the touch information of the plurality of objects, detecting movement of a first subset of the plurality of objects on the touch-sensitive surface; as well as In response to detecting the movement of the first subset of the multiple objects, while detecting touch information of a second subset of the multiple objects remaining stationary on the touch-sensitive surface, transitioning from a stationary state to a gesture state and performing a gesture input, wherein the gesture input is determined based on the movement of the first subset of the multiple objects. The method according to claim 1 , wherein the gesture input is a pointing input. The method of claim 1 , wherein the gesture input is a scroll input. The method of claim 1 , wherein the gesture input is determined based on a first number of objects in the first subset of the plurality of objects that are detected as being in motion. The method of claim 1 , wherein the second subset of the plurality of objects is ignored when determining the gesture input, the second subset of the plurality of objects being determined based on the first subset of the plurality of objects. The method of claim 1 , wherein the gesture input corresponds to a single finger gesture. The method of claim 1 , wherein the gesture input corresponds to a plurality of finger gestures.
8. The method according to claim 1, further comprising: after transitioning to the gesture state, detecting the first subset of the plurality of objects at rest on the touch-sensitive surface; as well as In response to detecting the first subset of the plurality of objects being stationary on the touch-sensitive surface, aborting transitioning from the gesture state to the stationary state.
9. The method of claim 1, wherein the second subset of the plurality of objects includes at least two objects.
10. A computing system comprising: monitor; one or more input devices, including a touch-sensitive surface; as well as One or more processors configured to execute the method according to any one of claims 1-9.
11. An electronic device, comprising: monitor; one or more input devices, including a touch-sensitive surface; as well as An apparatus for performing the method according to any one of claims 1 to 9.
12. A method comprising: At a computing device having a display and one or more input devices including a touch-sensitive surface: detecting touch information of a plurality of objects stationary on the touch-sensitive surface; while detecting the touch information of the plurality of objects, detecting movement of a subset of the plurality of objects on the touch-sensitive surface; as well as In response to detecting the motion of the subset of the plurality of objects: based on determining that one or more first criteria are satisfied, the one or more first criteria including at least one of the plurality of objects remaining stationary while detecting the motion of the subset of the plurality of objects and the subset of the plurality of objects including a first number of objects, performing a first input based on the motion of the first number of objects; as well as Based on determining that one or more second criteria are satisfied, the one or more second criteria including at least one of the plurality of objects remaining stationary notifying detection of the movement of the subset of the plurality of objects and the subset of the plurality of objects including a second number of objects different from the first number of objects, performing a second input different from the first input based on the movement of the second number of objects.
13. The method of claim 12, wherein the first input is determined based on the first number of objects in the subset of the plurality of objects being detected as moving, and wherein the second input is determined based on the second number of objects in the subset of the plurality of objects being detected as moving.
14. The method according to claim 12, further comprising: detecting touch information for the subset of the plurality of objects at rest on the touch-sensitive surface after detecting the motion of the subset of the plurality of objects and performing corresponding gestures; After detecting the touch information of the subset of the plurality of objects being stationary on the touch-sensitive surface, detecting subsequent movement of a second subset of the plurality of objects; as well as In response to detecting the touch information of the second subset of the plurality of objects at rest on the touch-sensitive surface, performing the corresponding gesture according to the movement of the second subset of the plurality of objects.
15. The method of claim 12, wherein the one or more first criteria and the one or more second criteria include criteria that are satisfied when at least two of the plurality of objects remain stationary to notify detection of the motion of the subset of the plurality of objects.
16. A method of selecting an input to a computer system including a touch sensing surface, the method comprising: tracking one or more contacts on or near the surface to obtain touch information for the contacts; determining a number of the one or more contacts; determining whether the one or more contacts are in motion; determining whether the one or more contacts are at a rolling stop; as well as selecting an input based on the number of contacts, whether the contacts are in motion, and whether the contacts are at a scroll stop; Wherein determining whether the one or more contacts are at a scroll stop further comprises: determining a first average velocity of the one or more contacts over a first time period; determining a second average velocity of the one or more contacts over a second time period, the second time period being shorter than the first time period; and The first average speed and the second average speed are compared to determine whether the one or more contacts are decelerating.
17. The method of claim 16, wherein selecting an input further comprises: When the number of contacts is three and the three contacts are all in a scroll stop motion, selecting a three-finger input; When the number of the contacts is four, and all four contacts stop vertical motion with scrolling, selecting to view all window inputs; When the number of the contacts is four, and all four contacts stop horizontal motion in a scrolling manner, selecting an application switching input; as well as When the number of contacts is two or five, or if not all contacts are in motion, the pointing input is maintained.
18. The method according to claim 16, further comprising: when the number of contacts is one, determining whether at least one additional contact is detected; When at least one additional contact is detected, determining a number of the additional contacts; as well as When the number of the additional contacts is not four, it is determined whether the at least one additional contact is made during the stop of the scrolling.
19. The method of claim 18, wherein selecting an input further comprises: when the number of the additional contacts is three, and if the at least one additional contact is made during scrolling stop, selecting a four-finger input; when the number of the additional contacts is two, and if the at least one additional contact is made during scrolling stop, selecting a three-finger input; as well as When the number of the additional contacts is one, and if the at least one additional contact is made during the scrolling stop, the scroll input or the click input is selected.
20. The method according to claim 17 or 19, wherein the three-finger input is a swipe input or a drag input.
Citation Information
Patent Citations
Touch input transitions
CN107122111A
Gestures for controlling, manipulating, and editing of media files using touch sensitive devices
CN101611373A
