Haptic interface for a computing device - Patents.com
Patent Information
- Application Number
- JP2024535267
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-11-21
- Filing Date
- 2022-11-30
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
【0033】 フィードバック感覚(効果)の各タイプに対して異なる波形が指定される。これらの感覚は、ユーザが確認のためにディスプレイを見る必要なく、ユーザに情報(例えば、これ以上ズームアウトできないこと)を伝えるために選択され得るが、それらは、ユーザがディスプレイから知覚するフィードバックを強化するように構成される。波形は、フィードバック感覚が邪魔はしないが顕著であるように構成され得る。加えて、または代替的に、波形は、フィードバック感覚が互いから容易に区別可能であるように構成され得る。各波形は、波タイプ、振幅、および持続時間を含む、特定のシグネチャを有する。
Smart Images

Figure 2024542856000001_ABST
Abstract
Description
[Background technology]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 290,702, filed December 17, 2021, which claims priority to and the benefit of U.S. Provisional Patent Application No. 17 / 990,893, filed November 21, 2022, the disclosures of which are incorporated herein by reference in their entireties.
[0002] background Computing devices, such as desktop and laptop computers, have a variety of user interfaces that allow users to interact with the computing device. For example, such interfaces may include a keyboard, a mouse or mouse pad, a trackpad, a touch screen, buttons, a stylus, etc. Through these interfaces, a user may control various functions of the computing device and user applications installed on the computing device.
[0003] The point-and-click paradigm began with the mouse as a physical representation of a two-dimensional, single-point cursor on a display screen. Over time, the mouse became overloaded with features to accomplish more and more quickly (e.g., trackball mice offering middle click, double click, right click, etc.). These additional features come with complexity and inflexibility. Trackpads (also known as touchpads, mousepads, or forcepads, depending on size and device type) were introduced to emulate a physical mouse on laptops. In recent years, trackpads, with their large surface area, have been increasingly leveraged for more innovative use cases that utilize multiple fingers, such as gestures.
[0004] A technical problem with this point-and-click paradigm interaction model is that the interaction is asymmetric. The user provides haptic input by "pressing to click," but may need to visually confirm the correctness of the output. For example, regardless of what the user clicks, the user may receive the same springboard recoil feedback. Haptic feedback seeks to close the loop on user / device interaction and provides the operating system with a way to physically interact with the user. This may include providing confirmation of a transaction in a payment app, enhanced realism in a game when using a game controller, feedback via a stylus, or even a fidgeting stress reliever. However, in many cases, users still rely on visually observing what appears on the screen or sounds played by the device to confirm an action or behavior when using a computing device. This can limit interactivity, degrade app or other program performance, and ultimately, negatively impact the quality of the human-machine interface. Summary of the Invention
[0005] Quick Overview Aspects of the present technology employ one or more trackpads configured to apply technical solutions to the asymmetric problem to provide haptic feedback across a variety of apps, interactive scenarios, and device types. The trackpad can be configured to respond to clicks and gestures across its entire area (in other words, corner to corner). This can facilitate the provision of improved user interface feedback, which can result in improved human-machine interfaces. In some implementations, the haptic responses provided by the trackpad can be customized to suit the needs of different users. The present technology provides technical solutions with new dimensions of input force to build various user interface (UI) features. As a result, haptic input and feedback features can be used to provide improved trackpad-based human-computer interfaces on computing devices such as laptops, netbooks, tablets, and other portable computers.
[0006] For example, haptics may provide device interaction in a responsive manner. As an example, haptic feedback may be provided when a user is actively engaged with the device (e.g., moving a hand over a haptic surface) as opposed to being intended to attract the user's attention, such as audible and visual alerts. In some examples, haptics may provide one of multiple mechanisms for completing a particular task for a given app or other program. Thus, even when an app or program runs on a device where haptics may not be available, performance of a particular task may still be possible. Thus, in most cases where possible, haptic feedback may be employed as supplemental or otherwise complementary feedback to the visual cues that a display presents to the user. Haptic feedback may also be complementary to audio information provided. Furthermore, in certain situations, such as when running a word processing or spreadsheet application or interacting with a file management interface, it may be desirable that haptic (e.g., force or duration sensitive) input does not result in a destructive action(s) that are difficult to recover from, such as, for example, permanently emptying the device's trash or deleting all text from a document. When a browser application is running, haptic feedback may be associated with various ways in which a user can interact with the browser UI or content within the browser, such as scrolling or dragging.
[0007] According to one aspect, a method of providing haptic feedback to a user of a computing device is provided. The method includes a user interface module of the computing device receiving a user input associated with a program of the computing device, one or more processors of the computing device determining a type or category (e.g., a kind) of an interaction corresponding to the user input, the one or more processors identifying whether the type or category of the interaction is associated with one or more haptic feedback effects of a set of haptic effects that may be curated, and upon identifying that the type or category of the interaction is associated with one or more haptic feedback effects, the one or more processors selecting a particular haptic feedback effect from the set of haptic effects, and the one or more processors causing a haptic feedback module of the computing device to provide the particular haptic feedback effect for sensation by the user. The set of haptic effects may include a bounce sensation, a limit sensation, a tick mark sensation, a toggle on / off feedback, and / or a click sensation. The set of haptic effects can further include a deep click sensation configured to provide tactile confirmation that the user has pressed hard enough to reveal some information about the selection.
[0008] In one example, when the type of interaction involves splitting a screen in a user interface, the particular haptic feedback effect is a bouncing sensation. In another example, when the type of interaction involves maximizing a window in a user interface, the particular haptic feedback effect is a bouncing sensation. In yet another example, when the type of interaction involves navigating between two or more virtual objects, the particular haptic feedback effect is a tick mark sensation. In a further example, when the type of interaction involves navigating between two or more virtual objects, the particular haptic feedback effect is a bump sensation.
[0009] In one example, when the interaction type is associated with moving a virtual object, the particular haptic feedback effect is a tick mark sensation to indicate that the virtual object is now movable. In another example, when the interaction type involves scrolling a set of displayed content, the particular haptic feedback effect is a hit sensation to indicate that no more scrolling in a given direction is possible. In yet another example, when the interaction type involves separating a tab from a virtual window, the particular haptic feedback effect is a tick mark sensation. In a further example, when the interaction type involves adjusting a control feature, the particular haptic feedback effect is a tick mark sensation.
[0010] In another example, when the interaction type involves turning a selectable option on or off, the particular haptic feedback effect is a toggle sensation, where there may be two different sets of feedback, one for toggling on and one for toggling off, and in yet another example, when the interaction type involves a user input that exceeds either a force threshold or a time threshold, the particular haptic feedback effect is a deep click sensation.
[0011] Alternatively or additionally to any of the above configurations, one or more settings associated with the curated set of haptic effects can be adjustable by a user.
[0012] According to another aspect, a computing device is provided. The computing device includes a memory configured to store a set of curated haptic effects, a display module configured to display a user interface to a user, and a user interface module configured to receive user input from a user of the computing device, the user input being associated with a program of the computing device, the computing device further includes a haptic feedback module configured to provide a haptic effect to the user, and one or more processors operably coupled to the memory, the display module, the user interface module, and the haptic feedback module. The one or more processors are configured to determine a type of interaction corresponding to the user input, identify whether the type of interaction is associated with one or more haptic feedback effects of the curated set of haptic effects, select a particular haptic feedback effect from the set of curated haptic effects upon identifying that the type of interaction is associated with the one or more haptic feedback effects, and cause the haptic feedback module to provide the particular haptic feedback effect for sensation by the user. Each haptic effect in the set of curated haptic effects can be configured to provide complementary or otherwise complementary feedback to the visual cues (and / or audible cues) presented to the user during interaction with the program.
[0013] In one example, the computing device includes a trackpad and the haptic feedback module is part of the trackpad. Alternatively or additionally, the set of curated haptic effects includes a bounce sensation, a limit sensation, a tick mark sensation, and a click sensation. The set of curated haptic effects may further include a deep click sensation, the deep click sensation configured to provide tactile confirmation that the user has pressed hard enough to reveal some information regarding the selection.
[0014] Alternatively or additionally to the above, one or more settings associated with the (curated) set of haptic effects may be adjustable by a user. The set of haptic effects may be curated according to a type or category of user interaction and / or according to a type of application to be executed on the computing device. Each haptic effect in the set of haptic effects may be configured to provide supplemental feedback to visual cues presented to the user during interaction with the program. Each haptic effect may alternatively or additionally be configured to provide supplemental feedback to audible cues presented to the user during interaction with the program. [Brief description of the drawings]
[0015] [Figure 1] 1 illustrates a client computing device in accordance with aspects of the present technology. [Diagram 2] 1 is a block diagram of a client computing device in accordance with aspects of the present technology. [Diagram 3] 14A-14C are diagrams illustrating example click waveforms in accordance with aspects of the present technology; [Figure 4] 14A-14C are diagrams illustrating example deep click waveforms in accordance with aspects of the present technology; [Diagram 5] 14A-14C are diagrams illustrating example tick mark waveforms in accordance with aspects of the present technique; [Figure 6]14A-14C are diagrams illustrating example impingement waveforms, in accordance with aspects of the present technique; [Figure 7] 14A-14C are diagrams illustrating example alert waveforms in accordance with aspects of the present technology; [Figure 8A] 11A-11C are diagrams illustrating example user inputs to a trackpad in accordance with aspects of the present technology. [Figure 8B] 11A-11C are diagrams illustrating example user inputs to a trackpad in accordance with aspects of the present technology. [Figure 9A] 11A-11C are diagrams illustrating other example user inputs, in accordance with aspects of the present technology; [Figure 9B] 11A-11C are diagrams illustrating other example user inputs, in accordance with aspects of the present technology; [Figure 10A] 14A-14C are diagrams illustrating examples of bounce feedback with split screens in accordance with aspects of the present technology; [Figure 10B] 14A-14C are diagrams illustrating examples of bounce feedback with split screens in accordance with aspects of the present technology; [Figure 10C] 14A-14C are diagrams illustrating examples of bounce feedback with split screens in accordance with aspects of the present technology; [Figure 11A] 14A-14C are diagrams illustrating examples of bounce feedback when maximizing a screen in accordance with aspects of the present technology; [Figure 11B] 14A-14C are diagrams illustrating examples of bounce feedback when maximizing a screen in accordance with aspects of the present technology; [Figure 11C] 14A-14C are diagrams illustrating examples of bounce feedback when maximizing a screen in accordance with aspects of the present technology; [Figure 12A] 11A-11C are diagrams illustrating examples of tick mark feedback when switching between virtual desks in accordance with aspects of the present technology. [Figure 12B] 11A-11C are diagrams illustrating examples of tick mark feedback when switching between virtual desks in accordance with aspects of the present technology. [Figure 12C] 11A-11C are diagrams illustrating examples of tick mark feedback when switching between virtual desks in accordance with aspects of the present technology. [Figure 13A]11A-11C are diagrams illustrating examples of hit feedback when a virtual desk is unavailable in accordance with aspects of the present technology; [Figure 13B] 11A-11C are diagrams illustrating examples of hit feedback when a virtual desk is unavailable in accordance with aspects of the present technology; [Figure 13C] 11A-11C are diagrams illustrating examples of hit feedback when a virtual desk is unavailable in accordance with aspects of the present technology; [Figure 14A] 14A-14C are diagrams illustrating examples of tick mark feedback upon the initiation of a drag event in accordance with aspects of the present technology. [Figure 14B] 14A-14C are diagrams illustrating examples of tick mark feedback upon the initiation of a drag event in accordance with aspects of the present technology. [Figure 14C] 14A-14C are diagrams illustrating examples of tick mark feedback upon the initiation of a drag event in accordance with aspects of the present technology. [Figure 15A] 11A-11C are diagrams illustrating examples of tick mark feedback upon initiation of a drag event for a virtual desk scenario in accordance with aspects of the present technology. [Figure 15B] 11A-11C are diagrams illustrating examples of tick mark feedback upon initiation of a drag event for a virtual desk scenario in accordance with aspects of the present technology. [Figure 15C] 11A-11C are diagrams illustrating examples of tick mark feedback upon initiation of a drag event for a virtual desk scenario in accordance with aspects of the present technology. [Figure 15D] 11A-11C are diagrams illustrating examples of tick mark feedback upon initiation of a drag event for a virtual desk scenario in accordance with aspects of the present technology. [Figure 16A] 11A-11C are diagrams illustrating examples of hit-type feedback when scrolling ends in accordance with aspects of the present technology; [Figure 16B] 11A-11C are diagrams illustrating examples of hit-type feedback when scrolling ends in accordance with aspects of the present technology; [Figure 17A] 14A-14C are diagrams illustrating examples of dead-end type feedback when a zoom limit is reached, in accordance with aspects of the present technique; [Figure 17B] 14A-14C are diagrams illustrating examples of dead-end type feedback when a zoom limit is reached, in accordance with aspects of the present technique; [Figure 18A] 14A-14C are diagrams illustrating examples of tick mark feedback upon separating tabs in accordance with aspects of the present technology; [Figure 18B] 14A-14C are diagrams illustrating examples of tick mark feedback upon separating tabs in accordance with aspects of the present technology; [Figure 18C] 14A-14C are diagrams illustrating examples of tick mark feedback upon separating tabs in accordance with aspects of the present technology; [Figure 19A] 11A-11C are diagrams illustrating examples of tick mark feedback upon the onset of a window drag event scenario in accordance with aspects of the present technology. [Figure 19B] 11A-11C are diagrams illustrating examples of tick mark feedback upon the onset of a window drag event scenario in accordance with aspects of the present technology. [Figure 19C] 11A-11C are diagrams illustrating examples of tick mark feedback upon the onset of a window drag event scenario in accordance with aspects of the present technology. [Figure 20A] 14A-14C are diagrams illustrating examples of tick mark feedback with segmented control in accordance with aspects of the present technology; [Figure 20B] 14A-14C are diagrams illustrating examples of tick mark feedback with segmented control in accordance with aspects of the present technology; [Figure 20C] 14A-14C are diagrams illustrating examples of tick mark feedback with segmented control in accordance with aspects of the present technology; [Figure 21A] 14A-14C are diagrams illustrating examples of tick mark feedback when aligning content in accordance with aspects of the present technology. [Figure 21B] 14A-14C are diagrams illustrating examples of tick mark feedback when aligning content in accordance with aspects of the present technology. [Figure 21C] 14A-14C are diagrams illustrating examples of tick mark feedback when aligning content in accordance with aspects of the present technology. [Figure 22A] 14A-14C are diagrams illustrating examples of toggle feedback when changing the state of a setting in accordance with aspects of the present technology; [Figure 22B] 14A-14C are diagrams illustrating examples of toggle feedback when changing the state of a setting in accordance with aspects of the present technology; [Figure 23A] 14A-14C are diagrams illustrating examples of toggle feedback when changing an on / off type setting in accordance with aspects of the present technology. [Figure 23B] 14A-14C are diagrams illustrating examples of toggle feedback when changing an on / off type setting in accordance with aspects of the present technology. [Figure 24A] 14A-14C are diagrams illustrating examples of deep click feedback in accordance with aspects of the present technology; [Figure 24B] 14A-14C are diagrams illustrating examples of deep click feedback in accordance with aspects of the present technology; [Diagram 25] FIG. 1 is a flow diagram according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] Detailed Description Overview Aspects of the present technology relate to providing rich haptic feedback to a user of a computing device to address technical problems involving asymmetric feedback. A technical solution to the problems involving asymmetric feedback involves using a curated set of haptic feedback effects that can be associated with different types of user interactions, types of programs, or other factors. The effects are provided to the user via a trackpad or similar device. The feedback can be configured to include bouncing, such as to indicate that a window or other object in the UI has been moved to a particular side of the display, limiting, such as to indicate that a predefined boundary of the UI has been reached, state changing, such as to indicate that the state of an object or app has changed, and / or toggling on or off, such as to provide a quick confirmation that a feature has been activated or deactivated. Each haptic feedback effect is associated with a different waveform or set of waveforms that provides the technical advantage of easily informing the user about some aspect of the user's interaction with the computing device.
[0017] Example Computing Devices 1A illustrates an example 100 showing a client device 102, such as a laptop computer. In this example, there is a display 104 for visually presenting content, as well as a user input section 108 having a keyboard 110 and a trackpad 112 as different user inputs. A microphone (not shown) may be provided for audible input. The display 104 may be configured as a touch screen. The keyboard 110 may be a virtual keyboard, for example, as part of another display. Although one trackpad 112 is shown, the device may have two or more trackpads disposed thereon.
[0018] The integrated webcam 114 may be used for video conferencing, interactive games, and the like. An indicator 116, such as an LED, may be illuminated to alert the user whenever the integrated webcam is in use. The client device may also include one or more other sensors 118 that may be used to supplement the visual information acquired by the webcam 114. By way of example, the one or more other sensors may include additional imaging devices, RF or ultrasonic based motion sensors, such as for use or acting as a human presence detector to help identify gestures.
[0019] 2 illustrates a block diagram of an exemplary client computing device 200, such as the laptop shown in FIG. 1, or an in-home device such as a tablet PC, netbook, smart display, or the like. As shown, the client computing device includes a processing module 202 having one or more computer processors, such as a central processing unit 204 and / or a graphics processor 206, and a memory module 208 configured to store instructions 210 and data 212. The processors may or may not operate in parallel and may include ASICs, controllers, and other types of hardware circuits. The processor is configured to receive information from a user through a user interface module 214 and to present information to the user on one or more display devices of a display module 216 having a display interface.
[0020] The user interface module 214 may receive commands or other input information from a user via user input and convert them for submission to a given processor. The user interface module may link to a web browser and other apps executed by the device's processing module (not shown). The user input may include one or more of a touch screen, keypad, trackpad, stylus, microphone, or other type of input device. The display module 216 may comprise appropriate circuitry for driving a display device to present graphics and other information to the user. By way of example, the graphics information may be generated by the graphics processor 206, while the CPU 204 manages the overall operation of the client device 200. The graphics information may display responses to user queries on the display module 216. For example, the processing module may execute a browser application or other service using instructions and data stored in the memory module 208 and present information associated with the browser application or other service to the user via the display module 216. The memory module may include a database or other storage for app-related information, etc.
[0021] The memory module 208 may be implemented as one or more of a computer-readable medium(s), a volatile memory device(s), or a non-volatile memory device(s). The memory module 208 may include, for example, a flash memory and / or a NVRAM and may be embodied as a hard drive or a memory card. Alternatively, the memory module 208 may also include a removable medium (e.g., a DVD, a CD-ROM, or a USB thumb drive). One or more areas of the memory module 208 may be writable, while other areas may comprise read-only (or otherwise write-protected) memory. In one implementation, the computer program product is tangibly embodied in an information carrier. Although FIG. 2 functionally illustrates the processor, memory module, and other elements of the client device 200 as being within the same general block, such components may or may not be housed within the same physical housing. For example, some or all of the instructions and data may be stored on an information carrier that is a removable storage medium (eg, an optical drive, a high density tape drive, or a USB drive), while others may be stored in a read-only computer chip.
[0022] The data 212 may be obtained, stored, or modified by the processor according to the instructions 210. For example, the data may be stored in a computing device register, in a relational database as a table with multiple different fields and records, in an XML document, or in a flat file. The data may also be formatted in any computing device readable format.
[0023] The instructions 210 may be any set of instructions to be executed directly (such as machine code) or indirectly (such as a script) by a processor. For example, the instructions may be stored as computing device code on a computing device readable medium. In that regard, the terms "instructions" and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by a processor, or in any other computing device language, including a script or collection of separate source code modules that are interpreted on demand or precompiled.
[0024] 2, the client device 200 includes a communications module 218 for communicating with other devices and systems, including other client devices, servers, and databases. The communications module 218 includes a wireless transceiver, or alternatively, the module may alternatively or additionally include a wired transceiver. The client device 200 may communicate with other remote devices via the communications module 218 using a variety of configurations and protocols, including short-range communications protocols such as Near Field Communication (NFC), Bluetooth, Bluetooth Low Energy (BLE) or other ad-hoc networks, the Internet, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company-proprietary communications protocols, Ethernet, WiFi, and HTTP, as well as combinations of the foregoing.
[0025] The example client device 200 as shown also includes one or more position and orientation sensors 220. The position and orientation sensors 220 are configured to determine the position and orientation of one or more portions of the client computing device 200. For example, these components may include a GPS receiver for determining the latitude, longitude, and / or altitude of the device, as well as another direction / speed detection device, such as an accelerometer, gyroscope, or inertial measurement unit (IMU). The client device 200 may also include one or more cameras 222 for capturing still images and recording video streams, such as an integrated webcam as discussed above, a speaker 224. A power module 226 provides power to the various system components.
[0026] In addition, user input includes a keyboard 228 and a microphone 230, which may comprise a microphone array having one or more transducers or other microphone elements distributed at various points along the housing of the computing device. The track pad 232 may include either a capacitive touch sensor unit 234a or a piezoelectric sensor unit 234b (or both), as well as a haptic feedback module 236. The haptic feedback module may include a controller and one or more feedback elements, e.g., a piezoelectric element configured to provide tactile (haptic) feedback to a user based on one or more electrical signals (waveforms) emitted by the controller. In some configurations, a touch-sensitive display may also incorporate a haptic feedback module.
[0027] Haptic Feedback Effects As noted above, haptic feedback effects may be associated with different types of user interactions, types of programs, or other factors. For example, when dragging a window, the user may receive a "bounce" feedback sensation when the dragged window reaches a bounce target (e.g., split screen or maximize target) within the display area. When dragging a tab out of a browser window, a "tick mark" feedback sensation may be provided when separation occurs. When a limit is reached when zooming in or out of a window, a "limit" feedback sensation may be provided to indicate, for example, that a maximum or minimum zoom or amount level has been reached. Also, a "click" feedback sensation may be provided when changing a preset level for a UI feature such as mouse speed (or otherwise modifying a UI element). The following are further examples of haptic effects that may be employed. None of the examples presented herein are intended to be limiting as to how the technology may be employed.
[0028] Bounce feedback provides tactile confirmation that the user has entered a bounce area in the UI, for example to move a window to the right / left edge of the screen to split the screen, or to move a window to the top of the screen to maximize it.
[0029] Constraint feedback may provide a haptic response to an action that becomes unresponsive after attempting to activate the action outside the action's boundaries. For example, a UI may support different virtual desks, each with a different set of apps or supporting certain features. Here, a user may attempt to use four fingers (or more or less fingers) to swipe a virtual desk or other desktop feature that is unavailable. Or, in another scenario, a user may attempt to keep scrolling when they reach the end of a page.
[0030] Tick mark feedback provides a discrete sense of a state change. For example, this feedback can be provided when there is a long press on an element to initiate a drag, or when three or four fingers are used to swipe between tabs or virtual desks.
[0031] Click feedback provides tactile confirmation that the user has clicked on the trackpad. Deep click feedback may provide tactile confirmation that the user has pressed hard enough to reveal more information about the user's selection within the UI.
[0032] Additionally, toggle (on / off) feedback may provide a brief tactile confirmation of activating / deactivating a feature.
[0033] A different waveform is designated for each type of feedback sensation (effect). These sensations may be selected to convey information to the user (e.g., that they cannot zoom out any further) without the user having to look at the display to confirm, but they are configured to enhance the feedback the user perceives from the display. The waveforms may be configured such that the feedback sensations are noticeable yet unobtrusive. Additionally or alternatively, the waveforms may be configured such that the feedback sensations are easily distinguishable from one another. Each waveform has a particular signature, including wave type, amplitude, and duration.
[0034] One example is a "click" waveform, such as that shown in FIG. 3. The click may include a square wave having a voltage amplitude of about 350mV-750mV (or more or less) with a duration of about 15ms-30ms. In the alternative, a sine wave may be used instead of a square wave. The click waveform parameters may be adjusted as long as the tactile sensation is perceived as sharp and quick. FIG. 4 illustrates a different click called a "deep click." Here, this is a sine wave having a voltage amplitude of about 400mv-600mV (or more or less) with a duration of about 20ms-40ms. As with the click waveform, the deep click waveform parameters may be adjusted. By way of example, the amplitude and / or duration may be increased or decreased, for example, by 10%-30%.
[0035] FIG. 5 illustrates a tick mark waveform that may be generated as a sine wave with a relatively short period of about 10 ms to 25 ms and an amplitude of about 350 mV to 750 mV. FIG. 6 illustrates an example of a bump waveform that may be seen to have a longer damped "bump". Here, the bump may have a peak amplitude of about 250 mV to 500 mV, and a duration of about 40 ms to 80 ms, or more or less. Finally, an example warning waveform is illustrated in FIG. 7. The warning may take different forms to provide, for example, a buzzer-type sensation or a repeating sensation, but a roll waveform is shown having an overall repeating pattern with a periodicity of about 10 ms to 25 ms and a peak amplitude of about 350 mV to 600 mV.
[0036] The above descriptions of waveforms are merely exemplary and the amplitude and / or duration of each may be adjusted as noted.
[0037] A user may interact in different ways via a trackpad of a computing device. For example, as shown in example 800 of FIG. 8A, a human may use a single finger, such as an index finger, to move a mouse around on a display, select an icon, or take some other action. FIG. 8A illustrates an input to a trackpad area of a laptop, while FIG. 8B illustrates another example of an input to a trackpad area of a tablet device. FIG. 9A illustrates a user's hand as the user uses his or her finger to perform a pinch or inverse pinch motion on an input surface (here, with a stylus held in the hand while moving). FIG. 9B also illustrates a user's hand as the user uses his or her finger to perform a pinch or inverse pinch motion on an input surface of a trackpad. In other examples, a user may slide different numbers of fingers on a trackpad to indicate different gestures or other actions.
[0038] In these types of situations, if haptic feedback is provided, the reason the feedback is being provided may be apparent to the user. For example, it may coincide with a visual and / or audible change in the user interface or may be reactive to a particular user action. As an example, selected haptic feedback may be provided simultaneously with a click or buzzer sound that is played while the user is interacting with the app. The device may be configured such that the haptic feedback does not surprise the user. Additionally, haptic feedback may be used relatively sparingly to avoid confusion and to avoid the feedback losing relevance.
[0039] Example Scenario 10A-10C illustrate a scenario involving bounce-back feedback with a split screen. A browser or other app may have multiple windows open simultaneously. As shown in FIG. 10A, there are two tabs within the browser and a window for tab #1 is presented. In FIG. 10B, the user clicks on tab #2, initiating the process of splitting the screen to view both tabs (windows) simultaneously. Upon bringing the window of the second browser tab to the edge of the display area, as shown in FIG. 10C by the thick black border of the displayed content, a bounce-back type feedback sensation is provided to the user via the trackpad to indicate that an event has occurred.
[0040] 11A-11C illustrate a scenario involving bouncing feedback when maximizing a window. Assume that a user wants to focus on a task by maximizing a window they are working on. In this case, as shown in FIGS. 11A-11B, the tab is detached (e.g., from the tab bar) and moved to the edge (e.g., top) of the viewable area of the screen to work on it more comfortably. As shown in FIG. 11C, when the edge (e.g., top) of the window reaches the edge (e.g., top) of the viewable area, a bouncing type feedback sensation is provided to the user via the trackpad to indicate that an action has occurred. Thus, in this scenario, feedback is provided when the window reaches a zone (e.g., top or other edge area) and not when it is released.
[0041] As noted above, the UI may support a set of virtual desks, where each desk may be associated with a different set of apps or may support certain features. For example, Chrome OS supports multiple desktops where different windows or tabs may be associated with a selected desk. Such virtual desks allow a user to organize their work into different virtual spaces. The user can easily move between different desks, which may help maintain focus and productivity. Figures 12A-12C show an example of a user using a hand (e.g., four fingers) to navigate between virtual desks. Here, a tick mark type haptic sensation is received by the user each time the user reaches a new desk (performing a desk switching action) through a gesture along the trackpad (or touchscreen). This provides tactile confirmation to the switching action.
[0042] In some situations, a user may attempt to swipe a desk that is not there. For example, an interface may limit the number of desks to 5-10, but the user may continue the gesture after reaching the last desk. Here, as shown in FIGS. 13A-13C, the user is attempting to move away from the last desk with a browser window visible. The visual transition shown from FIG. 13A to FIG. 13B to FIG. 13C indicates that the window "butts" against the side of the display area. Thus, the user will receive a butting type haptic sensation to indicate to the user that the user is attempting to navigate to a desk that does not exist or is unavailable.
[0043] 14A-14C illustrate examples when the system may provide a tick mark type haptic sensation upon the occurrence of a drag event. As shown in FIG. 14A, the UI displays an app launcher window with various apps available to the user. In FIG. 14B, the user selects a spreadsheet app, as indicated by the arrows. And in FIG. 14C, the user drags that app to rearrange the launcher window. Here, state change feedback via a tick mark sensation is received by the user, for example, when the user long clicks or presses the app icon to lift it and begin dragging it. In other words, upon applying the long click (or press), the app icon becomes movable. Similar feedback may be applied for other system interface elements other than the launcher window, such as a shelf or tray that contains one or more app launch icons.
[0044] 15A-15D illustrate another example when the system can provide a tick mark type haptic sensation based on a drag event, specifically a virtual desk drag event. As shown in these figures, a user can rearrange the virtual desk to better organize their work, such as by moving icons in a tray located along the upper region of the display. For example, a user can open an "overview" option for the virtual desk, which presents a tray. State-change (e.g., tick mark type) haptic feedback is then generated when the user long-clicks or presses the virtual desk to lift it up and begin dragging it.
[0045] 16A-16B illustrate an example for providing a bump-type feedback when scrolling ends. In this scenario, a user may read content from a website and continue scrolling to read more. However, upon reaching the end of the page, a bump-type haptic sensation is selected as feedback to indicate that there is no more scrolling in the current direction. FIGs. 17A-17B illustrate another example for providing a bump-type feedback when zooming in or out on a UI. Here, an image (or web page, slide, or other content) may be enlarged or decreased in size within a viewable area, such as by using a multi-finger gesture. If the zoom limit is reached but the gesture continues, feedback indicating that limit, such as a bump sensation, is communicated to the user.
[0046] 18A-18C illustrate an example of incorporating tick mark type feedback when "splitting" a tab, such as in a browser UI. In this example, a user may have many tabs open in a browser, but would like to focus on one to accomplish some task. By splitting a tab (tab #2) as shown in FIG. 18B, the user can create a separate browser window with only that tab, as shown in FIG. 18C. As an example, state change (tick mark type) feedback may be generated when a user long clicks / presses a browser tab and begins to lift and drag it, thus indicating to the user that the browser tab may now be moved.
[0047] 19A-19C illustrate examples where the system provides tick mark type haptic sensations based on drag events, particularly window drag events. As shown in these figures, a user may rearrange the virtual desk to better organize their work, such as by moving a window from one desk to another to better organize their work. State change (e.g., tick mark type) haptic feedback is generated when the user long clicks or presses on a window to lift it up and begin dragging it, thus indicating to the user that the browser tab can now be moved.
[0048] 20A-C illustrate an example of providing tick mark type feedback when using segmented controls, where tick mark feedback is generated when a user moves the cursor to a new position within a segmented control element (here, "delay before repeat", where the segmented control element spans a range of "long" and "short").
[0049] 21A-21C illustrate examples of providing tick mark type feedback when updating content, such as modifying a slide in a presentation app or moving an item in a drawing program. As shown in FIG. 21A, the text "Italy" is in a text box. A user may update this content by moving elements around and aligning them. Thus, in FIG. 21B, a horizontal bar is shown above the text box to indicate where the user is moving the cursor for updated placement, and FIG. 21C shows the moved text box. In this type of scenario, state change (e.g., tick mark) feedback is generated by the system when user input is received that moves, resizes, rotates, or otherwise adjusts an element to achieve some alignment with another feature of the app (e.g., aligning or snapping to a grid).
[0050] 22A-22B illustrate an example of providing toggle-type feedback when a user clicks on an adjustable icon. For example, as shown in FIG. 22A, there may be several settings presented in a quick settings box on the side of the UI. As shown in FIG. 22B, when a user selects a notification icon to modify it, for example, from "all apps on" to "off" or "selected apps on", toggle haptic feedback is provided. This may also be done when a feature has only two states, such as on / off. An example of this is shown in FIG. 23A-23B, where a user may toggle a "tap to click enable" setting between off (FIG. 23A) and on (FIG. 23B). In such an example, the toggle "on" feedback is different from the toggle "off" feedback, so that the user can easily identify whether the desired feature is enabled (toggle on) or disabled (toggle off). For example, one or more checkboxes may be used in the notification details view to enable or disable notifications from a particular app. Disabling toggles off such notifications, including the haptic component.
[0051] 24A-24B illustrate an example of providing deep click feedback in response to a user providing a touch input with a force that exceeds a threshold or with a duration that exceeds a temporal threshold (e.g., greater than 0.25 seconds, or greater or less than 0.25 seconds). For example, FIG. 24A shows a website in a browser. As shown in FIG. 24B, a user may place a cursor near a word of interest (e.g., "options") because the user wants to know more about it immediately without having to leave the context. By pressing hard enough, long enough, or otherwise indicating interest in the word, image, or other content item (e.g., by moving the cursor around the item to encircle or highlight it), a deep click haptic sensation is generated, preferably at the same time that the system provides supplemental information about the content item. Thus, in this example, the system may pop up a list of related options, link to a different web page about those items, or perform some other action associated with the options.
[0052] Another example includes providing haptic feedback when dragging and dropping between apps, e.g., when dragging into a droppable area. A further example includes providing haptic feedback when entering overview mode. This may include updating a current threshold gesture (e.g., a three-finger swipe up) with a continuing gesture that follows the user's movement and / or feedback when the threshold for launching the overview is met.
[0053] As shown in FIGS. 23A-B, the UI may introduce certain settings related to haptic touchpad capabilities. For example, click strength may allow the user to select how hard the user presses the trackpad. Haptic feedback may also be enabled and turned on (or off) by default. Turning it off may disable all advanced haptic feedback. The user may be able to tweak haptic sensation parameters, such as the amplitude and / or duration of wavelengths for different haptic feedback. Alternatively or additionally, in some scenarios, the user may be able to select which haptic feedback is associated with a certain user-device interaction. Thus, the system may provide a curated set of haptic feedback effects associated with different types of user interactions, types of programs, or other factors, while adjustments may be allowed by the user for some or all of the effects. For example, the set of haptic effects may be curated according to types or categories of user interactions, types of applications to be executed on the computing device, or both. This may include multi-level curation, where when a particular application is running, the system may select a subset of haptic effects that correspond to that application. In one scenario, all apps for a particular client device may have the same initial set of haptic effects. In another scenario, each app may be individually curated according to the type of UI and / or how a user can interact with the app. If any user preferences exist, they may be applied to further curate the haptic effects. Also, depending on what is occurring through the user's interaction with the program, one or more particular haptic effects from the curated set may be selected for use.Thus, while the various examples and scenarios described above and shown in the figures are illustrative of selected situations, variations to the haptic interface may exist.
[0054] According to one aspect of the present technology, when a user interacts with a computing device, the system can use information about the type of interaction, the type of app or other program being used, etc., to identify and select a haptic feedback effect from a set of curated haptic effects (e.g., from a list stored in memory). When a given condition is met by the interaction and / or the state of the app or other program, the system causes the trackpad (or other component) to generate the selected haptic feedback effect for sensation by the user. In some cases, the user may be able to modify one or more settings associated with the set of curated haptic effects, which may affect the identification or selection of a given effect in a particular user interaction scenario. In other cases, the user may only be able to modify the intensity of the click feedback.
[0055] FIG. 25 illustrates a method 2500 for providing haptic feedback to a user of a computing device according to the above. At block 2502, the method includes a user interface module of the computing device receiving a user input associated with a program of the computing device. At block 2504, the method includes one or more processors of the computing device determining a type of interaction corresponding to the user input. At block 2506, the method includes the one or more processors identifying whether the type of interaction is associated with one or more haptic feedback effects of the curated set of haptic effects. At block 2508, the method includes the one or more processors selecting a particular haptic feedback effect from the curated set of haptic effects upon identifying that the type of interaction is associated with one or more haptic feedback effects. At block 2510, the method includes the one or more processors causing a haptic feedback module of the computing device to provide a particular haptic feedback effect for sensation by the user.
[0056] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive and may be implemented in various combinations to achieve inherent advantages. These and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, and therefore the foregoing description of the embodiments should be considered as illustrative, rather than limiting, the subject matter defined by the claims. In addition, the provision of examples described herein, as well as clauses expressed as "such as," "including," and the like, should not be construed as limiting the subject matter of the claims to any particular example, but rather, the example is intended to illustrate only one of many possible embodiments. Furthermore, the same reference numbers in different drawings may identify the same or similar elements.
Claims
1. 1. A method for providing haptic feedback to a user of a computing device, comprising: a user interface module of the computing device receiving user input at the computing device; determining, by one or more processors of the computing device, a type or category of interaction corresponding to the user input; identifying whether a type or category of the interaction is associated with one or more haptic feedback effects of a set of haptic effects; Upon identifying that the type or category of interaction is associated with one or more haptic feedback effects, the one or more processors select a particular haptic feedback effect from the set of haptic effects; and causing a haptic feedback module of the computing device to provide the particular haptic feedback effect for sensation by the user; A method comprising:
2. The method of claim 1 , wherein the set of haptic effects includes a bounce sensation, a constraint sensation, a tick mark sensation, and a click sensation.
3. 3. The method of claim 2, wherein the series of haptic effects further includes a deep click sensation, the deep click sensation configured to provide tactile confirmation that the user pressed hard enough to reveal some information regarding a selection.
4. The method of claim 1 , wherein when the type or category of interaction involves splitting a screen in a user interface, the particular haptic feedback effect is a bouncing sensation.
5. The method of claim 1 , wherein when the type or category of interaction involves maximizing a window in a user interface, the particular haptic feedback effect is a bouncing sensation.
6. The method of claim 1 , wherein when the type or category of interaction involves moving between two or more virtual objects, the particular haptic feedback effect is a tick mark sensation.
7. The method of claim 1 , wherein when the type or category of interaction involves moving between two or more virtual objects, the particular haptic feedback effect is a bump sensation.
8. The method of claim 1 , wherein when the type or category of interaction is associated with moving a virtual object, the particular haptic feedback effect is a tick mark sensation indicating that the virtual object is currently movable.
9. 2. The method of claim 1, wherein when the type or category of interaction involves scrolling through a set of displayed content, the particular haptic feedback effect is a hit sensation to indicate that no more scrolling is possible in a given direction.
10. The method of claim 1 , wherein when the type or category of interaction involves separating a tab from a virtual window, the particular haptic feedback effect is a tick mark sensation.
11. The method of claim 1 , wherein when the type or category of interaction involves adjusting a control function, the particular haptic feedback effect is a tick mark sensation.
12. 2. The method of claim 1 , wherein when the type or category of interaction involves turning a selectable option on or off, the particular haptic feedback effect comprises a set of toggle sensations including at least one toggle-on sensation and one toggle-off sensation different from the toggle-on sensation.
13. The method of claim 1 , wherein when the type or category of interaction involves the user input exceeding either a force threshold or a time threshold, the haptic feedback effect is a deep click sensation.
14. The method of claim 1 , wherein one or more settings associated with the set of haptic effects are adjustable by the user.
15. The method of claim 1 , wherein the set of haptic effects is curated according to a type or category of the user interaction.
16. The method of claim 1 , wherein the set of haptic effects is curated according to a type of application to be executed on the computing device.
17. 1. A computing device comprising: a memory configured to store a sequence of haptic effects; a display module configured to display a user interface to a user; a user interface module configured to receive user input from a user of the computing device, the user input being associated with a program on the computing device, the computing device comprising: a haptic feedback module configured to provide a haptic effect to the user; and one or more processors operably coupled to the memory, the display module, the user interface module, and the haptic feedback module, the one or more processors: determining a type or category of interaction corresponding to the user input; identifying whether the type or category of interaction is associated with one or more haptic feedback effects of the set of haptic effects; selecting a particular haptic feedback effect from the set of haptic effects upon identifying that the type or category of interaction is associated with one or more haptic feedback effects; causing the particular haptic feedback module to provide the particular haptic feedback effect for sensation by the user; and 23. A computing device configured to:
18. The computing device of claim 17 , comprising a track pad, the haptic feedback module being part of the track pad.
19. The computing device of claim 17 , wherein the set of haptic effects includes a bounce sensation, a constraint sensation, a tick mark sensation, and a click sensation.
20. The computing device of claim 19 , wherein the set of haptic effects further comprises a set of toggle effects including a toggle-on effect and a toggle-off effect different from the toggle-on effect.
21. 20. The computing device of claim 19, wherein the series of haptic effects further includes a deep click sensation configured to provide tactile confirmation that the user pressed hard enough to reveal some information regarding a selection.
22. The computing device of claim 17 , wherein one or more settings associated with the set of haptic effects are adjustable by the user.
23. The computing device of claim 17 , wherein each haptic effect in the series of haptic effects is configured to provide supplemental feedback to visual cues presented to the user while interacting with the program.
24. The computing device of claim 17 , wherein each haptic effect in the series of haptic effects is configured to provide supplemental feedback to audible cues presented to the user while interacting with the program.
Citation Information
Patent Citations
Display system with tactile guidance
JP2005512241A
A semantic framework for variable haptic output
JP2017531850A
Human Computer Interface System
JP2019515370A
Image selection suggestions
JP2021504803A
Motion based pointing apparatus providing haptic feedback and control method thereof
US20100295667A1
