Haptic Interfaces for Computing Devices

Abstract

It provides a haptic feedback effect. [Solution] User input associated with a program on a computing device is received by the user interface module of the computing device. The processor of the computing device determines the type of interaction corresponding to the user input. The processor identifies whether the type of interaction is associated with a haptic feedback effect from a curated set of haptic effects. When identifying that the type of interaction is associated with a haptic feedback effect, the processor selects a specific haptic feedback effect from the curated set of haptic effects. The processor causes the haptic feedback module of the computing device to provide the specific haptic feedback effect for the user's sensation.

Description

Background Art

[0001] Cross - reference to related applications This application claims priority and the benefit of the filing date of U.S. Patent Application No. 17 / 990,893, filed on November 21, 2022, which claims priority and the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 290,702, filed on December 17, 2021, the entire disclosures of which are incorporated herein by reference.

[0002] Background Computing devices such as desktop and laptop computers have various user interfaces that enable a user to interact with the computing device. For example, such interfaces may include a keyboard, a mouse or mouse pad, a trackpad, a touch screen, buttons, a stylus, and the like. Through these interfaces, a user can control various functions of the computing device and user applications installed on the computing device.

[0003] The point - and - click paradigm started with the mouse as a physical representation of a two - dimensional single - point cursor on a display screen. Gradually, the mouse became overloaded with functions to achieve more things more quickly (e.g., trackball mice that provide middle - click, double - click, right - click, etc.). These additional functions are accompanied by complexity and lack of flexibility. The trackpad (also known as a touchpad, mouse pad, or force pad depending on size and device type) was introduced to emulate a physical mouse on a laptop. In recent years, trackpads with large surface areas are being increasingly utilized for more innovative use cases that utilize multiple fingers, such as gestures.

[0004] The technical problem with interaction models in this point-and-click paradigm is that the interaction is asymmetrical. Users provide haptic input by "pressing to click," but may need to visually confirm the accuracy of the output. For example, a user may receive the same dive board rebound feedback regardless of what they click. Haptic feedback requires closing the loop in user / device interaction and provides the operating system with a way to physically interact with the user. This can include providing transaction confirmation in payment apps, enhanced realism in games when using a game controller, feedback via a stylus, or even fiddle stress relief. However, often users still rely on visually observing what appears on the screen or on sounds played by the device to confirm an action or behavior when using a computing device. This limits interactivity, degrades the performance of apps or other programs, and ultimately negatively impacts the quality of the human-machine interface. [Overview of the Initiative]

[0005] Brief Overview A part of this technology employs one or more trackpads configured to apply a technical solution to an asymmetric problem, providing haptic feedback across various apps, interactive scenarios, and device types. The trackpad provides haptic feedback for clicks and gestures across its entire surface (in other words, from corner to corner). It can be configured to respond in various ways. This can facilitate the provisioning of improved user interface feedback, which can result in an improved human-machine interface. In some implementations, the haptic responses provided by the trackpad can be customized to suit the needs of different users. This technology provides a technical solution involving a new dimension of input force to build various user interface (UI) features. As a result, haptic input and feedback features can be used to provide improved trackpad-based human-computer interfaces on computing devices such as laptops, netbooks, tablets, and other portable computers.

[0006] For example, haptics can provide device interaction in a responsive manner. For instance, haptic feedback can be provided when the user is actively engaging with the device (e.g., moving their hand over a haptic surface), as opposed to audible and visual alerts, which are intended to attract the user's attention. In some cases, haptics can provide one of several mechanisms for completing a particular task for a given app or other program. Therefore, even when the app or program runs on a device where haptics may not be available, the performance of a particular task may still be possible. Thus, in as many cases as possible, haptic feedback can be employed as supplementary or otherwise complementary feedback to visual cues presented by the display to the user. Haptic feedback can also be complementary to the auditory information provided. Furthermore, in certain situations, such as when running document processing or spreadsheet applications, or interacting with file management interfaces, it may be desirable that haptic (e.g., force or duration-sensitive) input does not result in destructive actions that are difficult to recover, such as permanently emptying the device's trash or deleting all text from a document. When a browser application is running, haptic feedback can be associated with various ways in which the user can interact with the browser UI or content within the browser, such as scrolling or dragging.

[0007] According to one embodiment, a method is provided for providing haptic feedback to a user of a computing device. The method includes: a user interface module of the computing device receiving user input associated with a program of the computing device; one or more processors of the computing device determining a type or category (e.g., type) of interaction corresponding to the user input; one or more processors identifying whether the type or category of interaction is associated with one or more haptic feedback effects of a set of curated haptic effects; when identifying that the type or category of interaction is associated with one or more haptic feedback effects, one or more processors selecting a specific haptic feedback effect from the set of haptic effects; and one or more processors causing a haptic feedback module of the computing device to provide the specific haptic feedback effect for the user's sensation. The set of haptic effects may include bounce sensation, limit sensation, tick mark sensation, toggle-on / off feedback, and / or click sensation. The series of haptic effects may further include a deep-click sensation, which is configured to provide tactile confirmation that the user has pressed hard enough to reveal certain information regarding a selection.

[0008] In one example, when the type of interaction involves splitting the screen within the user interface, a specific haptic feedback effect is a bouncing sensation. In another example, when the type of interaction involves maximizing a window within the user interface, a specific haptic feedback effect is a bouncing sensation. In yet another example, when the type of interaction involves moving between two or more virtual objects, a specific haptic feedback effect is a tick mark sensation. In yet another example, when the type of interaction involves moving between two or more virtual objects, a specific haptic feedback effect is a dead end sensation.

[0009] In one example, when the type of interaction is associated with moving a virtual object, a particular haptic feedback effect is a tick mark sensation indicating that the virtual object is currently movable. In another example, when the type of interaction involves scrolling through a set of displayed content, a particular haptic feedback effect is a dead end sensation indicating that scrolling in a given direction is no longer possible. In yet another example, when the type of interaction involves separating a tab from a virtual window, a particular haptic feedback effect is a tick mark sensation. In yet another example, when the type of interaction involves adjusting a control feature, a particular haptic feedback effect is a tick mark sensation.

[0010] In another example, when the type of interaction involves turning a selectable option on or off, a particular haptic feedback effect is a toggle sensation. Here, there may be two distinct sets of feedback: one for turning the toggle on and the other for turning it off. Yet another example, when the type of interaction involves user input exceeding either a force threshold or a temporal threshold, a particular haptic feedback effect is a deep click sensation.

[0011] Alternatively or in addition to any of the above configurations, one or more settings associated with a curated set of haptic effects may be adjustable by the user.

[0012] In another embodiment, a computing device is provided. The computing device comprises a memory configured to store a curated set of haptic effects, a display module configured to display a user interface to a user, and a user interface module configured to receive user input from a user of the computing device, the user input being associated with a program of the computing device, and the computing device further comprises a haptic feedback module configured to provide haptic effects to the user, and one or more processors operably coupled to the memory, the display module, the user interface module, and the haptic feedback module. The one or more processors are configured to determine a type of interaction corresponding to the user input, to identify whether the type of interaction is associated with one or more haptic feedback effects of the curated set of haptic effects, to select a particular haptic feedback effect from the curated set of haptic effects when identifying that the type of interaction is associated with one or more haptic feedback effects, and to cause the haptic feedback module to provide a particular haptic feedback effect for the user's sensation. Each haptic effect in the curated set of haptic effects is a visual cue (and) presented to the user during interaction with the program. It may be configured to provide supplementary or otherwise complementary feedback to (or audible cues).

[0013] In one example, a computing device includes a trackpad, and a haptic feedback module is part of the trackpad. Alternatively or additionally, a curated set of haptic effects may include bounce, constraint, tick mark, and click sensations. A curated set of haptic effects may further include a deep click sensation, which is configured to provide tactile confirmation that the user has pressed hard enough to reveal some information about a selection.

[0014] Alternatively or additionally to the above, one or more settings associated with a (curated) set of haptic effects may be adjustable by the user. The set of haptic effects may be curated according to the type or category of user interaction, and / or the type of application to be run on the computing device. Each haptic effect in the set may be configured to provide supplementary feedback to visual cues presented to the user during interaction with the program. Each haptic effect may, alternatively or additionally, be configured to provide supplementary feedback to audible cues presented to the user during interaction with the program. [Brief explanation of the drawing]

[0015] [Figure 1] This figure illustrates a client computing device according to an embodiment of this technology. [Figure 2] This is a block diagram of a client computing device according to an embodiment of this technology. [Figure 3] This figure illustrates an example of a click waveform according to an embodiment of this technology. [Figure 4] This figure illustrates an example of a deep-click waveform according to an embodiment of this technology. [Figure 5] This figure illustrates an example of a tick mark waveform according to an embodiment of this technology. [Figure 6]A diagram illustrating an example of a collision waveform according to an aspect of the present technology. [Figure 7] A diagram illustrating an example of an alert waveform according to an aspect of the present technology. [Figure 8A] A diagram illustrating an example of user input to a trackpad according to an aspect of the present technology. [Figure 8B] A diagram illustrating an example of user input to a trackpad according to an aspect of the present technology. [Figure 9A] A diagram illustrating an example of other user input according to an aspect of the present technology. [Figure 9B] A diagram illustrating an example of other user input according to an aspect of the present technology. [Figure 10A] A diagram illustrating an example of bounce-back feedback with a split screen according to an aspect of the present technology. [Figure 10B] A diagram illustrating an example of bounce-back feedback with a split screen according to an aspect of the present technology. [Figure 10C] A diagram illustrating an example of bounce-back feedback with a split screen according to an aspect of the present technology. [Figure 11A] A diagram illustrating an example of bounce-back feedback when maximizing the screen according to an aspect of the present technology. [Figure 11B] A diagram illustrating an example of bounce-back feedback when maximizing the screen according to an aspect of the present technology. [Figure 11C] A diagram illustrating an example of bounce-back feedback when maximizing the screen according to an aspect of the present technology. [Figure 12A] A diagram illustrating an example of tick mark feedback when switching between virtual desktops according to an aspect of the present technology. [Figure 12B] A diagram illustrating an example of tick mark feedback when switching between virtual desktops according to an aspect of the present technology. [Figure 12C] A diagram illustrating an example of tick mark feedback when switching between virtual desktops according to an aspect of the present technology. [Figure 13A]A diagram illustrating an example of a collision feedback when a virtual desk according to an aspect of the present technology is unavailable. [Figure 13B] A diagram illustrating an example of a collision feedback when a virtual desk according to an aspect of the present technology is unavailable. [Figure 13C] A diagram illustrating an example of a collision feedback when a virtual desk according to an aspect of the present technology is unavailable. [Figure 14A] A diagram illustrating an example of a tick mark feedback at the start of a drag event according to an aspect of the present technology. [Figure 14B] A diagram illustrating an example of a tick mark feedback at the start of a drag event according to an aspect of the present technology. [Figure 14C] A diagram illustrating an example of a tick mark feedback at the start of a drag event according to an aspect of the present technology. [Figure 15A] A diagram illustrating an example of a tick mark feedback at the start of a drag event for a virtual desk scenario according to an aspect of the present technology. [Figure 15B] A diagram illustrating an example of a tick mark feedback at the start of a drag event for a virtual desk scenario according to an aspect of the present technology. [Figure 15C] A diagram illustrating an example of a tick mark feedback at the start of a drag event for a virtual desk scenario according to an aspect of the present technology. [Figure 15D] A diagram illustrating an example of a tick mark feedback at the start of a drag event for a virtual desk scenario according to an aspect of the present technology. [Figure 16A] A diagram illustrating an example of a collision type feedback when scrolling ends according to an aspect of the present technology. [Figure 16B] A diagram illustrating an example of a collision type feedback when scrolling ends according to an aspect of the present technology. [Figure 17A] A diagram illustrating an example of a collision type feedback when a zoom limit is reached according to an aspect of the present technology. [Figure 17B] This figure illustrates an example of a dead-end type feedback when the zoom limit is reached according to this embodiment of the technology. [Figure 18A] This figure illustrates an example of tick mark feedback when separating tabs according to an embodiment of this technology. [Figure 18B] This figure illustrates an example of tick mark feedback when separating tabs according to an embodiment of this technology. [Figure 18C] This figure illustrates an example of tick mark feedback when separating tabs according to an embodiment of this technology. [Figure 19A] This figure illustrates an example of tick mark feedback at the start of a window drag event scenario according to an aspect of this technology. [Figure 19B] This figure illustrates an example of tick mark feedback at the start of a window drag event scenario according to an aspect of this technology. [Figure 19C] This figure illustrates an example of tick mark feedback at the start of a window drag event scenario according to an aspect of this technology. [Figure 20A] This figure illustrates an example of tick-mark feedback with segmented control according to an embodiment of this technology. [Figure 20B] This figure illustrates an example of tick-mark feedback with segmented control according to an embodiment of this technology. [Figure 20C] This figure illustrates an example of tick-mark feedback with segmented control according to an embodiment of this technology. [Figure 21A] This figure illustrates an example of tick mark feedback when aligning content using an embodiment of this technology. [Figure 21B] This figure illustrates an example of tick mark feedback when aligning content using an embodiment of this technology. [Figure 21C] This figure illustrates an example of tick mark feedback when aligning content using an embodiment of this technology. [Figure 22A] This figure illustrates an example of toggle feedback when changing the setting state according to an embodiment of this technology. [Figure 22B] This figure illustrates an example of toggle feedback when changing the setting state according to an embodiment of this technology. [Figure 23A] This figure illustrates an example of toggle feedback when changing the on / off type setting according to an embodiment of this technology. [Figure 23B] This figure illustrates an example of toggle feedback when changing the on / off type setting according to an embodiment of this technology. [Figure 24A] This figure illustrates an example of deep click feedback according to an embodiment of this technology. [Figure 24B] This figure illustrates an example of deep click feedback according to an embodiment of this technology. [Figure 25] This is a flowchart according to the aspects of this disclosure. [Modes for carrying out the invention]

[0016] Detailed explanation Overview Aspects of this technology relate to providing rich haptic feedback to users of computing devices in order to address technical problems related to asymmetric feedback. Technical solutions to problems related to asymmetric feedback involve using a curated set of haptic feedback effects that can be associated with different types of user interaction, program type, or other factors. These effects are provided to the user via a trackpad or similar device. Feedback may include bounces, such as indicating that a window or other object in the UI has been moved to a particular side of the display; constraints, such as indicating that a default boundary of the UI has been reached; state changes, such as indicating that the state of an object or app has been changed; and / or toggles, such as providing quick confirmation that a feature has been activated or deactivated. Each haptic feedback effect is associated with a different waveform or set of waveforms that provides the technical advantage of easily informing the user about some aspect of the user's interaction with the computing device.

[0017] Example computing device Figure 1A illustrates an example 100 showing a client device 102, such as a laptop computer. In this example, there is a display 104 for visually presenting content, as well as a user input section 108 having a keyboard 110 and a trackpad 112 as different user inputs. A microphone (not shown) may be provided for audible input. The display 104 may be configured as a touchscreen. The keyboard 110 may be a virtual keyboard, for example, as part of another display. Although one trackpad 112 is shown, the device may have two or more trackpads positioned on it.

[0018] The integrated webcam 114 can be used for video conferencing, interactive games, etc. An indicator 116, such as an LED, indicates when the integrated webcam is in use. It can be illuminated at any time to warn the user. The client device may also include one or more other sensors 118 that can be used to complement the visual information acquired by the webcam 114. For example, one or more other sensors may include additional imaging devices, RF or ultrasonic-based motion sensors, etc., for use as a human presence detector or to help identify gestures by action.

[0019] Figure 2 illustrates a block diagram of an example client computing device 200, such as the laptop shown in Figure 1, or a home device such as a tablet PC, netbook, smart display, or similar. As shown, the client computing device includes a processing module 202 having one or more computer processors, such as a central processing unit 204 and / or a graphics processor 206, and a memory module 208 configured to store instructions 210 and data 212. The processors may or may not operate in parallel and may include ASICs, controllers, and other types of hardware circuitry. The processors are configured to receive information from the user through a user interface module 214 and to present information to the user on one or more display devices of a display module 216 having a display interface.

[0020] The user interface module 214 may receive commands or other input information from the user via user input and convert them for submission to a given processor. The user interface module may link to a web browser and other applications executed by the device's processing module (not shown). User input may include one or more of the following: touchscreen, keypad, trackpad, stylus, microphone, or other types of input devices. The display module 216 may have appropriate circuitry for driving a display device to present graphics and other information to the user. For example, graphic information may be generated by the graphics processor 206, while the CPU 204 manages the overall operation of the client device 200. Graphic information may display responses to user queries to the display module 216. For example, the processing module may use instructions and data stored in the memory module 208 to execute a browser application or other service and present information associated with that browser application or other service to the user via the display module 216. The memory module may include a database or other storage for application-related information, etc.

[0021] The memory module 208 may be implemented as one or more computer-readable media, volatile memory devices, or non-volatile memory devices. The memory module 208 may, for example, include flash memory and / or NVRAM and may be implemented as a hard drive or memory card. Alternatively, the memory module 208 may also include removable media (e.g., DVD, CD-ROM, or USB thumb drive). One or more areas of the memory module 208 may be writable, while other areas may comprise read-only (or otherwise write-protected) memory. In one implementation, a computer program product is tangibly embodied in an information carrier. Figure 2 functionally illustrates the processor, memory module, and other elements of the client device 200 as being within the same overall block, although such components may or may not be housed in the same physical housing. For example, some or all of the instructions and data may be stored on an information carrier, which is a removable storage medium (e.g., an optical drive, a high-density tape drive, or a USB drive), while others may be stored in a read-only computer chip.

[0022] The data 212 can be retrieved, stored, or modified by the processor in accordance with instruction 210. For example, the data may be stored in a computing device register, in a relational database as a table with multiple different fields and records, in an XML document, or in a flat file. The data can also be formatted in any computing device-readable format.

[0023] Instruction 210 may be any set of instructions to be executed by the processor directly (e.g., in machine code) or indirectly (e.g., in a script). For example, instructions may be stored as computing device code on a computing device-readable medium. In this regard, the terms “instruction” and “program” may be used synonymously herein. Instructions may be stored in object code format for direct processing by the processor, or in any other computing device language, including scripts or collections of independent source code modules that are interpreted on demand or pre-compiled.

[0024] As also shown in Figure 2, the client device 200 includes a communication module 218 for communicating with other devices and systems, including other client devices, servers, and databases. The communication module 218 includes a wireless transceiver, and alternatively, the module may also include a wired transceiver. The client device 200 may communicate with other remote devices via the communication module 218 using a variety of configurations and protocols, including short-range communication protocols such as Near Field Communication (NFC), Bluetooth®, Bluetooth® Low Energy (BLE) or other ad-hoc networks, the Internet, intranets, virtual private networks, wide area networks, local networks, private networks using communication protocols proprietary to one or more companies, Ethernet®, WiFi®, and HTTP, as well as combinations thereof.

[0025] A client device 200, as shown in the example, also includes one or more position and orientation sensors 220. The position and orientation sensors 220 are configured to determine the position and orientation of one or more parts of the client computing device 200. For example, these components may include a GPS receiver for determining the latitude, longitude, and / or altitude of the device, as well as other orientation / velocity sensing devices such as an accelerometer, gyroscope, or inertial measurement unit (IMU). The client device 200 may also include one or more cameras 222 for capturing still images and recording video streams, such as an integrated webcam as discussed above, and a speaker 224. A power module 226 provides power to various system components.

[0026] In addition, user input includes a keyboard 228 and a microphone 230 which may have a microphone array with one or more transducers or other microphone elements distributed at various points along the housing of the computing device. The trackpad 232 may include either (or both) a capacitive touch sensor unit 234a or a piezoelectric sensor unit 234b, as well as a haptic feedback module 236. The haptic feedback module may include a controller and one or more feedback elements, such as a piezoelectric element configured to provide haptic feedback to the user based on one or more electrical signals (waveforms) emitted by the controller. In some configurations, a touch-sensitive display may also incorporate a haptic feedback module.

[0027] Haptic Feedback Effect As noted above, haptic feedback effects are different types of user input Haptic feedback can be associated with actions, program types, or other factors. For example, when dragging a window, the user may receive a "bounce" feedback sensation when the dragged window reaches a bounce target within the display area (e.g., split screen or maximizing the target). When dragging a tab outside a browser window, a "tick mark" feedback sensation may be provided when separation occurs. When zooming in or out of a window, a "limit" feedback sensation may be provided when a limit is reached, for example, to indicate that the maximum or minimum zoom or quantity level has been reached. Also, a "click" feedback sensation may be provided when changing a preset level for UI features such as mouse speed (or otherwise modifying a UI element). The following are further examples of haptic effects that may be employed. None of the examples presented herein are intended to limit how the technology may be employed.

[0028] Bounce feedback provides tactile confirmation that the user has entered a bounce area within the UI, for example, to move a window to the right / left edge of the screen to split the screen, or to move a window to the top of the screen to maximize it.

[0029] Restriction feedback can provide haptic responses to actions that become unresponsive after attempting to activate an action outside its boundaries. For example, a UI might support different virtual desks, each having a different set of apps or supporting specific features. Here, the user might attempt to swipe with four fingers (or more or fewer fingers) to reach an unavailable virtual desk or other desktop feature. Or, in another scenario, the user might attempt to continue scrolling when they have reached the end of a page.

[0030] Tick ​​mark feedback provides a discrete sense of state change. For example, this feedback may be provided when there is a long press on an element to initiate a drag, or when three or four fingers are used to swipe between tabs or virtual desks.

[0031] Click feedback provides haptic confirmation that a user has clicked on the trackpad. Deep click feedback can provide haptic confirmation that a user has pressed hard enough to reveal further information about their selection within the UI.

[0032] Additionally, toggle (on / off) feedback can provide a brief tactile confirmation to activate / deactivate a feature.

[0033] A different waveform is assigned to each type of feedback sensation (effect). These sensations may be selected to convey information to the user (e.g., that they cannot zoom out any further) without requiring the user to look at the display for confirmation, but they are configured to enhance the feedback the user perceives from the display. The waveforms may be configured so that the feedback sensation is not intrusive but noticeable. In addition, or alternatively, the waveforms may be configured so that the feedback sensations are easily distinguishable from one another. Each waveform has a specific signature, including wave type, amplitude, and duration.

[0034] One example is the "click" waveform, as shown in Figure 3. A click can include a square wave with a voltage amplitude of approximately 350mV to 750mV (or more or less) over a period of approximately 15ms to 30ms. Alternatively, a sine wave may be used instead of the square wave. Click waveform parameters can be adjusted as long as the tactile sensation is perceived as sharp and rapid. Figure 4 illustrates a different type of click called a "deep click." Here, this is a sine wave with a voltage amplitude of approximately 400mV to 600mV (or more or less) over a period of approximately 20ms to 40ms. Similar to the click waveform, the deep click waveform parameters can be adjusted. For example, the amplitude and / or duration can be increased or decreased by, for example, 10% to 30%.

[0035] Figure 5 illustrates a tick mark waveform that can be generated as a sine wave with a relatively short duration of approximately 10 ms to 25 ms and an amplitude of approximately 350 mV to 750 mV. Figure 6 illustrates an example of a stab waveform that may appear to have a longer, more decayed "stab". Here, the stab may have a peak amplitude of approximately 250 mV to 500 mV and a duration of approximately 40 ms to 80 ms, or more or less. Finally, an example of a warning waveform is illustrated in Figure 7. Warnings may take different forms, for example, to give a buzzer-type sensation or a repeating sensation, but a roll waveform with an overall repeating pattern having a periodicity of approximately 10 ms to 25 ms and a peak amplitude of approximately 350 mV to 600 mV is shown.

[0036] The above description of the waveforms is for illustrative purposes only, and as noted, each amplitude and / or duration can be adjusted.

[0037] Users can interact with a computing device's trackpad in different ways. For example, as shown in Example 800 in Figure 8A, a person may use a single finger, such as their index finger, to move a mouse around on the display, select an icon, or perform some other action. Figure 8A illustrates input to the trackpad area of ​​a laptop, while Figure 8B illustrates another example of input to the trackpad area of ​​a tablet device. Figure 9A illustrates a user's hand performing a pinch or reverse pinch motion on the input surface using their fingers (here, while holding a stylus in their hand while moving their fingers). Figure 9B also illustrates a user's hand performing a pinch or reverse pinch motion on the input surface of a trackpad using their fingers. In other examples, a user may slide a different number of fingers across the trackpad to indicate different gestures or other actions.

[0038] In these types of situations, when haptic feedback is provided, the reason for the feedback may be obvious to the user. For example, it may coincide with visual and / or audible changes within the user interface, or it may be reactive to a specific user action. As an example, selected haptic feedback may be provided simultaneously with a click or buzzer sound played while the user interacts with the app. The device may be configured so that the haptic feedback does not startle the user. In addition, haptic feedback may be used relatively sparingly to avoid confusion and to avoid the feedback losing relevance.

[0039] Example scenario Figures 10A–10C illustrate a scenario involving bounce feedback with a split screen. A browser or other application can have multiple windows open simultaneously. As shown in Figure 10A, there are two tabs in the browser, with the window of tab #1 presented. In Figure 10B, the user clicks tab #2, initiating the process of splitting the screen to view both tabs (windows) simultaneously. As shown in Figure 10C by the thick black border of the displayed content, when the window of the second browser tab is brought to the edge of the display area, a bounce-type feedback sensation is provided to the user via the trackpad to indicate that the event has occurred.

[0040] Figures 11A–11C illustrate a scenario involving bounce feedback when maximizing a window. Here, we assume the user wants to concentrate on a task by maximizing the window they are working on. In this case, as shown in Figures 11A–11B, the tab is detached (e.g., from the tab bar) and moved to the edge of the visible area of ​​the screen (e.g., the top) for more comfortable work. As shown in Figure 11C, when the edge of the window (e.g., the top) reaches the edge of the visible area (e.g., the top), a bounce-type feedback sensation is provided to the user via the trackpad to indicate that the action has occurred. Thus, in this scenario, the feedback is provided when the zone (e.g., the top or other edge area) is reached, rather than when the window is released.

[0041] As noted above, the UI may support a set of virtual desks, each desk may be associated with a different set of apps or may support specific features. For example, Chrome OS has multiple desks, each with different windows or tabs that can be associated with a selected desk. It supports desktops. Such virtual desks allow users to organize their work across different virtual spaces. Users can easily move between different desks, which can help maintain focus and productivity. Figures 12A–12C show an example of a user using their hand (e.g., four fingers) to navigate between virtual desks. Here, a tick-mark type haptic sensation is received by the user each time they reach a new desk (perform a desk switching action) through gestures along the trackpad (or touchscreen). This provides tactile confirmation of the switching action.

[0042] In some situations, a user may attempt to swipe to a desk that is not present. For example, the interface may limit the number of desks to 5-10, but the user may continue the gesture after reaching the last desk. Here, as shown in Figures 13A-13C, the user is attempting to move away from the last desk while the browser window is displayed. The visual transitions shown from Figure 13A to Figure 13B and then to Figure 13C indicate that the window "hits" the side of the display area. Therefore, the user receives a haptic sensation of hitting a dead end to indicate that they are attempting to navigate to a desk that is either non-existent or unavailable.

[0043] Figures 14A–14C illustrate an example of when a system may provide a tick-mark type haptic sensation when a drag event occurs. As shown in Figure 14A, the UI displays an app launcher window with various apps available to the user. In Figure 14B, as indicated by the arrow, the user has selected a spreadsheet app. Also in Figure 14C, the user is dragging that app to rearrange the launcher window. Here, state change feedback via a tick-mark sensation is received by the user, for example, when the user long-clicks or presses an app icon to lift it and begin to drag it. In other words, when a long click (or press) is applied, the app icon becomes movable. Similar feedback may be applied for other system interface elements other than the launcher window, such as a shelf or tray containing one or more app launcher icons.

[0044] Figures 15A–15D illustrate another example of when a system may provide a tick-mark type haptic sensation based on drag events, particularly virtual desk drag events. As shown in these figures, a user may rearrange their virtual desk to better organize their work, for example, by moving icons in a tray located along the upper area of ​​the display. For example, a user may open an "Overview" option for the virtual desk, which presents the tray. Then, a state change (e.g., a tick mark) Haptic feedback is generated when the user long-clicks or presses on the virtual desk and then begins to lift and drag it.

[0045] Figures 16A and 16B illustrate an example of providing a dead-end type of feedback when scrolling ends. In this scenario, the user is reading content on a website and may continue scrolling to read more. However, when the end of the page is reached, a dead-end haptic sensation is selected as feedback to indicate that there is no further scrolling in the current direction. Figures 17A and 17B illustrate another example of providing a dead-end type of feedback when zooming in or zooming out on a UI. Here, an image (or web page, slide, or other content) may be enlarged or reduced in size within the visible area, for example, by using a multi-finger gesture. If the gesture continues while the zoom limit has been reached, feedback indicating that limit, such as a dead-end sensation, is the communication to the user.

[0046] Figures 18A–18C illustrate an example of incorporating tick-mark type feedback when "separating" tabs within a browser UI. In this example, the user may have many tabs open in the browser, but wants to focus on one to perform a certain task. By separating the tab (tab #2) as shown in Figure 18B, the user can create a separate browser window containing only that tab, as shown in Figure 18C. For example, state change (tick-mark type) feedback may be generated when the user long-clicks / presses a browser tab and begins to lift and drag it, thus indicating to the user that the browser tab can now be moved.

[0047] Figures 19A–19C illustrate an example where the system provides tick-mark type haptic feedback based on drag events, particularly window drag events. As shown in these figures, the user can better organize their work by rearranging their virtual desks, such as moving windows from one desk to another. State change (e.g., tick-mark type) haptic feedback is generated when the user long-clicks or presses a window to lift it and begin to drag it, thus indicating to the user that the browser tab can now be moved.

[0048] Figures 20A–20C illustrate an example of providing tick-mark type feedback when using segmented control. Here, tick-mark feedback is generated when the user moves the cursor to a new position within the segmented control element (here, "delay before repetition," where the segmented control element spans "long" and "short" ranges).

[0049] Figures 21A–21C illustrate examples of providing tick-type feedback when updating content, such as modifying slides in a presentation app or moving items in a drawing program. As shown in Figure 21A, the text "Italy" is in a text box. The user moves the element around. This content can be updated by moving and aligning them. Thus, in Figure 21B, a horizontal bar is shown above the text box to indicate where the user is moving the cursor for the updated arrangement, and Figure 21C shows the moved text box. In this type of scenario, state change (e.g., a tick mark) feedback is generated by the system when user input is received that moves, resizes, rotates, or otherwise adjusts the elements to achieve some alignment with another feature of the app (e.g., alignment to a grid or snapping).

[0050] Figures 22A and 22B illustrate an example of providing toggle-type feedback when a user clicks an adjustable icon. For example, as shown in Figure 22A, there may be several settings presented in a quick settings box next to the UI. As shown in Figure 22B, toggle haptic feedback is provided when a user selects a notification icon and modifies it, for example, from "Turn on all apps" to "Off" or "Turn on selected apps." This can also be done when a feature has only two states, such as on / off. An example of this is shown in Figures 23A and 23B, where a user may toggle the "Tap to enable click" setting between off (Figure 23A) and on (Figure 23B). In such an example, the toggle "on" feedback is different from the toggle "off" feedback, so the user can easily identify whether the desired feature is enabled (toggle on) or disabled (toggle off). For example, one or more checkboxes may be used in a notification details view to enable or disable notifications from a specific app. Disabling it toggles off such notifications, including haptic components.

[0051] Figures 24A and 24B illustrate an example of providing deep click feedback in response to a user providing touch input with force exceeding a threshold or with a duration exceeding a temporal threshold (e.g., longer or shorter than 0.25 seconds). For example, Figure 24A shows a website in a browser. As shown in Figure 24B, the user may have placed the cursor near a desired word (e.g., "options") because the user wants to know more about it immediately without having to leave the context. By pressing hard enough and for long enough, or by showing interest in that word, image, or other content item in other ways (e.g., by moving the cursor around the item to circle or highlight it), a deep click haptic sensation is generated, preferably at the same time as the system provides supplementary information about that content item. Thus, in this example, the system may pop up a list of relevant options, link to different web pages about those items, or perform some other action associated with the options.

[0052] Another example involves providing haptic feedback when dragging and dropping between apps, for example, when dragging to a droppable area. A further example involves providing haptic feedback when entering Overview mode. This could involve updating the current threshold gesture (e.g., a three-finger swipe up) with a continuous gesture that follows the user's movements and / or feedback when the threshold for launching Overview is met.

[0053] As shown in Figures 23A and 23B, the UI may introduce settings related to haptic touchpad capabilities. For example, click intensity may allow the user to choose how hard they press the trackpad. Haptic feedback may also be enabled and may be on (or off) by default. Turning it off may disable all advanced haptic feedback. The user may also be able to fine-tune haptic sensation parameters, such as the amplitude and / or duration of the wavelength for different haptic feedback. Alternatively or additionally, in some scenarios, the user may be able to choose which haptic feedback is associated with which user-device interaction. Thus, the system may provide a curated set of haptic feedback effects associated with different types of user interactions, program types, or other factors, while the user may be allowed to adjust some or all of the effects. For example, a set of haptic effects may be curated according to the type or category of user interaction, the type of application to be run on the computing device, or both. This may include multi-level curation, in which, when a particular application is running, the system may select a subset of haptic effects corresponding to that application. In one scenario, all apps for a particular client device may have the same initial set of haptic effects. In another scenario, each app may be individually curated according to the type of UI and / or how the user can interact with the app. Wherever user preferences exist, they may be applied to further curate the haptic effects. Also, depending on what is happening through the user's interaction with the program, one or more specific haptic effects from the curated set may be selected for use. Thus, the various examples and scenarios described above and shown in the diagrams illustrate the chosen situation, but variations of the haptic interface may exist.

[0054] According to one aspect of this technology, when a user interacts with a computing device, the system can use information about the type of interaction, the type of app or other program being used, etc., to identify and select a haptic feedback effect from a curated set of haptic effects (e.g., from a list stored in memory). When given conditions are met by the state of the interaction and / or the app or other program, the system causes the trackpad (or other component) to generate the selected haptic feedback effect for the user's sensation. In some cases, the user may modify one or more settings associated with the curated set of haptic effects, which may affect the identification or selection of a given effect in a particular user interaction scenario. In other cases, the user may only be able to modify the intensity of click feedback.

[0055] Figure 25 illustrates Method 2500 for providing haptic feedback to a user of a computing device as described above. In block 2502, the Method includes a user interface module of the computing device receiving user input associated with a program of the computing device. In block 2504, the Method includes one or more processors of the computing device determining the type of interaction corresponding to the user input. In block 2506, the Method includes one or more processors identifying whether the type of interaction is associated with one or more haptic feedback effects of a curated set of haptic effects. In block 2508, when identifying whether the type of interaction is associated with one or more haptic feedback effects, the Method includes one or more processors selecting a specific haptic feedback effect from the curated set of haptic effects. In block 2510, the Method includes one or more processors causing a haptic feedback module of the computing device to provide a specific haptic feedback effect for the user's sensation.

[0056] Unless otherwise stated, the alternative examples described herein are not mutually exclusive and may be implemented in various combinations to achieve their respective advantages. Since these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the above description of embodiments should be considered illustrative rather than limiting the subject matter defined by the claims. In addition, the provision of examples described herein, as well as sections expressed as “etc,” “including,” and similar, should not be interpreted as limiting the subject matter of the claims to specific examples, but rather that the examples are intended to illustrate only one of many possible embodiments. Furthermore, the same reference numeral in different drawings may refer to the same Alternatively, similar elements can be identified.

Claims

1. A method for providing haptic feedback to a user of a computing device, The user interface module of the computing device receives user input in the computing device, One or more processors of the computing device determine the type or category of interaction corresponding to the user input, The one or more processors identify whether the type or category of the interaction is associated with one or more haptic feedback effects of a set of haptic effects, When identifying that the type or category of interaction is associated with one or more haptic feedback effects, the one or more processors select a specific haptic feedback effect from the set of haptic effects, The one or more processors cause the haptic feedback module of the computing device to provide the specific haptic feedback effect for the user's sensation. Methods that include...

2. The method according to claim 1, wherein the series of haptic effects include a rebound sensation, a limiting sensation, a tick mark sensation, and a click sensation.

3. The method according to claim 2, wherein the series of haptic effects further include a deep-click sensation, the deep-click sensation is configured to provide tactile confirmation that the user has pressed hard enough to reveal some information regarding a selection.

4. The method according to claim 1, wherein the particular haptic feedback effect is a bouncy sensation when the type or category of interaction involves dividing the screen within the user interface.

5. The method according to claim 1, wherein the particular haptic feedback effect is a bouncy sensation when the type or category of interaction involves maximizing a window within the user interface.

6. The method according to claim 1, wherein when the type or category of interaction involves moving between two or more virtual objects, the particular haptic feedback effect is a tick mark sensation.

7. The method according to claim 1, wherein when the type or category of interaction involves moving between two or more virtual objects, the particular haptic feedback effect is a sense of hitting a dead end.

8. The method according to claim 1, wherein, when the type or category of interaction is associated with moving a virtual object, the particular haptic feedback effect is a tick mark sensation indicating that the virtual object is now movable.

9. When the type or category of interaction involves scrolling through a set of displayed content, the particular haptic feedback effect is scroll The method according to claim 1, wherein the roll has a sense of dead end to indicate that it cannot go any further in a given direction.

10. The method according to claim 1, wherein the type or category of interaction is involved in separating the tab from the virtual window, and the particular haptic feedback effect is a tick mark sensation.

11. The method according to claim 1, wherein the particular haptic feedback effect is a tick mark sensation when the type or category of interaction is involved in modulating the control function.

12. The method according to claim 1, wherein, when the type or category of interaction involves turning a selectable option on or off, the particular haptic feedback effect includes a set of toggle sensations, each including at least one toggle-on sensation and one toggle-off sensation distinct from the toggle-on sensation.

13. The method according to claim 1, wherein when the type or category of interaction involves the user input exceeding either a force threshold or a temporal threshold, the haptic feedback effect is a deep click sensation.

14. The method according to claim 1, wherein one or more settings associated with the series of haptic effects are adjustable by the user.

15. The method according to claim 1, wherein the series of haptic effects are curated according to the type or category of the user interaction.

16. The method according to claim 1, wherein the series of haptic effects are curated according to the type of application to be run on the computing device.

17. A computing device, A memory configured to store a series of haptic effects, A display module configured to show a user interface to the user, The computing device comprises a user interface module configured to receive user input from a user of the computing device, wherein the user input is associated with a program of the computing device, and the computing device A haptic feedback module configured to provide a haptic effect to the user, The memory, the display module, the user interface module, and the haptic feedback module are further comprising one or more processors operably coupled to the memory, the display module, the user interface module, and the haptic feedback module, the one or more processors Determining the type or category of interaction corresponding to the aforementioned user input, Identifying whether the type or category of the interaction is associated with one or more haptic feedback effects of the set of haptic effects, When identifying that the type or category of interaction is associated with one or more haptic feedback effects, the process involves selecting a specific haptic feedback effect from the set of haptic effects. The aforementioned specific haptic feedback module is provided for the user's sensations. To provide the aforementioned specific haptic feedback effect A computing device configured to do so.

18. The computing device according to claim 17, comprising a trackpad, wherein the haptic feedback module is part of the trackpad.

19. The computing device according to claim 17, wherein the series of haptic effects include a bounce sensation, a limiting sensation, a tick mark sensation, and a click sensation.

20. The computing device according to claim 19, wherein the series of haptic effects further includes a set of toggle effects, the set of haptic effects including a toggle-on effect and a toggle-off effect distinct from the toggle-on effect.

21. The computing device according to claim 19, wherein the series of haptic effects further include a deep-click sensation, the deep-click sensation is configured to provide tactile confirmation that the user has pressed hard enough to reveal some information regarding a selection.

22. The computing device according to claim 17, wherein one or more settings associated with the series of haptic effects are adjustable by the user.

23. The computing device according to claim 17, wherein each of the series of haptic effects is configured to provide supplementary feedback to visual cues presented to the user during interaction with the program.

24. The computing device according to claim 17, wherein each of the series of haptic effects is configured to provide supplementary feedback to an audible cue presented to the user during interaction with the program.

Images