Device, method and graphical user interface for displaying affordances on a background

Abstract

The present disclosure relates to a device, method, and graphical user interface for displaying an affordance on a background. An electronic device has a display and a touch-sensitive surface, which displays content and affordances on the display. The affordance is displayed on a portion of the content; the value of the same display attribute of the affordance is determined based on the value of the display attribute of a portion of the content of the affordance; and the value of the display attribute of the content is allowed to vary within a first value range and the value of the display attribute of the affordance is constrained to vary within a second value range that is smaller than the first value range; when displaying the content and affordance, the device detects a change in the appearance of the content of the affordance and in response to detecting a change in the appearance of the content of the affordance, the device changes the appearance of the affordance, including: increasing / decreasing the value of the display attribute of the affordance according to the magnitude of the change in the value of the display attribute of the content and the second value range, based on determining that the value of the display attribute of the content has decreased / increased.

Description

[0001] This application is a divisional application of the invention patent application with the application date of January 25, 2018, application number 201880001526.8, and invention name "Device, method and graphical user interface for displaying capability indication on a background". Technical Field

[0002] The present invention generally relates to electronic devices having a display and a touch-sensitive surface, including but not limited to electronic devices that display virtual enable representations (e.g., controls, indicators, visual guides, etc.) on their touch-sensitive display screens. Background Art

[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touch screen displays. Such surfaces are widely used to manipulate user interfaces on displays and objects therein. Exemplary user interface objects include digital images, videos, text, icons, and control elements (such as buttons) and other graphics.

[0004] Electronic computing devices often display virtual controls or visual guides on their displays. For example, keyboards, menus, dialog boxes, alerts, and other controls can be activated and manipulated (e.g., via touch input) to perform operations on a portable electronic device (e.g., a smartphone, tablet, or laptop). Indicators and visual guides can be overlaid on a background (e.g., a user interface of an application or a user interface of an operating system) to provide visual cues associated with the background or a particular area of ​​the screen regarding the type of input that can be provided and / or the type of operation that can be performed.

[0005] Existing methods for displaying controls, indicators, and visual guides can be cumbersome and inefficient. For example, controls, indicators, and visual guides can be unnecessarily distracting to a user as they navigate a user interface, or not clear or prominent enough against the background, leading to user errors and user confusion when the user interacts with the device, which can also negatively impact the energy consumption of the device. This latter consideration is particularly important in battery-powered devices.

[0006] In addition, some types of affordances are displayed over a variety of backgrounds and content, sometimes without movement for extended periods of time. As a result, a display may exhibit a ghost image (or afterimage) of the affordance after a period of use. Reducing and eliminating display afterimages is a long-standing challenge for display device manufacturers. Some existing approaches to addressing this problem (such as introducing screen savers or flashing icons) are not satisfactory solutions because they have side effects (e.g., causing eye fatigue, distraction, etc.) and lack efficacy in many cases (e.g., only available when the device is idle). Summary of the invention

[0007] Therefore, there is a need for electronic devices with prominent but less distracting affordances (e.g., virtual spaces, indicators, and visual guides) that help provide sufficient visual distinction to guide a user to provide the required input to achieve a desired result without unnecessarily distracting the user from content displayed on an application or system user interface. Additionally, for certain affordances that are displayed for an extended period of time without movement, there is a need to display the affordances in a manner that reduces or eliminates screen afterimages.

[0008] In addition, as background content continues to change over time, either automatically or in response to user input, the appearance of the affordance also needs to dynamically adapt in order to continue to remain efficient and effective for the purposes described above. Specifically, affordances that serve as controls or visual guides for gestures that trigger some commonly used system functions are displayed in many different scenarios (e.g., on user interfaces of different applications and operating systems). Sometimes, the background of the affordance in a given scenario (e.g., scrollable content, rapidly changing content, or unpredictable content) also changes dynamically, further requiring the appearance of the affordance to continuously adapt to changes in the appearance of the background after the affordance was initially displayed on the background. An example of such an affordance is a system-level affordance for indicating the starting area of ​​a home / multitasking gesture, which results in the dismissal of the currently displayed application user interface and the display of the home screen user interface, or the dismissal of a cover-like system information interface (e.g., a notification center or lock screen user interface) and the display of a previously displayed user interface (e.g., an application user interface or a home screen).

[0009] Additionally, sometimes, after an affordance is displayed on the application user interface, an operational context change occurs in the application, which can qualitatively change the likelihood that the user will interact with the affordance or need visual guidance from the affordance. In such cases, the balance between keeping the affordance prominent and making the affordance less distracting needs to be adjusted to maintain the effectiveness and efficiency of the user interface.

[0010] Additionally, for certain affordances that are displayed for an extended period of time without movement, it may be desirable to display the affordances in a manner that reduces or eliminates screen afterimages.

[0011] The above needs require new methods and interfaces to display affordances and adjust the appearance of affordances (e.g., virtual controls, indicators, and visual guides) on the background. Such devices, methods, and interfaces can reduce the cognitive burden on users and produce more effective human-computer interfaces. In addition, such devices, methods, and interfaces can reduce or eliminate screen afterimages, thereby reducing equipment maintenance costs and extending equipment life.

[0012] In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, a tablet computer, or a handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has a touch pad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, a memory, and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI mainly through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, functions optionally include image editing, drawing, presentation, word processing, spreadsheet production, games, telephone, video conferencing, email, instant messaging, exercise support, digital shooting, digital video recording, web browsing, digital music playback, note recording, digital video playback, and system-level operations (such as, displaying a home screen, locking a device, displaying a system-level notification screen, displaying a system-level control panel user interface, etc.). Executable instructions for performing these functions are optionally included in non-transitory computer-readable storage media or other computer program products configured for execution by one or more processors.

[0013] According to some embodiments, a method includes: at a device having a display and a touch-sensitive surface: displaying content and an enable representation on the display, wherein: the enable representation is displayed on a portion of the content; determining a value of the display attribute of the enable representation based on the value of the display attribute of the portion of the content on which the enable representation is displayed; and allowing the value of the display attribute of the content to vary within a first value range, and constraining the value of the display attribute of the enable representation to vary within a second value range, which is smaller than the first value range; when displaying the content and the enable representation, detecting a change in the appearance of the content displayed by the enable representation; and in response to detecting a change in the appearance of the content displayed by the enable representation, changing the appearance of the enable representation, including: based on determining that the value of the display attribute of the content has decreased, increasing the value of the display attribute of the enable representation based on the magnitude of the change in the value of the display attribute of the content and the second value range; and based on determining that the value of the display attribute of the content has increased, decreasing the value of the display attribute of the enable representation based on the magnitude of the change in the value of the display attribute of the content and the second value range.

[0014] According to some embodiments, a method includes: at a device having a display and a touch-sensitive surface: displaying a user interface of an application; when the user interface of the application is displayed in a first mode, displaying an enable representation having a first appearance on the user interface, wherein: the enable representation is displayed on a portion of the user interface, and values ​​of a set of one or more display attributes of the enable representation having the first appearance change according to a first set of one or more rules, according to changes in values ​​of a set of one or more display attributes of a portion of the user interface located below the enable representation; when the enable representation having the first appearance is displayed on a portion of the user interface displayed in the first mode, detecting a request to transition from displaying the user interface in the first mode to displaying the user interface in a second mode; and in response to detecting the request: displaying the user interface in the second mode; and displaying an enable representation having a second appearance on the user interface displayed in the second mode, wherein: values ​​of a set of one or more display attributes of the enable representation having the second appearance change according to a second set of one or more rules different from the first set of one or more rules, according to changes in values ​​of a set of one or more display attributes of the portion of the user interface located below the enable representation;

[0015] According to some embodiments, a method includes: at a device having a display and a touch-sensitive surface: displaying content and an enable representation on the display, wherein: the enable representation is displayed on a portion of the content; determining a value of the same display attribute of the enable representation based on the value of the display attribute of the portion of the content on which the enable representation is displayed; and allowing the value of the display attribute of the content to vary within a first value range, and constraining the value of the display attribute of the enable representation to vary within an enable representation appearance value range that is smaller than the first value range; when displaying the content and the enable representation and when the enable representation appearance value range is within a second value range, detecting a change in the appearance of the content displayed by the enable representation; and in response to detecting a change in the appearance of the display attribute of the enable representation The appearance change of the content that can be represented, changing the appearance of the capable representation, including: based on determining that the appearance change of the content meets the range switching standard: transferring the capable representation appearance value range to a third value range, wherein the third value range is different from the second value range, and the third value range is smaller than the first value range; and changing the value of the same display attribute of the capable representation based on the value of the display attribute of the content that can be represented, wherein the display attribute of the capable representation is constrained to change within the capable representation appearance value range; and based on determining that the appearance change of the content does not meet the range switching standard, changing the value of the same display attribute of the capable representation based on the value of the display attribute of the content that can be represented, while maintaining the capable representation appearance value range to the second value range.

[0016] According to some embodiments, an electronic device includes: a display; a touch-sensitive surface; optionally one or more memories for detecting the intensity of contact with the touch-sensitive surface; optionally one or more tactile output generators; one or more processors; and a memory storing one or more programs; the one or more programs are configured to be executed by one or more processors, and the one or more programs include instructions for performing or causing the operation of any method described herein to be performed. According to some embodiments, a non-transitory computer-readable storage medium stores therein instructions that, when executed by an electronic device, cause the device to perform the operation of any method described herein or cause the operation of any method described herein to be performed, the electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators. According to some embodiments, a graphical user interface located on an electronic device includes one or more elements displayed in any method described herein, the one or more elements being updated in response to input, the electronic device having a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, a memory, and one or more processors for executing one or more programs stored in the memory, as described in any method described herein. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, one or more optional sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators. And a device for performing or causing the operation of any of the methods described herein to be performed. According to some embodiments, an information processing device used in an electronic device includes a device for performing the operation of any method described herein or causing the operation of any method described herein to be performed, and the electronic device has a display, a touch-sensitive surface, one or more optional sensors for detecting the intensity of contact with the touch-sensitive surface, and optionally one or more tactile output generators.

[0017] Thus, improved methods and interfaces for navigating between user interfaces and interacting with control objects are provided for electronic devices having a display, a touch-sensitive surface, optionally one or more sensors for detecting intensity of contact with the touch-sensitive surface, optionally one or more tactile output generators, optionally one or more device orientation sensors, and optionally an audio system, thereby improving the effectiveness, efficiency, and user satisfaction of these devices. Such methods and interfaces can supplement or replace conventional methods for displaying affordances on a background. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0019] This patent or patent application file contains at least one drawing executed in color. Copies of this patent or patent application publication with one or more color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0020] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments.

[0021] Figure 1B is a block diagram illustrating example components for event processing according to some embodiments.

[0022] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.

[0023] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0024] Figure 4A An exemplary user interface for an application menu on a portable multifunction device is shown according to some embodiments.

[0025] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0026] FIG. 5A to FIG. 5D An exemplary user interface is shown that includes an enable representation having an appearance that adapts to a background appearance according to some embodiments.

[0027] Figure 5E According to some embodiments, a method for generating a FIG. 5A to FIG. 5D The affordances in the filter represent the appearance of the filter.

[0028] Fig. 5F A method for performing Figure 5E An exemplary reversal curve for the reversal shown in .

[0029] Figure 5G to Figure 5K A change in the appearance of an enable representation of a first enable representation appearance type (e.g., a "dark" enable representation type) is shown according to some embodiments.

[0030] Figure 5L to Figure 5PA change in the appearance of an enable indication of a second enable indication appearance type (e.g., a "bright" enable indication type) is shown according to some embodiments.

[0031] Figure 5Q The difference in appearance of two types of affordances when the same change occurs in the background is shown according to some embodiments.

[0032] Figure 5R The value range and inverse relationship of the display attributes of the capable representation according to some embodiments and the content below the implied capable representation appearance type and the highlighted capable representation appearance type are shown.

[0033] Figure 5S to Figure 5AA A user interface according to some embodiments is shown, which includes an enabling representation having an appearance that changes in response to a background and an operating mode that changes associated with the background.

[0034] Figures 5AB to 5AC The differences in the appearance of the enabling representation of the background in different operating modes according to some embodiments are shown.

[0035] Figure 5AD A user interface according to some embodiments is shown, which includes an enable representation that dynamically switches between enable representation appearance types based on changes in underlying content over time.

[0036] Figure 5AE The value range and inverse relationship of the display attributes of the enabling representation according to some embodiments and the content below the highlighted enabling representation appearance type, the hinted enabling representation appearance type and the transition enabling representation appearance type are shown.

[0037] Figure 5AF A gradual shift of the indicateable appearance type from a suggested indicateable appearance type to a highlighted indicateable appearance type through multiple transition indicateable appearance types according to some embodiments is shown.

[0038] Figures 5AG to 5AK is the display attribute of the affordance Figure 5AF The different affordances shown in are enlarged copies of the inverse relationships of the content below the type of representation.

[0039] FIG. 6A to FIG. 6C is a flowchart illustrating a method of changing the appearance of an affordance based on changes in the appearance of underlying content according to some embodiments.

[0040] 7A to 7E is a flow chart illustrating a method for changing the appearance of an affordance based on changes in the appearance of underlying content and changes in the mode of a user interface displaying the affordance, according to some aspects.

[0041] FIG. 8A to FIG. 8Fis a flowchart illustrating a method for changing the appearance of an affordance and the type of affordance appearance based on changes in the appearance of underlying content according to some embodiments. DETAILED DESCRIPTION

[0042] Affordances displayed according to conventional methods are usually visually distracting and may confuse the user interface. In addition, the appearance of affordances is usually fixed and does not adapt to changes in the content below or changes in the operating mode of the user interface below. The following embodiments disclose a method for displaying and changing the appearance of affordances based on changes in the content below, wherein the reversal of the display attributes of the content below provides a basis for determining the value of the same display attribute of affordances. In addition, the value range of the display attribute of affordances is constrained to a sub-range of the value range of the display attribute of the content below. For example, different value ranges of the brightness of affordances are assigned to different affordance appearance types (e.g., suggestive affordance appearance types and bright affordance appearance types) selected for backgrounds with different overall brightness levels (e.g., dark backgrounds and bright backgrounds). This improves the visibility of affordances relative to the changing background without causing inappropriate interference to the user. In some embodiments, depending on the operating mode of the user interface that displays affordances, the appearance of affordances changes with the appearance of the background according to different groups of rules, thereby providing a way to adjust the balance between the need to maintain the visibility of affordances and the need to reduce the inappropriate distraction caused by affordances when the operating scenario of affordances changes. In some embodiments, both the appearance of the affordance and the affordance appearance type change based on changes in the appearance of the underlying content (including instantaneous changes and cumulative changes over time).

[0043] the following Figure 1A to Figure 1B , Figure 2 as well as Figure 3 A description of an exemplary apparatus is provided. FIG. 4A to FIG. 4B , FIG. 5A to FIG. 5D , Figure 5G to Figure 5P , Figure 5S to Figure 5AA And 5AD shows an exemplary user interface having an affordance representation that changes its appearance based on changes in the appearance of underlying content according to some embodiments. Figure 5E , Figure 5Q , Figure 5R , Figure 5AB , Figure 5AC and 5AE to Figure 5AK The difference in appearance of affordances according to some embodiments and the methods for generating 5A to Figure 5D , Figure 5G to Figure 5P , Figure 5S to Figure 5AA as well as Figure 5AD The affordances shown in represent the lower range of appearance values. FIG. 6A to FIG. 6C , 7A to 7E as well as FIG. 8A to FIG. 8Fis a flowchart of a method for displaying and adjusting the appearance of an affordance representation according to some embodiments. FIG. 4A to FIG. 4B as well as 5A to 5AK The user interface, affordance appearance, and value ranges shown in are for illustration purposes only. FIG. 6A to FIG. 6C , 7A to 7E as well as FIG. 8A to FIG. 8F process.

[0044] Exemplary Devices

[0045] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, so as not to unnecessarily obscure various aspects of the embodiments.

[0046] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0047] The terms used in the description of various described embodiments herein are only for the purpose of describing specific embodiments, and are not intended to be limited. As used in the description of various described embodiments and in the appended claims, the singular forms "one" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the associated listed items. It will also be understood that the term "includes", "including", "comprises" and / or "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their grouping.

[0048] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0049] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple iPod from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0050] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0051] The device typically supports a variety of applications, such as one or more of the following applications: a note-taking application, a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital camcorder application, a web browsing application, a digital music player application, and / or a digital video player application.

[0052] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0053] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A 1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes simply referred to as a touch-sensitive display. The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0054] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, the movement of the touch-sensitive surface may be optionally interpreted or sensed by the user as the "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perception, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user. Providing tactile feedback to the user using tactile output enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0055] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0056] When a device (e.g., via one or more tactile output generators that move a movable mass to generate a tactile output) generates a tactile output with different tactile output patterns, the tactile output can produce different tactile sensations in a user holding or touching the device. Although the user's senses are based on the user's perception of the tactile output, most users will be able to recognize changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different operations have been performed. In this way, a tactile output with a tactile output pattern designed, selected, and / or arranged to simulate the characteristics of an object in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or a combination of any of the above) (e.g., size, material, weight, stiffness, smoothness, etc.); behavior (e.g., oscillation, displacement, acceleration, rotation, extension, etc.); and / or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device. In addition, tactile output is optionally generated to correspond to feedback that is independent of simulated physical properties (such as input threshold or object selection). Such tactile output will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device.

[0057] In some embodiments, the tactile output with a suitable tactile output mode serves as a prompt for an event of interest in the user interface or behind the screen in the device. Examples of events of interest include activation of affordances (such as real or virtual buttons or toggle switches) provided on the device or in the user interface, success or failure of the requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or identifying a type of input or gesture, etc. In some embodiments, a tactile output is provided to serve as a warning or prompt for an upcoming event or result that will occur unless a change in direction or interruption input is detected in time. In other cases, the tactile output is also used to enrich the user experience, improve the accessibility of the device to users with visual or motor difficulties or other accessibility requirements, and / or improve the efficiency and function of the user interface and / or device. Optionally, the tactile output is compared with audio input and / or visual user interface changes, which further enhances the user's experience when the user interacts with the user interface and / or device, and is conducive to better transmission of information about the state of the user interface and / or device, and this reduces input errors and improves the efficiency of the user's operation of the device.

[0058] It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0059] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory 102 by other components of the device 100 (such as one or more CPUs 120 and peripheral device interface 118) is optionally controlled by a memory controller 122.

[0060] Peripheral interface 118 may be used to couple the device's input and output peripherals to memory 102 and CPU(s) 120. Processor(s) 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0061] In some embodiments, peripherals interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0062] RF (radio frequency) circuit 108 receives and transmits RF signals, also referred to as electromagnetic signals. RF circuit 108 converts electrical signals into / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, user identity modules (SIM) cards, memory, and the like. RF circuit 108 optionally communicates with networks and other devices through wireless communications, such as the Internet (also referred to as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). The wireless communication optionally uses any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Profile (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution-Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11d), IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols that have not been developed on the date of filing of this document.

[0063] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single or dual-ear headset) and input (e.g., a microphone).

[0064] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., Figure 2 208) optionally includes an up / down button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206).

[0065] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives an electrical signal from the touch-sensitive display system 112 and / or sends an electrical signal to the touch-sensitive display system 112. The touch-sensitive display system 112 displays a visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual outputs correspond to user interface objects. Examples of user interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls. In some embodiments, some or all of the visual outputs correspond to indicators and visual guides, which provide visual cues of the types of input and / or operations associated with different areas of the user interface or screen. Examples of indicators and visual guides include, but are not limited to, arrows, bars, overlays, spotlights, or other visually distinct areas or shapes designed to provide visual cues to the user. As used herein, the term "affordance" refers to user interactive graphical user interface objects and / or indicators and visual guides displayed on a background (e.g., a portion of an application user interface or a system user interface).

[0066] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from the user based on tactile and / or haptic contact. The touch-sensitive display system 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) that is displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0067] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as the . iPod and Technology found in.

[0068] The touch-sensitive display system 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi or greater). The user optionally uses any suitable object or attachment such as a stylus, finger, etc. to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of ​​the finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer / cursor positions or commands for performing the actions desired by the user.

[0069] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touch screen.

[0070] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0071] Device 100 optionally also includes one or more optical sensors 164 . Figure 1AAn optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. One or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), one or more optical sensors 164 optionally capture static images and / or videos. In some embodiments, an optical sensor is located on the rear portion of the device 100 opposite to the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for self-portraits, for video conferencing when the user is watching other video conference participants on the touch screen, etc.).

[0072] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor coupled to a force sensor controller 159 in the I / O subsystem 106 is shown. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear of the device 100 opposite to the touch-sensitive display system 112 located on the front of the device 100.

[0073] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0074] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1AA tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106 is shown. In some embodiments, one or more tactile output generators 167 include one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components for converting electrical signals into tactile outputs on the device). One or more tactile output generators 167 receive tactile feedback generation instructions from the tactile feedback module 133 and generate tactile outputs on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in juxtaposition or proximity to a touch-sensitive surface (e.g., a touch-sensitive display system 112), and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on a rear portion of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0075] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on an analysis of data received from the one or more accelerometers. The device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0076] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, as shown in FIG. 1A and FIG. Figure 3. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information about the device's position and / or posture.

[0077] The operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0078] The communication module 128 facilitates communication with other devices through one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices or indirectly via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a device compatible with some of Apple Inc. (Cupertino, California). iPod A multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in iPod devices. In some embodiments, the external port is compatible with some of Apple Inc. (Cupertino, California) iPod A Lightning connector that is the same as, similar to, and / or compatible with the Lightning connector used in iPod devices.

[0079] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0080] The contact / motion module 130 optionally detects gesture input from a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0081] In some embodiments, detecting a finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of the strength of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between a finger press event and a finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the strength of the finger contact during the tap reaches a given strength threshold (greater than a nominal contact detection strength threshold), such as a light press or deep press strength threshold. Therefore, a finger tap gesture can meet a specific input criterion that does not require the characteristic strength of the contact to meet a given strength threshold to meet the specific input criterion. For clarity, the finger contact in a tap gesture is generally required to meet a nominal contact detection strength threshold to detect a finger press event, below which the contact is not detected. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In the case where the device is capable of detecting contact with a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0082] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture may be optionally detected based on the satisfaction of criteria that are independent of the intensity of the contacts included in the gesture or that do not require the contacts performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a zoom gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on a touch-sensitive surface having less than a threshold amount of movement. Thus, a statement that a particular gesture recognition criterion does not require the intensity of the contacts to meet a corresponding intensity threshold in order to meet the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied when the contacts in the gesture do not meet the corresponding intensity threshold, and can also be satisfied when one or more contacts in the gesture meet or exceed the corresponding intensity threshold. In some embodiments, a tap gesture is detected based on determining that a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding strength threshold during the predefined time period, and a swipe gesture is detected based on determining that the contact moves greater than a predefined amount, even if the contact is above the corresponding strength threshold at the end of the contact movement. Even in specific implementations where detection of gestures is affected by the strength of the contact performing the gesture (e.g., the device detects a long press more quickly when the strength of the contact is above the strength threshold, or the device delays detection of a tap input when the strength of the contact is higher), detection of these gestures does not require the contact to reach a particular strength threshold (e.g., even if the amount of time required to recognize the gesture varies), as long as the criteria for recognizing the gesture can be met without the contact reaching the particular strength threshold.

[0083] In some cases, the contact intensity threshold, duration threshold, and movement threshold are combined in various combinations to create a heuristic algorithm to distinguish between two or more different gestures for the same input element or area, so that multiple different interactions with the same input element can provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria does not require the intensity of the contact to meet the corresponding intensity threshold to meet the particular gesture recognition criteria does not exclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures with criteria that are met when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the intensity of the contact to meet the corresponding intensity threshold to meet the first gesture recognition criterion) competes with a second gesture recognition criterion for a second gesture (which depends on the contact reaching the corresponding intensity threshold). In such a competition, if the second gesture recognition criterion for the second gesture is met first, the gesture is optionally not recognized as meeting the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves a predefined amount of movement before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in this case, the first gesture recognition criterion for the first gesture still does not require the intensity of the contact to meet the corresponding intensity threshold to satisfy the first gesture recognition criterion because if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with an intensity of the contact that does not increase to above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture by the first gesture recognition criterion. Therefore, a particular gesture recognition criterion that does not require the intensity of the contact to meet the corresponding intensity threshold to satisfy the particular gesture recognition criterion will (A) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases fail to satisfy the particular gesture recognition criterion (e.g., for a long press gesture) if a set of competing intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-related gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input, and in this sense still depends on the intensity of the contact relative to the intensity threshold (e.g., for a long press gesture that competes with the deep press gesture for recognition).

[0084] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0085] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application program, etc., and also receives coordinate data and other graphic attribute data together if necessary, and then generates screen image data to be output to the display controller 156.

[0086] Haptic feedback module 133 includes various software components for generating instructions (e.g., instructions used by haptic feedback controller 161) to generate tactile output at one or more locations on device 100 using tactile output generator 167 in response to user interaction with device 100.

[0087] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0088] The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and maps / navigation widgets).

[0089] The application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0090] Contacts module 137 (sometimes called address book or contact list);

[0091] Telephone module 138;

[0092] Video conferencing module 139;

[0093] Email client module 140;

[0094] Instant messaging (IM) module 141;

[0095] Fitness support module 142;

[0096] A camera module 143 for still images and / or video images;

[0097] Image management module 144;

[0098] Browser module 147;

[0099] Calendar module 148;

[0100] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets acquired by the user, and a user-created widget 149-6;

[0101] A widget creator module 150 for forming a user-created widget 149-6;

[0102] Search module 151;

[0103] Video and music player module 152, optionally composed of a video player module and a music player module;

[0104] Notepad module 153;

[0105] Map module 154; and / or

[0106] Online video module 155.

[0107] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0108] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contacts module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding names to the address book; deleting one or more names from the address book; associating one or more telephone numbers, one or more email addresses, one or more physical addresses, or other information with a name; associating images with names; categorizing and classifying names; providing telephone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or instant messaging 141; and the like.

[0109] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a character sequence corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing a corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communications optionally use any of a variety of communication standards, protocols, and technologies.

[0110] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0111] In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0112] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs) or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs or IMPS).

[0113] In combination with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and video and music player module 152, fitness support module 142 includes executable instructions for creating fitness (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for fitness; and displaying, storing, and transmitting fitness data.

[0114] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0115] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, displaying (e.g., in a digital slide show or album), and storing still images and / or video images.

[0116] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0117] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0118] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5), or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

[0119] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page to a desktop widget).

[0120] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0121] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).

[0122] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0123] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with the maps (e.g., driving routes; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0124] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0125] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0126] In some embodiments, the device 100 is a device where the operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0127] A predefined set of functions that are performed exclusively through a touch screen and / or a touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, a home menu, or a root menu. In such embodiments, a touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0128] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0129] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event distributor module 174. In some embodiments, the application 136-1 includes an application internal state 192, which indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0130] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information ready to be displayed by the application 136-1, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0131] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information received by peripherals interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0132] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0133] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0134] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0135] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0136] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0137] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain actively participating views.

[0138] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0139] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In another embodiment, event classifier 170 is a standalone module, or is part of another module (such as contact / motion module 130) stored in memory 102.

[0140] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, the corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more event recognizers in event recognizers 180 are part of an independent module, such as a user interface toolkit (not shown) or a higher level object from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views in application view 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0141] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event from the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least one subset of the following two: metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0142] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (e.g., from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0143] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0144] In some embodiments, event definition 187 includes a definition of an event for a corresponding user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with a sub-event and an object that triggers the hit test.

[0145] In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays the delivery of the event information until it has been determined whether the sub-event sequence does or does not correspond to the event type of the event identifier.

[0146] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0147] In some embodiments, the corresponding event identifier 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event identifiers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event identifiers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0148] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag captures the flag and performs a predefined process.

[0149] In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with a sub-event sequence or to an actively participating view. An event handler associated with a sub-event sequence or with an actively participating view receives the event information and performs a predetermined process.

[0150] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video or music player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0151] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0152] It should be understood that the above discussion of event processing for user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events defining an event to be recognized.

[0153] Figure 2 A device having a touch screen (e.g., Figure 1AThe portable multifunction device 100 of the present invention is a touch-sensitive display system 112 of the present invention. The touch screen optionally displays one or more graphics within the user interface (UI) 200. In these embodiments and in other embodiments described below, the user can select one or more of the graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, when the user interrupts contact with the one or more graphics, selection of the one or more graphics will occur. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of the fingers that have been in contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with the graphics will not select the graphics. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0154] The device 100 optionally also includes one or more physical buttons, such as a "home" button, or a menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

[0155] In some embodiments, the device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for powering the device on / off and for locking the device, one or more volume adjustment buttons 208, a user identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button in a pressed state for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting the strength of contact on the touch-sensitive display system 112 and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0156] Figure 3300 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes a circuit system (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. The I / O interface 330 also optionally includes a keyboard and / or a mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on the device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located away from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100 ( Figure 1A ) stored in memory 102 of portable multifunction device 100, or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.

[0157] Figure 3Each element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device. Each module in the above-mentioned identified module corresponds to a group of instructions for performing the above-mentioned functions. The above-mentioned identified module or program (that is, instruction set) need not be implemented as a separate software program, process or module, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores the subgroup of the above-mentioned modules and data structure. In addition, memory 370 optionally stores other modules and data structure not described above.

[0158] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0159] Figure 4A An exemplary user interface of an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0160] One or more signal strength indicators for one or more wireless communications, such as cellular signals and Wi-Fi signals;

[0161] ·time;

[0162] Bluetooth indicator;

[0163] Battery status indicator;

[0164] A tray 408 with common application icons, such as:

[0165] o an icon 416 of the phone module 138 labeled “Phone” which optionally includes an indicator 414 of the number of missed calls or voice messages;

[0166] o an icon 418 of the email client module 140 labeled “Mail”, which optionally includes an indicator 410 of the number of unread emails;

[0167] o An icon 420 of the browser module 147 labeled “Browser”; and

[0168] o An icon 422 labeled "Music" of the video and music player module 152; and

[0169] Icons of other applications, such as:

[0170] o Icon 424 of IM module 141 labeled “Message”;

[0171] o Icon 426 labeled “Calendar” of calendar module 148;

[0172] o Icon 428 labeled “Photos” of the image management module 144;

[0173] o An icon 430 labeled “Camera” of the camera module 143;

[0174] o Icon 432 labeled “Online Video” of the online video module 155;

[0175] o Icon 434 labeled "Stock Market" of the Stock Market widget 149-2;

[0176] o an icon 436 labeled “MAP” of the map module 154;

[0177] o Icon 438 labeled "Weather" of the Weather widget 149-1;

[0178] o Icon 440 labeled as “Clock” of the alarm clock widget 149-4;

[0179] o an icon 442 labeled “Fitness Support” of the fitness support module 142;

[0180] o An icon 444 of the notepad module 153 labeled "Notepad"; and

[0181] o An icon 446 for a settings application or module that provides access to settings for the device 100 and its various applications 136.

[0182] It should be noted that Figure 4A The icon labels shown in are exemplary only. For example, other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0183] Figure 4B A touch-sensitive surface 451 (eg, Figure 3 A device (e.g., a tablet or touch pad 355) Figure 3 Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting intensity of contact on touch-sensitive surface 451 and / or one or more tactile output generators 359 for generating tactile output for a user of device 300.

[0184] Although many of the examples that follow will be given with reference to input on a touch screen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with touch-sensitive surface 451 at a location corresponding to a corresponding location on the display (e.g., Figure 4B 460 and 462) (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B 450 in ) is separated, the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0185] In addition, although the following examples are given primarily with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detecting contact, followed by ceasing to detect contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and finger contact are optionally used simultaneously.

[0186] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 3 Touchpad 355 or Figure 4BIn the event that an input (e.g., a press input) is detected on the touch-sensitive surface 451 in the display, the particular user interface element is adjusted according to the detected input. In the case that the touch-sensitive surface 451 in the display includes a touch-screen display (e.g., Figure 1A A touch-sensitive display system 112 or Figure 4A In some implementations of a touch screen in a touch screen display, a contact detected on the touch screen acts as a "focus selector" such that when an input (e.g., a press input by a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without a corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves in accordance with the movement of the focus between different areas of the user interface. Regardless of the specific form that the focus selector takes, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to convey a user-desired interaction with the user interface (e.g., by indicating to the device an element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user desires to activate the corresponding button (rather than other user interface elements shown on the device display).

[0187] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units). The intensity of the contact is used as an attribute of the user input, thereby allowing the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or a physical control / mechanical control such as a knob or button).

[0188] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some specific implementations, the user of the device is provided with a software setting for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0189] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted off, before or after contact begins to move, before contact ends, before or after contact is detected to increase in intensity, and / or before or after contact is detected to decrease in intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half-maximum value of the contact intensity, the 90% maximum value of the contact intensity, the value generated by low-pass filtering the contact intensity over a predefined time period or starting from a predefined time, etc. In some embodiments, the duration of the contact is used in determining the feature strength (e.g., when the feature strength is the average of the strength of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0190] In some embodiments, a portion of a gesture is identified for determining a characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an end position (e.g., a drag gesture) where the strength of the contact increases. In this example, the characteristic strength of the contact at the end position may be based only on a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the end position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the strength of the swipe contact to achieve the purpose of determining the characteristic strength.

[0191] The user interface diagrams described herein optionally include various intensity diagrams that illustrate the intensity of contacts on the touch-sensitive surface relative to one or more intensity thresholds (e.g., contact detection intensity threshold IT0, light press intensity threshold IT1, and light touch intensity threshold IT2). L , Deep compression intensity threshold IT D (For example, at least initially above IT L ) and / or one or more other intensity thresholds (e.g., L Low intensity threshold IT H )). This intensity graph is not typically part of the displayed user interface, but is provided to help explain the graph. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, when a contact is detected with a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT0, contacts lower than the nominal contact detection intensity threshold are no longer detected), the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing operations associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface illustrations.

[0192] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers a first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion including both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, as long as a delay time passes between satisfying the first strength threshold and satisfying the second strength threshold, the strength of the contact exceeding the second strength threshold greater than the first strength threshold of the light press during the input triggers a second response. The duration of the delay time is typically less than 200ms (milliseconds) (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second strength threshold, where the delay time increases as the second strength threshold increases). The delay time helps avoid accidentally identifying a deep press input. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold increases. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs.For other deep press inputs, the response to detecting the deep press input does not depend on time-based criteria.

[0193] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., situational noise), focus selector position, etc. Exemplary factors are described in U.S. patent application serial numbers 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0194] For example, in some embodiments, the dynamic intensity threshold varies over time based in part on the intensity of the touch input over time. The dynamic intensity threshold is the sum of two components: a first component that decays over time after a predefined delay time p1 from when the touch input is initially detected, and a second component that tracks the intensity of the touch input over time. The initial high intensity threshold of the first component reduces accidental triggering of a "deep press" response while still allowing an immediate "deep press" response if the touch input provides sufficient intensity. The second component reduces the unintentional triggering of a "deep press" response by gradual intensity fluctuations of the touch input. In some embodiments, the "deep press" response is triggered at the point in time when the touch input meets the dynamic intensity threshold.

[0195] In another example, in some embodiments, a dynamic intensity threshold (e.g., intensity threshold I D) and two other intensity thresholds (the first intensity threshold IT H and the second intensity threshold I L ) is used in combination. In some embodiments, although the touch input satisfies the first intensity threshold IT before time p2 H and the second intensity threshold IT L , but no response is provided until a delay time p2 has elapsed. In some embodiments, the dynamic intensity threshold decays over time, wherein the decay is from the second intensity threshold IT L The associated response is triggered at a time point (e.g., time p2) after a predefined delay time p1. This type of dynamic intensity threshold reduction immediately follows the trigger with a lower threshold intensity (such as the first intensity threshold IT H Or the second intensity threshold I L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold IT D The associated response.

[0196] In another example, in some embodiments, after a delay time p2 has passed since the touch input was initially detected, the trigger is triggered with the intensity threshold IT L At the same time, after a predefined delay time p1 has passed since the touch input was initially detected, the dynamic intensity threshold (eg, intensity threshold I D ) decays. Therefore, between the trigger and the intensity threshold I L The associated response then reduces the intensity of the touch input, and then increases the intensity of the touch input without releasing the touch input, which can trigger the intensity threshold IT D The associated response is not limited to the touch input even when the intensity of the touch input is below another intensity threshold (e.g., intensity threshold I L ) is also the case.

[0197] The contact characteristic strength is lower than the light press strength threshold IT L The intensity increases to between the light pressure intensity threshold IT L and deep compression intensity threshold IT D The intensity between IT and the deep press intensity threshold is sometimes referred to as a "light press" input. D The intensity of the deep compression increases to above the deep compression intensity threshold IT D The intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. LA decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting a contact lifted off the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some illustrations, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0198] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the strength of the contact (or multiple contacts) increases to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting that the strength of the corresponding contact increases to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and the strength of the contact subsequently decreases to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0199] In some embodiments, the device employs intensity hysteresis to avoid unexpected input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the contact strength increases from an intensity equal to or below a hysteresis strength threshold to an intensity equal to or above a press input strength threshold and, optionally, the contact strength subsequently decreases to an intensity equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in contact strength or a decrease in contact strength, depending on the scenario).

[0200] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input are optionally triggered in response to detecting that the intensity of the contact increases above the press input intensity threshold, the intensity of the contact increases from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, triggering of these operations is also dependent on satisfying a time-based criterion (e.g., a delay time has elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0201] User interface and associated processes

[0202] Attention now turns to embodiments of a user interface ("UI") and associated processes that may be implemented on an electronic device such as portable multifunction device 100 or device 300 having a display, a touch-sensitive surface, and (optionally) one or more sensors for detecting intensity of contacts with the touch-sensitive surface.

[0203] FIG. 5A to FIG. 5D An exemplary user interface with an affordance (e.g., a home affordance) according to some embodiments is shown, which affordance indicates a gesture start area on a touch-sensitive display screen for a navigation gesture (e.g., a gesture for navigating to a home screen user interface). In some embodiments, affordances for controlling or providing guidance about other functions or operations of the device are also displayed using the methods described herein. The user interfaces in these figures are used to illustrate the processes described below, including FIG. 6A to FIG. 6C , 7A to 7E as well as FIG. 8A to FIG. 8F For ease of explanation, some embodiments in the implementation will be discussed with reference to operations performed on a device with touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representative point corresponding to the finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on touch-sensitive display system 112.

[0204] For ease of explanation, some of the embodiments will be discussed with reference to operations performed on a device that does not have a home button, and uses a gesture that meets predefined criteria to dismiss the currently displayed user interface and display the home screen user interface. FIG. 5A to FIG. 5D Not shown, but in some embodiments, a home button (e.g., a mechanical button, a solid-state button, or a virtual button) is included on the device and is used to dismiss the currently displayed user interface and display the home screen user interface (e.g., in response to a single press input) and / or display a multitasking user interface (e.g., in response to a double press input).

[0205] exist FIG. 5A to FIG. 5D In the present invention, when the device displays any user interface, a gesture that begins at the bottom of the screen (e.g., within a predefined area of ​​the device that is adjacent to an edge of the display (e.g., an edge area including a predefined portion (e.g., 20 pixels wide) of the display near the bottom edge of the device)) invokes a user interface navigation process and optionally guides navigation between multiple user interfaces based on the speed and direction of the input and, optionally, based on movement parameters and characteristics of a currently displayed user interface object (e.g., a reduced scale representation of the currently displayed user interface).

[0206] An exemplary user interface for an application operating on an electronic device includes an affordance (e.g., home affordance 5002) that provides visual guidance to the user regarding: the location of an edge area where a navigation gesture may be initiated; and, optionally, whether navigation is limited to the current operating mode of the currently displayed application (e.g., the absence of a home affordance indicating that navigation is limited and that a confirming input or an enhanced navigation gesture (e.g., swipe up after a press input or swipe up after a touch and hold input) is required to navigate between user interfaces (e.g., as Figure 5X to Figure 5AA In some embodiments, the home enable representation is not activatable or does not respond directly to touch input in a manner similar to a virtual button. In some embodiments, the home enable representation or other enable representation implemented using the methods described herein responds to touch input that includes contact directly on the enable representation.

[0207] Figure 5A 5004 is shown displaying the content of web page 5004. Home affordance 5002 is displayed as a portion of the content displayed near the bottom edge of touch screen 112 overlaid. The user interface navigation process is activated by contact 5006, which begins at a location below, above, or near home affordance 5002 and travels upward from the bottom of the screen, as shown in FIG. FIG. 5A to FIG. 5B shown.

[0208] exist Figure 5B , the web browsing user interface is represented by Figure 5A The web browser user interface in card 5008 is replaced. FIG. 5A to FIG. 5C , as the input moves up the screen, card 5008 shrinks dynamically, showing a blurred view 5010 of the home screen in the background. In some embodiments, the amount of blur applied to the home screen changes dynamically based on the distance of contact 5006 from the bottom of the screen.

[0209] In some embodiments, such as Figure 5B As shown, when the user interface navigation process is activated by input made through contact 5006, home affordance 5002 ceases to be displayed (e.g., home affordance 5006 is not directly manipulable by touch input, and ceases to be displayed when it has fulfilled its purpose of providing visual guidance to the user regarding the navigation gesture). When the size of card 5008 is sufficiently small, other cards (e.g., cards 5012 and 5014, representing a system control panel user interface and a user interface of a recently opened application, respectively) are displayed next to card 5008. Optionally, terminating input (e.g., lifting off contact 5006) when multiple cards are displayed causes the device to enter a multitasking mode (e.g., displaying an application switcher user interface that allows the user to select an application to replace a web browser application as the foreground application).

[0210] exist Figure 5C In the example, when contact 5006 continues to move upward and satisfies a predefined home navigation criterion (e.g., a predefined feature (e.g., position, speed, etc.) of contact 5006 satisfies a predefined threshold), other cards on the screen stop being displayed, and only card 5008 remains on the screen. When the input is terminated (e.g., the lift-off of contact 5006 is detected), only card 5008 is displayed at the same time (e.g., similar to Figure 5C ), the device displays the home screen 5016, such as Figure 5D As shown (e.g., the dashed oval indicates the lift-off position of contact 5006). When home screen 6016 is displayed on touch screen 112, home affordance 5002 is not displayed on the touch screen.

[0211] Since it is necessary to dismiss the currently displayed user interface and go to the home screen in many scenarios (for example, when displaying a different application or other system-level user interface (for example, a notification center user interface, a cover user interface, a control panel user interface, etc.)), the home indicator 5002 needs to be displayed on all kinds of backgrounds, which can also change spontaneously or in response to user manipulation and instantaneously and over time. In addition, in order to give the user greater freedom in deciding where to start a navigation gesture along the bottom edge area of ​​the touch screen, the home indicator 5002 is designed to have a larger horizontal span to indicate the extensibility of the reaction area for gestures, and to have a relatively small width to avoid cluttering the screen excessively and causing unnecessary interference to the user. Therefore, even if the underlying content is static over time, a portion of the content below the home indicator 5002 may include changes in color and brightness and other display properties in different portions of the portion of the content. Thus, in some embodiments, the appearance of each sub-portion of the affordance (e.g., each pixel or each small cluster of pixels) is individually determined based on the appearance of the content directly beneath the affordance sub-portion (and, optionally, the appearance of content that extends slightly beyond the boundaries of the affordance sub-portion (e.g., by a blur or averaging effect applied to the content or affordance)). Figure 5A As shown, home affordance 5002 is displayed on a portion of content 5004 in a web page. The portion of the content includes areas with different brightness levels, and the resulting home affordance 5002 also includes brightness changes along its length (e.g., horizontal range).

[0212] In some embodiments, such as Figure 5E As shown, multiple image processing filters (e.g., in sequence or without limiting the order of the filters) are applied to background content beneath the affordance to determine the appearance of the affordance. For example, an original full-color image of the content is desaturated to obtain a brightness map of the content. The brightness of the content is inverted (e.g., according to a predefined inversion relationship between the brightness values ​​of the background and the brightness values ​​of the affordance (e.g., Fig. 5F , Figure 5R , Figure 5AE etc.) to obtain the brightness value of the affordance at each pixel of the affordance. The inverse relationship between the brightness of the affordance and the brightness of the underlying content is used as an example of a corresponding relationship between the values ​​of the selected display attributes of the affordance and the underlying content. Other types of display attributes, such as grayscale values ​​or variations of brightness, may also be used in various embodiments.

[0213] like Figure 5EAs shown, inversion creates a contrast in appearance between the affordance and the underlying content. When a portion of the underlying content is brighter (e.g., has a higher brightness value), the corresponding portion of the affordance is darker (e.g., has a lower brightness value). For example, inversion is performed on different portions of the desaturated background content having different brightness values ​​(e.g., portions surrounded by circles marked 1, 2, 3, and 4 in the desaturated strip) to obtain corresponding portions of the affordance having different brightness values ​​(e.g., portions surrounded by circles marked 1, 2, 3, and 4 in the inversion strip). Fig. 5F An exemplary inversion curve for generating affordance-representative brightness values ​​from corresponding background brightness values ​​is shown. Figure 5E The background marked by circles in and the values ​​of the corresponding parts of the affordances are marked by circles marked 1, 2, 3, and 4, as shown in Fig. 5F The shadow of the affordance is also shown in Fig. 5F , and are reproduced in the circles marked 1, 2, 3, and 4 in . In some embodiments, after the inversion is performed, a thresholding process is performed on the brightness values ​​to reduce the dynamic range of the brightness values. For example, the brightness value of each pixel of the enabling representation is limited to 50% of the maximum brightness of the enabling representation to produce a softer appearance with lower internal visual contrast (e.g., comparing the inverted enabling representation with the thresholded enabling representation). In some embodiments, in order to further reduce internal variations and contrast within the enabling representation, a blur filter is applied to average the brightness variations over multiple neighboring pixels in the content, and therefore the brightness variations over multiple neighboring pixels in the enabling representation. Finally, the resulting enabling representation has wider stroke brightness variations that correspond to the brightness variations in the underlying content.

[0214] When generating an affordance that stands out against a different background, while even a simple inversion of the brightness values ​​will produce sufficient contrast in most cases, using the affordance's full range of brightness values ​​will typically result in a harsher appearance, which may be distracting to the user. Therefore, it is advantageous to constrain the affordance's brightness value range to be a subrange of the content's brightness value range. Additionally, depending on the expected brightness level of the underlying content, the affordance's brightness range values ​​are constrained to either a "dark" affordance value range or a "bright" affordance value range, thereby producing a "dark" affordance or a "bright" affordance. In some embodiments, the affordance appearance type (e.g., "dark" vs. "bright") does not change after the affordance is initially displayed, even if the appearance of the underlying content changes from very dark to very light, or vice versa (e.g., Figure 5G to Figure 5PAs shown). In some embodiments, the affordance appearance type (e.g., "dark" vs. "light") does not change in response to a momentary change in content (e.g., temporarily reversing the content brightness level for a short period of time), but will eventually change in response to a more permanent change in content (e.g., maintaining the reversal of the content brightness level over a longer time scale). In some embodiments, the affordance appearance type (e.g., "light" or "dark" or a specific range of appearance values ​​for the affordance) is selected based on the initial brightness level of the underlying content when the affordance is first displayed, and the affordance maintains the affordance appearance type until a context switch event occurs (e.g., switching between applications, switching between an application or a system user interface, or switching between two system user interfaces, etc.), and the affordance appearance type is re-determined based on the underlying content in the new context. As an example, Figure 5R An exemplary inverse relationship for a "bright" enable representation and a "dark" enable representation is shown (e.g., the curves are labeled "LA" and "DA", respectively), where the content brightness value range (e.g., the values ​​along the horizontal axis) is the entire range from black to white (e.g., the grayscale values ​​of a grayscale image are [0, 1], or the brightness values ​​of a color image are [0, 255], or the brightness or other similar display attribute is [0, 100%]), and the enable representation brightness value range (e.g., the values ​​along the vertical axis) is constrained to be within an upper value range (e.g., the value range of the "bright" enable representation) or a lower value range (e.g., the value range of the "dark" enable representation). In some embodiments, the two value ranges do not overlap (e.g., separated by a gap). As Figure 5R As shown, two curves (eg, curve LA and curve DA) demonstrate that an increase in the brightness of the content results in a decrease in the brightness of the affordance.

[0215] Fig. 5F A method for performing Figure 5E 5017. In this example, the affordance value range constrained between the upper and lower limits is greater than half of the background value range and does not include pure black (value = 0) and pure white (e.g., value = 1). In some embodiments, the inversion curve is continuous and does not include continuous points.

[0216] Figure 5G to Figure 5K A change in the appearance of an enable representation (e.g., enable representation 5002-DA) of a first enable representation appearance type (e.g., a "dark" enable representation type) is shown according to some embodiments. Figure 5G to Figure 5KScrolling of content 5018 shown in a web browser user interface displayed on touch screen 112 is shown. As content 5018 scrolls, a portion of the content located below affordance 5002-DA displayed near the bottom edge of the touch screen changes. In other words, during scrolling of content 5018, different portions of content 5018 are moved to affordance 5002-DA below.

[0217] like Figure 5G As shown, in some embodiments, when the enable representation 5002-DA is initially displayed (e.g., when a web browser application is opened and a web browser interface is initially displayed along with content 5018), the overall brightness level of that portion of content 5018 is evaluated, and an appropriate enable representation appearance type is selected for the enable representation. In this particular example, the portion of content 5018 located below the enable representation is relatively dark (e.g., the overall brightness level is below a predefined brightness threshold), and the enable representation type corresponding to the "darker" portion of the enable representation brightness value range (e.g., a range of [0, 0.4]) is selected as the enable representation type for the enable representation. For example, using Figure 5E The set of filters shown generates affordances, and the inverse relations used are those that imply affordance appearance types (e.g., Figure 5R The curve DA shown in FIG.

[0218] exist Figure 5H In the example, a user interface navigation gesture (e.g., a home / multitasking gesture performed by contact 5006, such as FIG. 5A to FIG. 5D Scroll input (e.g., by contact 5020 moving upward on the touch screen) that begins outside (e.g., above) the responsive area of ​​5002-DA. The scroll input causes content 5018 of the web page to scroll upward and portions of content 5018 that were not previously displayed to arrive below affordance 5002-DA. Figure 5H At the moment depicted in , the portion of content 5018 directly below affordance 5002-DA is completely white (e.g., has a brightness value of 1 or 100%), and correspondingly, the brightness value of affordance 5002-DA is completely black (e.g., has a brightness value of 0 or 0%), as shown based on the inverse relationship of the affordance appearance type (e.g., as shown in Figure 5R The value of the curve DA in FIG.

[0219] FIG. 5I to FIG. 5KContent 5018 is shown scrolling upward beneath affordance 5002-DA as the scrolling input continues (e.g., contact 5020 continues to move upward and then contact 5020 lifts off at a terminal velocity). The appearance of affordance 5002-DA changes based on the portion of content 5018 currently located beneath affordance 5002-DA. The brightness of affordance 5002-DA is based on an inverse relationship that suggests the type of affordance appearance (e.g., as shown in FIG. 1 ). Figure 5R The value of the curve DA in FIG.

[0220] Specifically, in Figure 5J At the moment depicted in , the portion of content 5018 directly below affordance 5002-DA is completely white on the left side (e.g., a brightness value of 1 or 100%) and completely black on the right side (e.g., a brightness value of 0 or 0%), and correspondingly, half of affordance 5002-DA on the left side is completely black (e.g., a brightness value of 0 or 0%). Figure 5H ), and the right half of affordance 5002-DA is not completely white. Instead, the right half of affordance 5002-DA is gray (e.g., the affordance brightness value is greater than 0 (e.g., 0.4)), as shown based on the inverse relationship of the affordance appearance type (e.g., Figure 5R In other words, the range of brightness values ​​that can be represented (eg, 5002-DA) is constrained to be below the maximum brightness threshold (eg, 0.4).

[0221] Figure 5L to Figure 5P A change in the appearance of an enable indication of a second enable indication appearance type (e.g., a "bright" enable indication type) is shown according to some embodiments.

[0222] Figure 5L to Figure 5P Scrolling of content 5018 shown in a web browser user interface displayed on touch screen 112 is shown. Scrolling is Figure 5G to Figure 5K Reversal of the scroll depicted in .

[0223] like Figure 5L As shown, in some embodiments, when the enable indication 5002-LA is initially displayed (e.g., when a web browser application is opened and a web browser interface is initially displayed along with content 5018), the overall brightness level of that portion of content 5018 is evaluated, and an appropriate enable indication appearance type is selected for the enable indication. In this particular example, the portion of content 5018 located below the enable indication is relatively bright (e.g., the overall brightness level is above a predefined brightness threshold), and the enable indication type corresponding to the "brighter" portion of the enable indication brightness value range (e.g., a range of [0.6, 1]) is selected as the enable indication type for the enable indication. For example, using Figure 5E The set of filters shown generates the affordances, and the inverse relations used are the inverse relations of the light affordances (e.g., Figure 5R The curve LA shown in FIG.

[0224] exist Figure 5L to Figure 5M In the example, a user interface navigation gesture (e.g., a home / multitasking gesture performed by contact 5006, such as FIG. 5A to FIG. 5D 5002-LA).

[0225] exist Figure 5M At the moment depicted in , the portion of content 5018 directly below affordance 5002-LA is completely white on the left (e.g., a brightness value of 1 or 100%) and completely black on the right (e.g., a brightness value of 0 or 0%), and accordingly, half of affordance 5002-LA on the right is completely white, and half of affordance 5002-LA on the left is not completely black. In contrast, the left half of affordance 5002-LA is gray (e.g., a brightness value greater than 0 (e.g., 0.6)), as shown in the inverse relationship based on the bright affordance appearance type (e.g., as shown in Figure 5R In other words, the range of brightness values ​​of the affordances of the bright affordances is constrained to be above a minimum brightness threshold (e.g., 0.6). Figure 5J and Figure 5M As depicted, for the same background, the overall appearance of the suggestive energy representation 5002-DA is darker than the overall appearance of the optical energy representation 5002-LA.

[0226] Figure 5N to Figure 5P Content 5018 is shown scrolling downward under affordance 5002-LA as the scrolling input continues (e.g., contact 5022 continues to move downward and then contact 5022 is lifted off at a terminal velocity). The appearance of affordance 5002-LA changes based on the portion of content 5018 currently located below affordance 5002-DA. The brightness of affordance 5002-LA is based on an inverse relationship of the bright affordance appearance type (e.g., as shown in FIG. 1 ). Figure 5R The value of the above formula is determined by the curve LA in FIG.

[0227] exist Fig.5OAt the moment depicted in , the portion of content 5018 directly below affordance 5002-LA is completely white (e.g., a brightness value of 1 or 100%), and accordingly, the brightness value of affordance 5002-LA is not completely black. Instead, affordance 5002-LA is gray (e.g., a brightness value greater than 0 (e.g., 0.6)), as shown in the inverse relationship based on the appearance type of the bright affordance (e.g., as shown in Figure 5R In other words, the range of brightness values ​​that can be represented by the highlight is constrained to be above the minimum brightness threshold (eg, 0.6).

[0228] Figure 5Q The difference in appearance of two types of affordance appearance types (e.g., LA and DA) of affordance 5002 is shown when the same change occurs in the background (e.g., content 5018) according to some embodiments.

[0229] Figure 5Q Listed Figure 5G to Figure 5P The affordances for each state shown in 5002 are shown in FIG. The states are divided into five groups, each group corresponding to a corresponding state of the content 5018 shown in the web browser user interface. For example, from top to bottom, the five groups correspond to: (i) Figure 5G and Figure 5P (ii) Figure 5H and Fig.5O (iii) Fig.5I and Figure 5N (iv) Figure 5J and Figure 5M and (v) Figure 5K and Figure 5L .

[0230] like Figure 5Q As shown, for each group corresponding to the corresponding content state, the implied enabling representation appearance type of the enabling representation (e.g., comparing the DA version and LA version of the enabling representation 5002 under the same content strip) has an overall darker appearance (e.g., lower overall brightness) than the bright enabling representation appearance type.

[0231] Figure 5R Shown are the value ranges and inverse relationships of display properties of an affordance (e.g., brightness or grayscale value) and underlying content that implies an affordance appearance type and highlights an affordance appearance type according to some embodiments.

[0232] Figure 5Q The differences in appearance shown in are also reflected in Figure 5R , wherein the brightness value range of the indication that the bright indication can be represented is completely higher than the brightness value range of the indication that the implied indication can be represented, and the two value ranges are optionally non-overlapping.

[0233] Figure 5S to Figure 5AA A user interface according to some embodiments is shown, which includes an enabling representation having an appearance that changes in response to a background and an operating mode that changes associated with the background user interface.

[0234] exist Figure 5S , a web browser application is launched and web browser user interface 5024 is displayed on the touch screen. In this example, the web browser user interface is displayed in a landscape orientation according to the orientation of device 100. Home affordance 5002 is displayed near the bottom edge of the touch screen in a first state (e.g., fully visible state / high contrast state 5002-A). The affordance appearance type of affordance 5002 is optionally selected based on an initial overall brightness level of a portion of the web page content located below affordance 5002.

[0235] FIG. 5S to FIG. 5T Shown is a selection of a media item (e.g., a movie clip "Live Bright") for playback (e.g., in Figure 5S In response to selecting the media item, a media player application is launched and a user interface (e.g., user interface 5028) of the media player application is displayed on the touch screen. Figure 5T In the embodiment of the present invention, the media player application is operated with a first mode (for example, full screen mode with displayed controls, or interactive mode).When just starting media playback, user interface 5028 includes a plurality of control areas covering the media playback area (for example, the media content occupying the entire screen basically), and these control areas include various controls, such as a media scrubber, a "done" button, a volume slider control, a rewind control, a pause / play control, and a fast forward control for closing the media player application and returning to the web browser application user interface 5024. These controls are initially displayed on the media playback area, because the user will likely want to adjust the default starting position or the volume selected by the device after seeing how the media content looks or sounds for the first time, or return to a previous application.In some embodiments, when switching to media player user interface 5028 from web browser user interface 5024, the scene switching event is recorded by the device, and the affordance representation appearance type of affordance representation 5002 is re-determined based on the initial overall brightness level of a part of the media content below affordance representation 5002 when first starting media playback. Regardless of whether the enable representation appearance type is re-determined, the enable representation 5002 is initially displayed on the user interface 5028 in a fully visible state (e.g., a high contrast state).

[0236] Figure 5UIndicates that for a first predetermined amount of time (e.g., 10 seconds) after media playback is initiated, the control region remains visible over the media content and affordance 5002 remains in a fully visible state (e.g., full contrast state) on user interface 5028. During this time, the appearance of affordance 5002 is determined according to a first set of rules. In some embodiments, the first set of rules includes a set of filters (e.g., Figure 5E ) and inverse relationships (such as Figure 5R ), wherein the first set of filter parameters and / or the shape of one or more inversion curves are optionally preselected for the first state of the affordance. Figure 5S to Figure 5U , affordance 5002-A is a gray affordance with no color information, even though the underlying content is in full color.

[0237] exist Figure 5V In the embodiment, a first predetermined amount of time (e.g., 10 seconds) after media playback is initiated has expired. In response to the first predetermined amount of time expiring, the control area ceases to be displayed over the media content. The fading of the control area may occur instantaneously or through a short animation. The expiration of the first predetermined amount of time and / or the cessation of the control area being displayed on the media player user interface indicates that the media player user interface is now operating in a second mode (e.g., a full screen display mode without displayed controls, or a media consumption mode). Additionally, in response to the expiration of the first predetermined amount of time, the device determines the appearance of the affordance 5002 according to a second set of rules that are different from the first set of rules. In other words, the affordance is displayed in a second state (semi-visible state / low contrast state 5002-A'). In some embodiments, the second set of rules includes a set of filters (such as, Figure 5E ) and inverse relationships (such as Figure 5R), where the second set of parameters of the filter and / or the shape of one or more inversion curves are optionally pre-selected for a second state of the enabling representation 5002 (e.g., semi-visible state / low contrast state 5002-A'). In some embodiments, in the low contrast state, the enabling representation 5002 retains some of the color of the underlying content. For example, instead of fully desaturating the underlying content to obtain a luminance map of the content, only 70% of the color value (e.g., RGB value) of each pixel is desaturated, and 30% of the color information of each pixel in the underlying content is retained in the final appearance of the enabling representation. In some embodiments, the transparency level of the enabling representation is adjusted so that the enabling representation is not completely opaque and some color information of the underlying content is transmitted to the pixels of the enabling representation. In some embodiments, after performing the luminance inversion, the remaining color saturation of the enabling representation is increased (e.g., by 30%) to make the enabling representation appear slightly more vibrant, thereby better blending with the background. The reduced visibility or contrast of the second state of affordance 5002 is in consideration of the reduced likelihood that a user will want to interact with any controls after an initial period of time after first initiating media playback. In some embodiments, the transition from the first state to the second state is optionally a gradual and continuous transition over a plurality of intermediate states between the first state and the second state, as opposed to an abrupt and discrete transition. A gradual transition is less likely to distract a user from viewing the media content.

[0238] like Figure 5V to Figure 5W As shown, when the enabling representation 5002 is in a second state (eg, semi-visible state / low contrast state 5002-A'), the appearance of the enabling representation 5002 changes based on a second set of rules in accordance with changes in the content located beneath the enabling representation.

[0239] exist Figure 5X In some embodiments, after the affordance 5002 is in the second state for a second predetermined amount of time (e.g., 5 seconds), the affordance 5002 transitions from the second state to an invisible state (e.g., referred to as the third state), or in other words, the affordance completely fades out and stops being displayed on the media content. In some embodiments, the transition from the second state to the third state is optionally a gradual and continuous transition over a plurality of intermediate states between the second state and the third state, as opposed to abrupt and discrete transitions. A gradual transition is less likely to distract the user from viewing the media content.

[0240] Figure 5Y to Figure 5AAIt is shown that after affordance 5002 is no longer displayed over media content 5028, media playback continues until input is detected (e.g., movement of the device, a tap input or a swipe input on a touch screen through contact, contact near a bottom edge area of ​​the display, etc.). In response to detecting the input, affordance 5002 is redisplayed over media content 5028.

[0241] like Figure 5Y As shown, in response to movement of device 100 (or other types of input, such as a tap, swipe, or downward touch near the bottom edge area of ​​the display), the enabling representation 5002 is redisplayed on the media content 5028 in a second state (e.g., semi-visible / low contrast state 5002-A') (e.g., together with other control areas). In some embodiments, the control area is not redisplayed in response to the input, and the device continues to operate in the full-screen display mode (without displayed controls) of the media player application. In some embodiments, if another input (e.g., a tap input or a swipe input) is not detected within a threshold amount of time, the enabling representation is stopped from being displayed again. If the required input is detected within a threshold amount of time, the enabling representation is displayed again in the first state, and the media control area is also optionally redisplayed. The user interface returns to the first operating mode of the media player application (e.g., full-screen playback mode with displayed controls, or interactive mode). As opposed to displaying the enable indication 5002 in the first state and immediately returning to the first operating mode, initially providing the enable indication 5002 in the second state in response to the first input provides the user with some indication of the location of the home enable indication and the state of the user interface, but also takes into account that the input may be unintentional and the user does not actually want to be distracted from watching the media content. If the user's intention is to use the control and / or the enable indication 5002, a confirmation input from the user is required (e.g., a continued touch by the same contact for a threshold amount of time, a press input by the same contact with a threshold press intensity, or a second tap input by another contact).

[0242] Figure 5Z and Figure 5AA It is shown that in some embodiments, when the affordance 5002 is not displayed (e.g., as Figure 5X ) or when the affordance 5002 is displayed in a second state (e.g., semi-transparent / low contrast state 5002-A′) (e.g., as shown in Figure 5YIn some embodiments, when the device is in a state of being displayed, input by contact (e.g., contact 5030) is detected on the touch screen. In response to input by contact 5030, the enable indication 5002 is displayed (if not already displayed) in a second state (e.g., semi-visible / low contrast state 5002-A'). In addition, the media control area is also redisplayed on the media player user interface. In other words, the media player application returns to the first operating mode. In some embodiments, continuous touch input near the bottom edge area of ​​the display causes the device to first redisplay the enable indication in the second state (e.g., when contact is detected), and then redisplay the enable indication in the first state (e.g., when the contact is maintained for more than a threshold amount of time with less than a threshold amount of movement since the downward touch). In some embodiments, after redisplaying the enable indication in the first state, when movement of the contact is detected (without lift-off of the contact), the device recognizes a navigation gesture. In some embodiments, instead of requiring the contact to be kept substantially still for a threshold amount of time in order to redisplay the enable indication in the first state, the device requires the strength of the contact to exceed a predefined light press strength threshold with less than a threshold amount of movement of the contact. In some embodiments, after redisplaying the enabling representation in the first state, the device recognizes a navigation gesture when movement of the contact is detected (without lift-off of the contact).

[0243] exist Figure 5AA In some embodiments, in response to the media player application returning to the first operating state (e.g., the interactive state), the enable indication 5002 also transitions from the second state (e.g., the semi-visible state / low contrast state 5002-A') back to the first state (e.g., the fully visible state / high contrast state 5002-A). In some embodiments, the transition from the second state to the first state is optionally a discrete transition, as opposed to multiple intermediate states between the first state and the second state. An abrupt transition is more likely to alert the user that the operating mode of the user interface has changed, and shortens the waiting time for the user to access controls and provide subsequent input. In some embodiments, when a confirmation input is detected (e.g., a second tap after the enable indication is displayed in the second state in response to a first tap, or a continuous touch by the same contact that triggered the display of the enable indication in the second state), the enable indication 5002 is directly changed from the second state to the first state. Figure 5Y The state shown in Figure 5AA The status shown in Figure 5Z The status shown in .

[0244] Figures 5AB to 5AC The differences in the appearance of the enabling representation of the background in different operating modes according to some embodiments are shown.

[0245] exist Figure 5AB, the first state 5002-A of the affordance is opaque and is used when the media player user interface operates in a first operating state (e.g., an interactive state) and when media controls are displayed on media content. The second state 5002-A' is a semi-transparent state and is used when the media player user interface operates in a second operating state (e.g., a protected state or a media consumption state) and media controls are not displayed on media content. The third state is a state in which the affordance is no longer displayed and the user interface continues to operate in the second operating state (e.g., a protected state or a media consumption state). The affordance 5002 passes through these states when no input is received within a predetermined amount of time after the media player is started (e.g., in full screen or landscape mode).

[0246] exist Figure 5AC In the first state 5002-A, the enabling representation is started in an invisible state or is not displayed on the media content (for example, after the enabling representation 5002 stops being displayed due to the absence of user input). In response to a first input or a first portion of the input, the enabling representation is displayed in a second state, while the media player user interface is maintained in a second operating state (for example, a protected state or a media consumption state). Then, in response to a second input or a second portion of the input, the enabling representation is displayed in a first state 5002-A, and the user interface returns to the first operating state (for example, an interactive state). In the first state 5002-A, the enabling representation is opaque. In the second state 5002-A', the enabling representation is translucent.

[0247] In some embodiments, the affordance appearance type of the affordance is fixed and does not change when the underlying content changes. This provides a consistent affordance appearance that reduces distraction to the user. However, in some scenarios, when the content of the affordance display changes greatly, the fixed affordance type may not provide sufficient contrast on the underlying background after the content has changed from an overall dark tone to an overall light tone, or vice versa. In addition, sometimes, the switching of the content brightness level is a short-term switch (e.g., scrolling through a line of black text on a white background), and in this case, switching the affordance type in response to such a short-term change may be inefficient, confusing, and distracting. On the other hand, if the switching of the content brightness level is a more permanent or long-term switch (e.g., flipping from one page (e.g., a page showing the warm tones of the evening sky) to another page (e.g., showing a starry night scene)), keeping the affordance appearance type fixed may result in insufficient visibility of the affordance for an extended period of time.

[0248] In order to address the above-mentioned problems, while still balancing the need to maintain visibility without causing undue interference to the user, in some embodiments, the device allows the affordance to switch its affordance appearance type, and accordingly, when a predefined range switching criterion is met, the affordance appearance value range of the value is transferred from one value range to another value range. In some embodiments, when the measure of the overall brightness state of the content located below the affordance (and optionally located around it) (e.g., the cumulative value and aggregate value of the brightness value) exceeds a predefined threshold due to changes in the appearance of the content over time, the range switching criterion is met. In some embodiments, the measure of the overall brightness state of the content takes into account the brightness level of the relevant part of the content over a period of time (e.g., using a weighted running average), and also tends to keep the current affordance appearance type of the affordance unchanged (e.g., the bias is optionally achieved by giving a higher weight to the older brightness level of the content and a smaller weight to the newer brightness level of the content). Due to the cumulative effect and the bias towards the current affordance appearance type, the short-term goal of keeping the affordance appearance stable relative to temporary changes in the content, and continuing to provide sufficient affordance prominence on the changing content is met.

[0249] Figure 5AD A user interface according to some embodiments is shown, which includes an enable representation that dynamically switches between enable representation appearance types based on changes in underlying content over time.

[0250] Figure 5A A simple example is shown in which affordance 5002 begins as an implied affordance (e.g., initially in a state such as Figure 5G Then, the content below affordance 5002 changes to a relatively bright state (e.g., changes to a state such as shown in FIG. 1 ) by scrolling, for example. Figure 5K ). In this example, the enable representation appearance type of enable representation 5002 does not change during scrolling of the content, for example because the scrolling is relatively fast and the preference to maintain the currently selected enable representation appearance type has not been overcome by the change in appearance of the content over a short period of time. After scrolling of the content stops, the enable representation remains overlaid on the content shown in the bright state. Over time, the overall brightness state of the content gradually changes, and the brightness level of the current background gradually takes over and dominates the brightness level of the previously displayed background (for example, when the enable representation was initially displayed or during scrolling of the content). Finally, at time t1, the measure of the overall brightness state of the background exceeds a predefined threshold and the range switch trigger criteria is met. In some embodiments, in response to detecting that the range switch trigger criteria are met, the device immediately switches the enable representation appearance type and displays the enable representation according to the newly selected enable representation appearance type, for example as shown in Figure 5ADThe user interface on the right is shown. Figure 5L , but in this example, the user does not have to close the web browser application and restart it in order for the enable indication to be displayed as a highlighted enable indication over the content 5018.

[0251] In some embodiments, when the range switching trigger criteria are met, the device begins to gradually transition from a first enable indication appearance type (e.g., a hint enable indication appearance type) to a second enable indication appearance type (e.g., a bright enable indication appearance type). For example, during a predetermined transition period (e.g., T=t2-t1, 5 seconds), the enable indication appearance value range passes through one or more intermediate ranges between the value ranges of the first appearance type and the second appearance type. At any time during the transition period, the appearance of the enable indication is determined based on the specific intermediate value range currently being used as the enable indication appearance value range. Figure 5AD As shown in the middle user interface in , when the background content is the same, the affordance representation has an intermediate brightness level, which is between the suggestive affordance representation and the bright affordance representation.

[0252] In some embodiments, during the transition period, a measure of the overall brightness state of the underlying content continues to be updated over time, with the brightness level of the content at a more recent time taking over the brightness level of the content at an earlier time. If the range switch trigger criteria are met again (e.g., with the same threshold used for the earlier switch or a different threshold depending on the currently selected enable representation appearance type), the switch to the second enable representation appearance type is not fully completed, and the enable representation returns to the first enable representation appearance type. In this particular example, the content does not change, and the range switch trigger criteria will no longer be met during the transition period, and as a result, the switch to the second enable representation appearance type is fully completed at time t2 (e.g., the period between t1 and t2 is a predefined transition period). After the switch to the second enable representation appearance type is completed, the measure of the overall brightness state of the underlying content continues to be updated over time, and when the range switch trigger criteria are met again due to cumulative changes in the underlying content (e.g., due to switching scenarios, scrolling, etc.), the switch may occur again back to the first enable representation appearance type.

[0253] Figure 5AE The value range and inverse relationship of the display attribute (e.g., brightness) of the affordance according to some embodiments and the underlying content of the bright affordance appearance type (A), the hint affordance appearance type (C), and the transition affordance appearance type (B) are shown. In some embodiments, Figure 5AE The reversal curve shown in is optionally used to generate Figure 5AD The indication in represents the appearance of 5002.

[0254] In the example shown in 5AE, the shapes of the inversion curves of different appearance affordance representation types (A), (B), (C) are the same. Using inversion curves with the same shape allows the correspondence value between the background brightness value and the affordance representation brightness value for each point in the graph to be calculated and stored in a data table, so that when the transition between affordance representation types continuously passes through many intermediate value ranges, the brightness of each pixel on the affordance representation can be simply determined by looking up in the table based at least in part on the brightness of the corresponding pixel in the background. For example, during a transition period, each of a plurality of evenly spaced time points is associated with a corresponding intermediate value range between a bright affordance representation value range and an implicit affordance representation value range, and even if the content changes continuously during the transition period, the affordance representation appearance can still be quickly determined at each time point based on the inversion curve of the corresponding intermediate value range at that time point.

[0255] like Figure 5AE As shown in , the inversion curve 5032 includes two discontinuities. The left discontinuity point 5034 is introduced to resolve the interference point (e.g., at 25.4% background brightness), which is an interception of the isoluminance line 5036 (e.g., the affordance indication brightness = background brightness) and the inversion curve without the discontinuity point. The left discontinuity point allows the affordance indication to have a brightness value that is not completely the same as the background brightness, thereby avoiding the possibility of an "invisible" affordance indication appearing in certain special cases. Similarly, the right discontinuity point 5038 is introduced to resolve the interference point (e.g., at 74.51% brightness), which is an interception of the isoluminance line 5036 and the inversion curve without the discontinuity point. The right discontinuity point allows the affordance indication to have a brightness value that is not completely the same as the background brightness, thereby avoiding the possibility of an "invisible" affordance indication appearing in certain special cases.

[0256] In addition, near discontinuities 5036 and 5038, special corrections are made to the affordance indication brightness so that a strict inversion is not always observed (e.g., increasing background brightness corresponds to decreasing affordance indication brightness, and vice versa). For example, in the correction zone for left discontinuity 5036, the affordance indication brightness on the side with higher background brightness is relatively flat, while the affordance indication brightness on the side with lower background brightness includes a smaller inversion of the normal inversion relationship. In the correction zone for right discontinuity 5038, the affordance indication brightness on the side with lower background brightness includes a relatively larger inversion of the normal inversion relationship. The precise shape of these correction zones is designed to take into account the responsiveness of human vision to brightness values ​​within these zones, as well as the need to create sufficient contrast between the affordance indication and the background near interference points. For example, the width of the interference zone and the adjustment of the normal inversion relationship depend on the amount of contrast required for the affordance indication on the background at these interference points.

[0257] Figure 5AEEach of the figures (A), (B) and (C) shown in also includes a background brightness bar and an affordance response bar (displayed below the figure area). The background brightness bar and affordance response bar of each figure show the corresponding background brightness and affordance brightness generated according to the inversion curve 5032 in the same figure. In addition, the background color of the figure (A) of the implied affordance is white, so that all shades of the implied affordance can be shown (for example, including completely black affordances, but not including completely white affordances). The background color of the figure (C) of the bright affordance is black, so that all shades of the bright affordance can be shown (for example, including completely white affordances, but not including completely black affordances). The background color of the figure (B) of the transition affordance is gray (50% brightness). The affordance brightness with a background brightness of 50% is a slightly lighter gray (for example, a brightness value below 50%), as shown by comparing the affordance response halfway along the horizontal axis with the 50% gray bar displayed at the bottom of the figure (B).

[0258] Figure 5AF A gradual transition from a suggestive representation appearance type (e.g., represented in Figure (A)) to a bright representation appearance type (e.g., represented in Figure (C)) through multiple transition representation appearance types (e.g., represented in Figures (B-1), (B-2), and (B-3)) is shown according to some embodiments.

[0259] In some embodiments, when the range switching trigger criteria are met at time t1, the device begins to gradually transition from a first enable indication appearance type (e.g., an implied enable indication appearance type) to a second enable indication appearance type (e.g., a bright enable indication appearance type). Then, during a predetermined transition period (e.g., T=t2-t1), the enable indication appearance value range is transferred between the value ranges of the first appearance type and the second appearance type (e.g., up or down depending on the direction of the switch) through multiple intermediate ranges (e.g., the total number of intermediate ranges depends on the refresh rate of the display and the value gap between the upper limit of the enable indication brightness of the implied enable indication and 1). At any time during the transition period T, the appearance of the enable indication is determined based on a specific intermediate value range, which is currently used as the enable indication appearance value range.

[0260] In some embodiments, during the transition period, a measure of the overall brightness state of the underlying content continues to be updated over time, with the brightness level of the content at a more recent time taking over the brightness level of the content at an earlier time. If the range switch trigger criteria are met again (e.g., with the same threshold used for the earlier switch or a different threshold depending on the currently selected enable representation appearance type), the switch to the second enable representation appearance type is not fully completed, and the transfer of the enable representation appearance value range is reversed in direction and the enable representation may eventually return to the first enable representation appearance type. If the range switch trigger criteria are not met a second time during the transition period, the switch to the second enable representation appearance type is completed at the end of the transition period. After the switch to the second enable representation appearance type is completed, the measure of the overall brightness state of the underlying content continues to be updated over time, and when the range switch trigger criteria are met again due to cumulative changes in the underlying content (e.g., due to switching scenarios, scrolling, etc.), the switch may occur again back to the first enable representation appearance type.

[0261] In the example shown in 5AF, the shapes of the inversion curves for different appearance affordance types (A), (B-1), (B-2), (B-3), and (C) are the same. Using inversion curves with the same shape allows the corresponding relationship values ​​between the background brightness value and the affordance brightness value for each point in the graph (or the inversion curve for each intermediate value range) to be calculated and stored in a data table, so that when the transition between affordance types passes through many intermediate value ranges continuously, the brightness of each pixel on the affordance can be simply determined by a quick lookup in the data table based at least in part on the brightness of the corresponding pixel in the background. For example, during a transition period, each of a plurality of evenly spaced time points is associated with a corresponding intermediate value range between a bright affordance value range and an implied affordance value range, and even if the content changes continuously during the transition period and the underlying content has large brightness changes under different parts of the affordance, the affordance appearance can still be quickly determined at each time point based on the data pre-stored in the data table.

[0262] Figures 5AG to 5AK is the display attribute of the affordance Figure 5AF The different affordances shown in are enlarged copies of the inverse relationships of the content below the type of representation.

[0263] FIG. 6A to FIG. 6C 6 is a flowchart illustrating a method 6000 for changing the appearance of an affordance based on a change in the appearance of underlying content according to some embodiments. The method 6000 is performed on an electronic device (e.g., device 300, Figure 3 ; or a portable multifunction device 100, Figure 1A) is performed on a touch-sensitive surface. In some embodiments, the electronic device includes one or more sensors for detecting the intensity of contact with the touch-sensitive surface. In some embodiments, the touch-sensitive surface and the display are integrated into a touch-sensitive display. In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 6000 are optionally combined, and / or the order of some operations is optionally changed.

[0264] Method 6000 involves displaying an affordance (an affordance indicating an acceptable starting area for a gesture to display a home screen) on content in a manner such that a display attribute of the affordance (e.g., a grayscale value or a brightness value) is dynamically changed based on a change in the same display attribute of the underlying content. Specifically, the value of the display attribute of the affordance changes in a direction opposite to the change in the value of the same display attribute of the underlying content (e.g., the grayscale value of the affordance is the inverse of the grayscale value of the underlying content). In addition, the value of the display attribute of the affordance is constrained to a smaller range of values ​​than the value of the display attribute of the underlying content. Thus, the device is able to provide the affordance in a less distracting or intrusive manner while maintaining sufficient affordance visibility as the appearance of the content continues to change (e.g., due to scrolling, context switching, and dynamic content playback). Providing an affordance representation having an appearance that dynamically changes based on the appearance of the underlying content in the manner described herein enhances the operability of the device (e.g., by providing guidance to the user regarding the required inputs for the desired results without causing undue distractions to the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by utilizing the required inputs to help the user achieve the desired results and reduce user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently). Providing affordance representations in the manner described herein allows the on-screen affordance representation to effectively replace hardware buttons that provide the same functionality (e.g., displaying the home screen) in many different user interface scenarios, which helps reduce the manufacturing and maintenance costs of the device. In addition, the claimed solution constrains the value range of the affordance representation to eliminate the black-on-white and white-on-black contrast between the affordance representation and the background content, thereby reducing the risk of display afterimages caused by displaying the affordance representation for an extended period of time on a white or black background. Known causes of screen afterimage retention are the prolonged display of non-moving images (e.g., system-level affordances such as the home gesture indicator), combined with non-uniform use of pixels (e.g., this is most severe in situations where the contrast between foreground and background content is high). In certain usage scenarios (e.g., implementing a system-level home affordance), the proposed solutions (e.g., reducing extremely high contrast while maintaining the visual prominence of the affordance) effectively address the afterimage problem. The afterimage problem in mobile phone displays and its causes have been well documented in industry literature for many years and remain a problem in many commercial products. So far, attempts have been made to address the problem by having the affordances move around the screen or simply disappear after a period of inactivity. However, such solutions can make the device less operable.The claimed solution, in which the inversion of a display property of the underlying content provides a basis for determining a value of the same display property of the affordance within a sub-range of display property values, allows the affordance to be used at the same location without the risk of creating afterimages.

[0265] Method 6000 is performed on a device having a display and a touch-sensitive surface (e.g., a touch screen display that functions as both a display and a touch-sensitive surface). The device displays (6002) content (e.g., a home screen, a widget screen, a desktop, a user interface of an application, a media player user interface, etc.) and an affordance (e.g., a home affordance indicating a home gesture reaction area on the display) on the display, wherein: the affordance is displayed on a portion of the content; the value of a display attribute of the affordance (e.g., a grayscale value or brightness value of an image (e.g., a color image or a monochrome image)), an intrinsic display parameter other than the grayscale value or brightness value (e.g., hue, saturation, etc. of a full-color image, or a derived display parameter calculated based on one or more intrinsic display parameters (e.g., a grayscale value or brightness value of a full-color image, or a variant or equivalent thereof)) is determined based on the value of the same display attribute of a portion of the content displayed on the affordance; and the value of the display attribute of the content is allowed to be displayed on a first A range of values ​​(e.g., a range [0, 1], where a "range" is mathematically defined as the difference in values ​​between a range maximum and a range minimum), and a value of a display attribute of an enable representation is constrained to vary within a second range of values ​​that is smaller than a first range of values ​​(e.g., one of the ranges [0, 0.4], [0.6, 1], [0.1, 0.7], [0, 0.7], or [0.3, 1], etc., where a "sub-range" of a "first range" is defined as a range in which a maximum value is smaller than a maximum value of the first range and a minimum value is greater than a minimum value of the first range, or a range in which a maximum value is smaller than a maximum value of the first range and a minimum value is the same as a minimum value of the first range, or a range in which a maximum value is the same as a maximum value of the first range and a minimum value is greater than a minimum value of the first range); when displaying content and an enable representation, the device detects (6004) a change in the appearance of the content of the display enable representation. In response to detecting a change in the appearance of content displayed as an enable indication, the device changes (6006) the appearance of the enable indication, including: based on determining that the value of the display attribute of the content has decreased, increasing the value of the display attribute of the enable indication based on the magnitude of the change in the value of the display attribute of the content and the second value range (e.g., when content located below the enable indication becomes darker, the enable indication becomes brighter); and based on determining that the value of the display attribute of the content has increased, decreasing the value of the display attribute of the enable indication based on the magnitude of the change in the value of the display attribute of the content and the second value range (e.g., when content located below the enable indication becomes brighter, the enable indication becomes darker). This is useful, for example, Fig. 5F , Figure 5G to Figure 5P , Figure 5Qand Figure 5R , where a display property (e.g., brightness) of an affordance changes based on changes in the same display property (e.g., brightness) of the underlying content (e.g., when the content is scrolled). Additionally, the affordance appearance value range of the display property (e.g., brightness) is constrained to be a sub-range of the value range of the same display property of the underlying content (e.g., a full value range from black (e.g., 0 or 0%) to white (e.g., 1 or 100%)).

[0266] In some embodiments, the change in the appearance of the content is (6008) caused by content navigation (e.g., a portion of the content below the affordance changes due to scrolling, paging, etc. of the content). Figure 5G to Figure 5P Changing the value of the same display attribute of the affordance at the same time as the display attribute of the underlying content changes due to content navigation enhances the operability of the device (e.g., by maintaining adequate affordance visibility throughout content navigation and helping the user provide input required to achieve desired results) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby additionally improving power efficiency and battery life of the device (e.g., by reducing user errors and helping the user use the device more quickly and efficiently).

[0267] In some embodiments, the change in the appearance of the content is (6010) caused by the content changing over time (e.g., the content is a video or animation being played, and the image of the video or animation displayed changes over time). Figure 5T to Figure 5U Changing the value of the same display attribute of the affordance while the display attribute of the underlying content changes as the content changes over time enhances the operability of the device (e.g., by maintaining sufficient affordance visibility throughout the content changing over time and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby additionally improving power efficiency and battery life of the device (e.g., by reducing user errors and helping the user use the device more quickly and efficiently).

[0268] In some embodiments, the enable representation has (6012) a first variant and a second variant, the first variant having a first set of end values ​​for a second value range, the second variant having a second set of end values ​​for the second value range, the second set of end values ​​being different from the first set of end values ​​(e.g., for a "bright enable representation" initially displayed on a bright background, the second value range has a first starting point and a first end point, and for a "suggestive enable representation" initially displayed on a dark background, the second value range has a second starting point and a second end point that are different from the first starting point and the second end point, respectively). For example, this Figure 5A and Figure 5R . In some embodiments, the value range of the first variant does not overlap with the value range of the second variant (e.g., all values ​​in the value range of the "bright enable representation" are greater than all values ​​in the value range of the "implied enable representation"). Providing two variants of the enable representation with different value ranges allows the device to further improve the visibility of the enable representation while keeping the appearance of the enable representation unobtrusive on different types of content, thereby enhancing the operability of the device (e.g., by maintaining sufficient visibility of the enable representation for different types of content and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing interference to the user and reducing user errors when using or interacting with the device).

[0269] In some embodiments, a first variant of the enable representation is displayed based on determining that the content corresponds to a first application (6014), and a second variant of the enable representation is displayed based on determining that the content corresponds to a second application different from the first application (for example, the "light" and "dark" enable representations used in the currently displayed application are selected by the application developer of the currently displayed application, and the device displays the first variant or the second variant of the enable representation based on the enable representation selection parameter specified in the program code of the currently displayed application or the corresponding group end value of the second value range (for example, when changing the value of the display attribute of the enable representation, the first group end value or the second group end value of the second value range is used)). Allowing different applications to use different enable representation variants helps application developers customize the appearance of the enable representation based on application scenarios, thereby further improving the compatibility between the appearance of the enable representation and the appearance of the application content, thereby enhancing the operability of the device (for example, by maintaining sufficient visibility of the enable representation without causing undue interference to the user and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (for example, by reducing interference to the user and reducing user errors when using or interacting with the device).

[0270] In some embodiments, the appearance of a first portion of the content of the display affordance is changed (6016) by an amount that is different from the appearance of a second portion of the content of the display affordance; and changing the appearance of the affordance includes changing the appearance of a first portion of the affordance corresponding to the first portion of the content by an amount that is different from the appearance of a second portion of the affordance corresponding to the second portion of the content (e.g., the affordance changes based on a change in the appearance of a portion of the corresponding content located below the first portion of the affordance). For example, this Figure 5Q . For example, if the underlying content changes (e.g., the affordance representation is a blurred / desaturated / inverted version of a portion of the content), the appearance of the affordance representation changes, and the appearance of different portions of the affordance representation reflects the appearance of the content located beneath the different portions of the affordance representation. Allowing different amounts of change to be applied to the display properties of different portions of the affordance representation based on different amounts of change that occur in the display properties of different portions of the underlying content enhances the operability of the device (e.g., by maintaining sufficient affordance visibility throughout content changes and helping the user provide the input needed to achieve a desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby additionally improving power efficiency and battery life of the device (e.g., by reducing user errors and helping the user use the device more quickly and efficiently).

[0271] In some embodiments, the first value range is (6018) a continuous value range, and the second value range includes a discontinuity that corresponds to at least a first value of a display attribute in the first value range. For example, in some embodiments, for a small value range close to 0.5 for a display attribute of content located below the enable representation, the value of the display attribute of the enable representation is discontinuous and jumps from a first value below 0.5 to a second value above 0.5. In some embodiments, the device uses a discontinuous function to calculate the value of the same display attribute of the enable representation based on the value of the display attribute of the underlying content to ensure that the appearance of the enable representation is not too close to the appearance of the underlying content (for example, to ensure that a gray enable representation is not displayed on top of gray content whose grayscale value is very close to that of the gray enable representation). Figure 5AEAn inversion curve including two discontinuous points (e.g., 5034 and 5038) of the value of the display attribute of the affordance is shown. Using a discontinuous value range for the display attribute of the affordance while maintaining a continuous value range for the value display of the underlying content helps avoid the affordance adopting a value that is too close to the value of the underlying content, thereby resulting in insufficient affordance visibility. Using a discontinuous value range for the display attribute of the affordance enhances the operability of the device (e.g., by maintaining sufficient affordance visibility throughout content changes and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby additionally improving power efficiency and battery life of the device (e.g., by reducing user errors and helping the user use the device more quickly and efficiently).

[0272] In some embodiments, the affordance has (6020) a first variant and a second variant (e.g., a "bright affordance" and an "implied affordance"), a first value range corresponding to the first variant of the affordance and a second value range corresponding to the second variant of the affordance do not overlap (e.g., the value range of the "bright affordance" is separated from the value range of the "implied affordance" by a "cutoff value range"), and the device dynamically selects one of the first variant and the second variant to be displayed on the content at a predetermined time (e.g., when an application is started, or when there is a scene change in a video, or when a user interface within an application is switched, etc.) based on the initial value of a display attribute of the content. For example, if the content initially has a darker grayscale value (e.g., less than 0.5), the affordance has an initial value in the bright grayscale value range (e.g., greater than 0.6); and if the content initially has a brighter grayscale value (e.g., greater than 0.5), the affordance has an initial value in the dark grayscale range (e.g., less than 0.4). The precise grayscale value of the affordance is optionally obtained by the following process: the grayscale value of the content is inverted by a corresponding inversion function of the range of light or dark values ​​associated with the initial appearance of the affordance. In some embodiments, if the affordance is a bright affordance (e.g., grayscale value of 0.9) initially displayed on dark content (e.g., grayscale value of 0.2), then as the content below the affordance becomes brighter and brighter (e.g., grayscale value increases toward 1), the affordance becomes darker and darker (e.g., grayscale value decreases toward 0), but the grayscale value of the affordance is constrained by a minimum threshold grayscale value (e.g., 0.6), which is still brighter than the center grayscale value (e.g., 0.5) on the grayscale [0, 1]. As the content below the affordance becomes darker and darker (e.g., grayscale value decreases toward 0), the affordance becomes brighter and brighter (e.g., grayscale value increases toward 1), until when the content becomes completely black, it becomes completely white (e.g., reaches the end value 1 on the grayscale [0, 1]). In another example, if the affordance is an implied affordance (e.g., grayscale value 0.2) that is initially displayed over bright content (e.g., grayscale value 0.9), then as the content beneath the affordance becomes darker (e.g., grayscale value decreases toward 0), the affordance becomes brighter (e.g., grayscale value increases toward 1), but the affordance's grayscale value is constrained by a maximum threshold grayscale value (e.g., 0.4), which is still darker than the center grayscale value (e.g., 0.5) on the grayscale [0, 1]. As the content beneath the affordance becomes brighter (e.g., grayscale value increases toward 1), the affordance becomes darker (e.g., grayscale value decreases toward 0), until it becomes completely black (e.g., reaches the end value 0 on the grayscale [0, 1]) when the content becomes completely white. Figure 5G showing the selection of a hint representation based on an initial overall brightness state of relatively dark underlying content, and Figure 5L Selection of a bright affordance based on an initial overall brightness state of relatively brighter underlying content is shown. Allowing an application to dynamically select from two different variants of affordances (e.g., for darker content versus brighter content) further improves the appearance of affordances based on application context, thereby further improving compatibility between the appearance of affordances and the appearance of underlying content, thereby enhancing device operability (e.g., by maintaining sufficient affordance visibility without unduly distracting the user and helping the user provide input necessary to achieve a desired result), and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0273] In some embodiments, displaying the enablement representation includes (6022) displaying the enablement representation at a first size (e.g., near the bottom edge of the device) when the device is in a first orientation, and the method includes: while displaying the enablement representation at the first size, detecting rotation of the device from the first orientation to a second orientation different from the first orientation (e.g., rotation of the device causes displayed content to change from a first user interface orientation (e.g., a portrait orientation) to a second user interface orientation (e.g., a landscape orientation)); and in response to detecting the rotation of the device from the first orientation to the second orientation, displaying the enablement representation at a second size different from the first size (and optionally, displaying the enablement representation at a different location (e.g., near a new bottom edge of the device as defined based on the second orientation of the device)). In some embodiments, a longer version of the enablement representation is displayed when the device is in a landscape orientation, and a shorter version of the enablement representation is displayed when the device is in a portrait orientation. Displaying affordances of different sizes when the device is rotated improves the visual compatibility between the appearance of the affordances and the orientation of the device (and therefore the orientation of the content), thereby enhancing the operability of the device (e.g., by maintaining sufficient affordance visibility without unduly distracting the user and helping the user provide the input necessary to achieve desired results) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0274] In some embodiments, the display attribute is (6024) a grayscale value (e.g., a grayscale value having a full value range of [0, 1], indicating that the grayscale value ranges from black (e.g., grayscale value = 0) to white (e.g., grayscale value = 1). Providing an indication that changes its grayscale value based on the grayscale value of the underlying content enhances the operability of the device (e.g., by maintaining sufficient visibility of the indication without unduly distracting the user and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0275] In some embodiments, the current value of a display property of the content is obtained (6026) by blurring a portion of the content (e.g., by applying a blur function (e.g., a Gaussian blur function) with a predefined blur radius to the content area below the affordance, and applying a blur function with at least one blur radius around the content area directly below the affordance). In some embodiments, after blurring the content, other filters are applied, such as desaturation and / or changing the opacity. Providing an affordance with the same display properties derived based on the display properties of the underlying content of a blurred version enhances the operability of the device (e.g., by maintaining sufficient affordance visibility without unduly distracting the user and helping the user provide the input necessary to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0276] In some embodiments, the current value of a display property of the content is (6028) obtained by desaturating a portion of the content (e.g., by converting the color value (e.g., RGB value, HSL value, or HSV value) of each pixel in the content area below the affordance representation or at least one blur radius around the area to a corresponding scalar value (e.g., grayscale value) on a monochrome scale (e.g., grayscale). In some embodiments, after the content is desaturated, other filters are applied, such as blurring and / or changing the opacity. Providing an affordance representation with the same display properties derived based on the display properties of the underlying content of a desaturated version enhances the operability of the device (e.g., by maintaining sufficient affordance representation visibility without unduly distracting the user and helping the user provide the input necessary to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0277] In some embodiments, the affordance indicates (6030) a reaction area for initiating a predefined gesture input (e.g., a home / multitasking gesture for displaying a home screen and / or an application switcher user interface). FIG. 5A to FIG. 5D. In some embodiments, the enable representation is not an actual button. Tapping or pressing the enablement indicator does not trigger any function of the device. In some embodiments, when the enablement representation is no longer displayed (for example, after it fades out in the full-screen mode of the application), the predefined gestures (for example, home / multitasking gestures) still work as when the enablement representation is displayed. In some embodiments, when user input is detected (for example, home / multitasking gestures or tap input on the display, etc.), the enablement representation is redisplayed in full-screen content display mode. The enablement representation is a narrow enablement representation with a relatively small height and length. The enablement representation indicating the reaction area for initiating the predefined gesture generally does not need to have particularly enhanced visibility relative to the underlying content, because the reaction area of ​​the gesture is generally wider than the reaction area of ​​the button, so it is more advantageous that the enablement representation can reduce interference with the user, thereby helping to avoid user errors when interacting with the device. Thus, using affordances with dynamically changing display attributes in the manner described herein enhances device operability (e.g., by maintaining sufficient affordance visibility without unduly distracting the user and assisting the user in providing the input necessary to achieve desired results) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0278] It should be understood that FIG. 6A to FIG. 6C The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., method 7000 and method 8000) are also applicable in a similar manner to the details of other processes described above with respect to FIG. 6A to FIG. 6C Method 6000 described herein. For example, the contacts, gestures, user interface objects, application views, control panels, controls, affordances, position thresholds, directional conditions, inversion curves, filters, value ranges, navigation criteria, movement parameters, focus selectors, and / or animations described above with reference to method 6000 optionally have one or more of the features of the contacts, gestures, user interface objects, application views, control panels, controls, position thresholds, directional conditions, navigation criteria, movement parameters, focus selectors, and / or animations described herein with reference to other methods described herein (e.g., methods 7000 and 8000). For the sake of brevity, these details are not repeated here.

[0279] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3described) or application-specific chips.

[0280] References FIG. 6A to FIG. 6C The described operations are optionally performed by Figure 1A to Figure 1B . For example, the detection operation and the change operation are optionally implemented by the event classifier 170, the event identifier 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects the contact on the touch-sensitive display 112, and the event distributor module 174 transmits the event information to the application 136-1. The corresponding event identifier 180 of the application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on the user interface, or the rotation of the device from one orientation to another orientation. When the corresponding predefined event or sub-event is detected, the event identifier 180 activates the event handler 190 associated with the detection of the event or sub-event. The event handler 190 optionally uses or calls the data updater 176 or the object updater 177 to update the application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0281] 7A to 7E 7000 is a flow chart illustrating a method for changing the appearance of an affordance based on changes in the appearance of underlying content and changes in the mode of a user interface displaying the affordance according to some aspects. Figure 3 ; or a portable multifunction device 100, Figure 1A ) is performed on a touch-sensitive surface. In some embodiments, the electronic device includes one or more sensors for detecting the intensity of contact with the touch-sensitive surface. In some embodiments, the touch-sensitive surface and the display are integrated into a touch-sensitive display. In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 600 are optionally combined, and / or the order of some operations is optionally changed.

[0282] Method 7000 involves displaying an affordance (an affordance indicating an acceptable starting area for a gesture to display a home screen) on an application user interface in the following manner: wherein depending on the current display mode of the application user interface, a set of display properties of the affordance (e.g., grayscale values, brightness values, opacity, hue, saturation, etc.) are changed based on a set of display properties of underlying content according to two different sets of rules. For example, when the application user interface is displayed in an interactive mode (e.g., where user input is expected to be frequent), the appearance of the affordance changes in a first manner (e.g., based on a first set of rules) based on the appearance of the underlying content, making the affordance more noticeable; and when the application user interface is displayed in a full-screen content display mode (e.g., where content viewing may be the primary goal), the appearance of the affordance changes in a second manner (e.g., based on a second set of rules) based on the appearance of the underlying content, making the affordance less distracting to the user. Thus, providing an affordance representation that changes its appearance in different ways based on the appearance of the underlying content based on the display mode of the application user interface enhances the operability of the device (e.g., by providing the user with an appropriate amount of guidance on the required inputs for the desired results without causing undue interference to the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by utilizing the required inputs to help the user achieve the desired results and reduce user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently). Providing affordance representations in the manner described herein allows the on-screen affordance representation to effectively replace hardware buttons that provide the same function (e.g., displaying the home screen) in many different user interface scenarios, which helps reduce the manufacturing and maintenance costs of the device. Providing affordance representations in the manner described herein also helps reduce and eliminate the problem of afterimages on the display.

[0283] Method 7000 is performed on a device having a display and a touch-sensitive surface (e.g., a touch screen display that functions as both a display and a touch-sensitive surface). The device displays (7002) a user interface of an application (e.g., a media player user interface, a browser user interface, an instant messaging user interface, a map user interface, a phone user interface, a game user interface, etc.) in a first mode (e.g., a content display mode that does not include full screen content, a windowed mode, or a default mode). When displaying the user interface of the application in the first mode, the device displays (7004) an enable representation having a first appearance on the user interface (e.g., a home enable representation indicating a home gesture reaction area on the display), wherein: the enable representation is displayed on a portion of the user interface (e.g., the enable representation is displayed in a first predefined area of ​​the display (e.g., a home enable representation display area located near the bottom center area of ​​the display), and according to a first set of one or more rules, the values ​​of a set of one or more display properties of the enable representation having the first appearance change according to a set of one or more display properties according to a change in the values ​​of a set of one or more display properties of a portion of the user interface located below the enable representation (e.g., the set of one or more display properties of the enable representation are obtained by the following process: applying a first set of filters to desaturate, blur, change the opacity and / or invert the brightness or grayscale value of the image of a portion of the user interface located below the enable representation)). When an enable representation having a first appearance is displayed on a portion of a user interface displayed in a first mode, the device detects (7006) a request (e.g., a request generated by an application providing the user interface or an operating system based on current operating conditions (e.g., the absence of input for a long period of time, predefined criteria for switching between operating modes, etc.), or a user request (e.g., a tap input or a swipe input performed by contact on a touch screen, etc.)) to transition from displaying the user interface in the first mode to displaying the user interface in a second mode (e.g., a full-screen content display mode).In response to detecting the request: the device displays (7008) the user interface in a second mode (e.g., in a full-screen content display mode, a portion of the original user interface is enlarged, some user interface elements in the user interface (such as an application menu bar, scroll bars, etc.) are removed from the user interface, and the system status bar previously displayed with the user interface is also removed from the display); and the device displays an affordance having a second appearance on the user interface displayed in the second mode, wherein: according to a second set of one or more rules different from the first set of one or more rules, the values ​​of a set of one or more display attributes of the affordance having the second appearance change according to changes in the values ​​of a set of one or more display attributes of a portion of the user interface located below the affordance (e.g., the set of one or more display attributes of the affordance is obtained by applying a second set of filters to desaturate, blur, change the opacity and / or invert the brightness or grayscale value of the image of a portion of the user interface located below the affordance) (e.g., the affordance having the second appearance is a variation of the affordance having the first appearance, but both are derived from the portion of the user interface located below the affordance, but using different sets of filters, or the same set of filters with different adjustment parameters). This is. Figure 5S to Figure 5W , where for example when the device is initially in a first state (e.g., Figure 5T and Figure 5U The affordance is then displayed in a fully visible state (e.g., Figure 5V and Figure 5W The low contrast state in the display can indicate 5002.

[0284] In some embodiments, when an affordance having a second appearance is displayed on a user interface displayed in a second mode: based on determining that a fade-out criterion is met, the device stops (7010) displaying the affordance on the user interface displayed in the second mode (e.g., while maintaining the user interface displayed in the second mode); and based on determining that the fade-out criterion is not met, maintaining the display of the affordance having the second appearance on the user interface displayed in the second mode. For example, this Figure 5X, where the enable representation 5002 in the low contrast state eventually disappears completely after an additional period of time without user input. In some embodiments, the fade-out criteria require that no user input is detected on the touch-sensitive surface for at least a predefined threshold amount of time in order for the fade-out criteria to be met (for example, the fade-out criteria is met when the device does not detect any touch input anywhere on the touch-sensitive surface within 30 seconds after entering full-screen content display mode, or when the device does not detect any touch input near the bottom center area of ​​the touch screen display within 30 seconds after entering full-screen content display mode (for example, other portions of the display may still continue to receive and respond to user input without affecting the determination of the fade-out criteria)). Fading out or maintaining the display of an affordance displayed in a predefined display mode of an application user interface based on predefined criteria enhances the operability of the device (e.g., by providing the user with an appropriate amount of guidance regarding the required inputs for a desired result without unduly distracting the user, which can reduce user errors when operating the device) and makes the user-device interface more efficient (e.g., by helping the user achieve desired results with the required inputs and reducing user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently).

[0285] In some embodiments, the user interface of an application displayed in a first mode includes (7012) a representation of content (e.g., a video, a game, a document, a song album picture) that occupies less than all of the display area of ​​the display (e.g., the user interface of the application displayed in the first mode is displayed simultaneously with a system status bar on the display); and the user interface of the application displayed in a second mode includes a representation of content that occupies all of the display area of ​​the display (e.g., in a full-screen content display mode, a portion of the original user interface is enlarged, some user interface elements in the user interface (such as an application menu bar, scroll bars, etc.) are removed from the user interface, and the system status bar previously displayed with the user interface is also removed from the display). Providing affordances that change their appearance in different ways based on the appearance of the underlying content, depending on whether the content is displayed in a regular display mode or a full-screen display mode, enhances the operability of the device (e.g., by providing the user with appropriate guidance as to the required inputs for a desired result without unduly distracting the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by helping the user achieve the desired result with the required inputs and reducing user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently).

[0286] In some embodiments, at least one of the first appearance and the second appearance of the affordance (e.g., as reflected by the brightness, intensity, or grayscale value of the affordance) is based on (7014) an inversion of a portion of the user interface located below the affordance. For example, in some embodiments, a portion of the user interface located below the affordance is desaturated to obtain a monochrome image, the monochrome image is blurred, and then the brightness or grayscale values ​​of the pixels in the blurred monochrome image are inverted to obtain the brightness or grayscale values ​​of the pixels in the affordance. Providing an affordance with display properties derived based on an inversion of the underlying content enhances the operability of the device (e.g., by maintaining sufficient affordance visibility without causing undue distraction to the user and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distraction to the user and reducing user errors when using or interacting with the device).

[0287] In some embodiments, a first set of rules requires (7016) applying a first inversion amount to a portion of a user interface located below the enable representation to obtain a first appearance of the enable representation, a second set of rules requires applying a second inversion amount to a portion of the user interface located below the enable representation to obtain a second appearance of the enable representation, and the second inversion amount is less than the first inversion amount (e.g., the second set of rules reduces the inversion amount of the portion of the user interface located below the enable representation to obtain a set of display properties of the enable representation). Changing the amount of inversion applied to the underlying content to obtain the enable representation helps to adjust the visibility of the enable representation based on the display mode of the application user interface, thereby enhancing the operability of the device (e.g., by intelligently balancing the requirement for sufficient visibility of the enable representation with the requirement for unobtrusiveness and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0288] In some embodiments, at least one of the first appearance and the second appearance of the enable representation (e.g., as reflected by the brightness, intensity, or grayscale value of the enable representation) is obtained by (7018) blurring a portion of the user interface located below the enable representation. For example, in some embodiments, a portion of the user interface located below the enable representation is desaturated to obtain a monochrome image, the monochrome image is blurred, and then the brightness or grayscale values ​​of the pixels in the blurred monochrome image are inverted to obtain the brightness or grayscale values ​​of the pixels in the enable representation. Providing an enable representation with display properties obtained by blurring the underlying content enhances the operability of the device (e.g., by maintaining sufficient visibility of the enable representation without causing undue distraction to the user and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distraction to the user and reducing user errors when using or interacting with the device).

[0289] In some embodiments, a first set of rules requires (7020) applying a first amount of blur (e.g., a Gaussian blur function) to a portion of a user interface located below the affordance to obtain a first appearance of the affordance, a second set of rules requires applying a second amount of blur (e.g., a Gaussian blur function) to a portion of a user interface located below the affordance to obtain a second appearance of the affordance, and the second amount of blur is less than the first amount of blur (e.g., the second blur function has a smaller blur radius than the first blur function) (e.g., the second set of rules reduces the amount of blur of the portion of the user interface located below the affordance to obtain a set of display properties of the affordance). Changing the amount of blur applied to the underlying content to obtain the affordance helps to adjust the visibility of the affordance according to the display mode of the application user interface, thereby enhancing the operability of the device (e.g., by intelligently balancing the requirement for sufficient visibility of the affordance with the requirement for unobtrusiveness and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0290] In some embodiments, at least one of the first appearance and the second appearance of the enable representation (e.g., as reflected by the brightness, intensity, or grayscale value of the enable representation) is obtained by (7022) desaturating a portion of the user interface located below the enable representation. For example, in some embodiments, a portion of the user interface located below the enable representation is desaturated to obtain a monochrome image, the monochrome image is blurred, and then the brightness or grayscale values ​​of the pixels in the blurred monochrome image are inverted to obtain the brightness or grayscale values ​​of the pixels in the enable representation. Providing an enable representation with display properties obtained by desaturating the underlying content enhances the operability of the device (e.g., by maintaining sufficient visibility of the enable representation without causing undue distraction to the user and helping the user provide the input required to achieve the desired result) and makes user device operation more efficient (e.g., by reducing distraction to the user and reducing user errors when using or interacting with the device).

[0291] In some embodiments, a first set of rules requires (7024) applying a first amount of desaturation to a portion of a user interface located below the enable representation to obtain a first appearance of the enable representation, a second set of rules requires applying a second amount of desaturation to a portion of the user interface located below the enable representation to obtain a second appearance of the enable representation, and the second amount of desaturation is less than the first amount of desaturation (e.g., the second set of rules reduces the amount of desaturation of the portion of the user interface located below the enable representation to obtain a set of display properties of the enable representation). Changing the amount of desaturation applied to the underlying content to obtain the enable representation helps to adjust the visibility of the enable representation based on the display mode of the application user interface, thereby enhancing the operability of the device (e.g., by intelligently balancing the requirement for sufficient visibility of the enable representation with the requirement for unobtrusiveness and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0292] In some embodiments, when the user interface is displayed in the second mode without displaying the enable indication, the device detects (7026) an input that satisfies a first enable indication redisplay criterion, wherein the first enable indication redisplay criterion is satisfied when the input is detected on the touch-sensitive surface (e.g., at a location corresponding to the first display location); and in response to detecting the input that satisfies the enable indication redisplay criterion, the device redisplays the enable indication on the user interface displayed in the second mode. For example, this Figure 5X to Figure 5AAIn some embodiments, when the affordance is redisplayed on the user interface displayed in the second mode, the values ​​of a set of one or more display attributes of the affordance change according to changes in the values ​​of a set of one or more display attributes of a portion of the user interface according to a second set of one or more rules. In some embodiments, when the affordance is redisplayed on the user interface displayed in the first mode, the values ​​of a set of one or more display attributes of the affordance change according to changes in the values ​​of a set of one or more display attributes of a portion of the user interface according to a first set of one or more rules. Redisplaying the affordance after fading out the affordance based on predefined criteria enhances the operability of the device (e.g., by providing the user with an appropriate amount of guidance on the required inputs for the desired results without causing undue interference to the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results with the required inputs and reducing user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently).

[0293] In some embodiments, when the user interface is displayed in the second mode without displaying the enable indication, the device detects (7028) an input that satisfies a second enable indication redisplay standard, wherein when the input is a request to change from displaying the user interface in the second mode to displaying the user interface in the first mode, the second enable indication redisplay standard is satisfied; and in response to detecting the input that satisfies the second enable indication redisplay standard, the device redisplays the user interface in the first mode; and the device redisplays the enable indication on the user interface redisplayed in the first mode. For example, this Figure 5X to Figure 5AA In some embodiments, when the affordance is redisplayed on the user interface displayed in the first mode, the values ​​of a set of one or more display attributes of the affordance change according to changes in the values ​​of a set of one or more display attributes of a portion of the user interface, according to a first set of one or more rules. The affordance is redisplayed when transitioning from the second display mode to the first display mode, which enhances the operability of the device (e.g., by providing the user with appropriate guidance on the required inputs for the desired results without causing undue interference to the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results with the required inputs and reducing user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently).

[0294] In some embodiments, at least one of the first appearance and the second appearance of the affordance is dynamically adjusted based on (7030) a change that occurs in a portion of the user interface that is located below the affordance (e.g., a change in a portion of the user interface due to dynamic content in the user interface or due to navigation through content displayed in the user interface). For example, this Figure 5T to Figure 5W . For example, when the user interface is scrolled, or when the user interface displayed in the second mode is a full-screen movie, game, or web page that is constantly evolving and refreshing, the appearance of the affordance is also constantly updated to reflect the changes in the user interface below. When the user interface is displayed in the first mode, the appearance of the affordance changes dynamically according to a first set of rules, and when the user interface is displayed in the second mode, the appearance of the affordance changes dynamically according to a second set of rules. Dynamically changing the appearance of the affordance based on changes in the appearance of the content below enhances the operability of the device (e.g., by providing the user with appropriate guidance on the required inputs for the desired results as the content continues to change over time without causing undue distractions to the user), and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results with the required inputs and reducing user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently).

[0295] In some embodiments, the first appearance is (7032) generated based on a first set of filters (e.g., blur, desaturation, and inversion) applied to a portion of the user interface located below the affordance (and an edge area around that area), the second appearance is generated based on a second set of filters applied to a portion of the user interface located below the affordance, and for two or more filters in the second set of filters, the first set of filters includes corresponding filters of the same type but with different adjustment parameters (e.g., blur filters with different blur radii, inversion filters with different inversion curves, opaque filters with different transparency levels, desaturation filters with different desaturation ratios, etc.). Using a set of filters with different adjustment parameters to provide affordances that change their appearance in different ways based on the appearance of the underlying content allows the affordance to maintain a relatively consistent appearance and be less distracting to the user during switching display modes, thereby enhancing the operability of the device (e.g., by providing the user with an appropriate amount of guidance on the required input for the desired result without causing undue interference to the user, which can reduce user errors when operating the device), and making the user-device interface more effective (e.g., by helping the user achieve the desired result with the required input and reducing user errors when operating / interacting with the device).

[0296] In some embodiments, in response to detecting a request: the device generates (7034) one or more intermediate appearances for the enable representation, the one or more intermediate appearances being between the first appearance and the second appearance; and the device displays the one or more intermediate appearances for the enable representation as a transition between displaying the enable representation having the first appearance and displaying the enable representation having the second appearance. In some embodiments, the intermediate appearance is displayed on the user interface in a second mode, and the intermediate appearance is an interpolation between the first appearance and the second appearance of the enable representation. Generating the intermediate appearance of the enable representation to bridge the transition of the appearance of the enable representation during the display mode switching of the application user interface makes it less distracting to the user, thereby enhancing the operability of the device (e.g., by providing the user with an appropriate amount of guidance on the required input for the desired result without causing undue interference to the user, which can reduce user errors when operating the device), and making the user-device interface more effective (e.g., by utilizing the required input to help the user achieve the desired result and reduce user errors when operating / interacting with the device).

[0297] In some embodiments, generating one or more intermediate appearances between a first appearance and a second appearance for an affordance includes (7036) gradually transitioning from a first set of rules to a second set of rules (e.g., changing the magnitude of one filter applied to underlying content to generate the affordance without changing the magnitude of other filters applied to underlying content to generate the affordance). By gradually transitioning a set of rules for generating the appearance of an affordance to generate an intermediate appearance of an affordance, the intermediate appearance is generated so that the user's attention is less distracted, thereby enhancing the operability of the device (e.g., by providing the user with appropriate guidance on the required inputs for the desired results without causing undue interference to the user, which can reduce user errors when operating the device), and making the user-device interface more efficient (e.g., by utilizing the required inputs to help the user achieve the desired results and reducing user errors when operating / interacting with the device).

[0298] In some embodiments, the affordance having the first appearance has (7038) a first opacity, and the affordance having the second appearance has a second opacity that is less than the first opacity (e.g., the color of the user interface is displayed as being transparent). Changing the opacity of the affordance based on the display mode of the application user interface helps to adjust the visibility of the affordance according to the display mode of the application user interface, thereby enhancing the operability of the device (e.g., by intelligently balancing the requirement for sufficient visibility of the affordance with the requirement for unobtrusiveness and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0299] In some embodiments, the affordance having the first appearance and the affordance having the second appearance have (7040) the same size and position on the display. Maintaining the size and position of affordances in different display modes of the application user interface helps maintain the continuity of the affordance appearance during user interface switching, thereby enhancing the operability of the device (e.g., by maintaining the user's context and helping the user provide the input required to achieve the desired result) and making user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0300] In some embodiments, the visual distinction between the affordance formed (7042) by the first set of one or more rules and the user interface of the application is greater than the visual distinction between the affordance formed by the second set of one or more rules and the user interface of the application. Changing the visual distinction of the affordance based on the display mode of the application user interface helps to adjust the visibility of the affordance according to the display mode of the application user interface, thereby enhancing the operability of the device (e.g., by intelligently balancing the requirement for sufficient visibility of the affordance with the requirement for unobtrusiveness and helping the user provide the input required to achieve the desired result) and making the user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device).

[0301] It should be understood that 7A to 7E The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., method 6000 and method 8000) are also applicable in a similar manner to the details of other processes described above with respect to 7A to 7E Method 7000 described herein. For example, the contacts, gestures, user interface objects, application views, control panels, controls, affordances, position thresholds, directional conditions, inversion curves, filters, value ranges, navigation criteria, movement parameters, focus selectors, and / or animations described above with reference to method 7000 optionally have one or more of the features of the contacts, gestures, user interface objects, application views, control panels, controls, position thresholds, directional conditions, navigation criteria, movement parameters, focus selectors, and / or animations described herein with reference to other methods described herein (e.g., methods 6000 and 8000). For the sake of brevity, these details are not repeated here.

[0302] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0303] References 7A to 7E The described operations are optionally performed by Figure 1A to Figure 1B 112. For example, the detection operation and the display operation are optionally implemented by the event classifier 170, the event identifier 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects the contact on the touch-sensitive display 112, and the event distributor module 174 transmits the event information to the application 136-1. The corresponding event identifier 180 of the application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on the user interface, or the rotation of the device from one orientation to another orientation. When the corresponding predefined event or sub-event is detected, the event identifier 180 activates the event handler 190, which is associated with the detection of the event or sub-event. The event handler 190 optionally uses or calls the data updater 176 or the object updater 177 to update the application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0304] FIG. 8A to FIG. 8F 8000 is a flowchart illustrating a method for changing the appearance of an affordance and the type of affordance appearance according to a change in the appearance of underlying content according to some embodiments. Figure 3 ; or a portable multifunction device 100, Figure 1A ) is performed on a touch-sensitive surface. In some embodiments, the electronic device includes one or more sensors for detecting the intensity of contact with the touch-sensitive surface. In some embodiments, the touch-sensitive surface and the display are integrated into a touch-sensitive display. In some embodiments, the display is a touch screen display, and the touch-sensitive surface is on the display or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 8000 are optionally combined, and / or the order of some operations is optionally changed.

[0305] Method 8000 involves displaying an affordance (e.g., an affordance indicating an acceptable starting area for a gesture for performing a predefined operation in a user interface (such as displaying a home screen or an application switcher user interface)) on content in a manner such that the same display properties of the affordance are dynamically changed based on changes in display properties of the underlying content (e.g., a grayscale value or a brightness value). In addition, the method requires that the affordance appearance value range switch between two different value ranges (e.g., a "suggestive affordance" value range and a "bright affordance" value range) depending on whether the change in the display property of the underlying content satisfies a predefined range switching criterion. Thus, the device is able to adapt the appearance of the affordance to changes in the underlying content so as to maintain visual contrast between the affordance and the underlying content, and to provide the affordance in a less distracting or less intrusive manner (e.g., by avoiding rapid flashing of the affordance due to changing the affordance appearance too quickly, which in some cases can distract the user), without being constrained by the affordance appearance type initially selected (e.g., a "bright" or "dark" affordance type). Providing an affordance representation having an appearance that changes dynamically based on the appearance of the underlying content and allowing the affordance representation appearance value range to switch dynamically over time based on changes in the underlying content in the manner described herein enhances the operability of the device (e.g., by providing the user with visual guidance about the required inputs for the desired results without causing undue distractions to the user, which can reduce user errors when operating the device), and makes the user-device interface more efficient (e.g., by utilizing the required inputs to help the user achieve the desired results and reduce user errors when operating / interacting with the device), thereby improving the battery life of the device (e.g., by helping the user use the device more quickly and efficiently). Providing affordance representations in the manner described herein allows on-screen affordance representations to effectively replace and improve hardware buttons that provide the same function (e.g., displaying the home screen) in many different user interface scenarios, which helps reduce the manufacturing and maintenance costs of the device. Providing affordance representations in the manner described herein also helps reduce and eliminate the problem of afterimages on the display.

[0306] Method 8000 is performed on a device having a display and a touch-sensitive surface (e.g., a touch screen display that functions as both a display and a touch-sensitive surface). The device displays (8002) content (e.g., a home screen, a widget screen, a desktop, a user interface of an application, a media player user interface, etc.) and an affordance (e.g., a home affordance indicating a home gesture reaction area on the display) on the display, wherein: the affordance is displayed on a portion of the content; the value of a display attribute of the affordance (e.g., a grayscale value or brightness value of an image (e.g., a color image or a monochrome image)), an intrinsic display parameter other than a grayscale value or brightness value (e.g., hue, saturation, etc. of a full-color image, or a derived display parameter calculated based on one or more intrinsic display parameters (e.g., a grayscale value or brightness value of a full-color image, or a slight variation or equivalent thereof)) is based on a portion of the content displayed on which the affordance is displayed ; and the value of the display attribute of the content is allowed to vary within a first value range (e.g., the range [0, 1], optionally on a scale of 0% brightness to 100% brightness, where 0% brightness is 0 and 100% brightness is 1), and the value of the display attribute of the affordance is constrained to vary within an affordance appearance value range that is smaller than the first value range (e.g., for a suggestive affordance, the affordance appearance value range is [0, 0.4], optionally on a scale of black to white, where black is 0 and white is 1; or for a bright affordance, the value range is [0.6, 1], optionally on a scale of 0% brightness to 100% brightness, where 0% brightness is 0 and 100% brightness is 1; both of these ranges are smaller than the value range [0, 1] of the content). When displaying content and an enable representation and when the enable representation appearance value range is a second value range (e.g., a black enable representation appearance value range of [0, 0.4], optionally on a black to white scale where black is 0 and white is 1), the device detects (8004) a change in the appearance of the content displayed by the enable representation.In response to detecting a change in the appearance of content displayed as an affordance, the device changes (8006) the appearance of the affordance (e.g., based on a short-term appearance change strategy and a long-term appearance change strategy), including: based on determining that the change in the appearance of the content satisfies a range switching criterion (e.g., when a measure of the overall brightness or darkness of the content below and around the affordance (e.g., the cumulative value and aggregate value of display properties (e.g., grayscale values ​​or brightness)) exceeds a first predefined threshold due to a change in the appearance of background content, the range switching criterion is satisfied): shifting the affordance appearance value range to a third range of values ​​(e.g., an affordance appearance value range of a light affordance [0.6, 1], optionally on a scale from black to white, where black is 0 and white is 1), where the third value range is different from the second value range (e.g., the third value range includes at least one value that is not included in the second value range and optionally does not overlap with the second value range) (e.g., when the range switching criterion is satisfied, the currently selected affordance type is changed from a previously selected affordance type (e.g., a suggestive affordance) to an alternative affordance type (a bright affordance) Representation)), and a third value range (e.g., [0.6, 1], optionally on a scale from black to white, where black is 0 and white is 1) is smaller than the first value range; and the value of the same display attribute of the enable representation is changed according to the value of the display attribute of the content of the display enable representation (the value of the display attribute of each pixel of the enable representation is determined based on a first predefined value mapping corresponding to the currently selected enable representation type (e.g., bright enable representation)), wherein the display attribute of the enable representation is constrained to vary within the enable representation appearance value range; and based on determining that the appearance change of the content does not meet the range switching criteria, the value of the same display attribute of the enable representation is changed according to the value of the display attribute of the content of the display enable representation (e.g., the value of the display attribute of each pixel of the enable representation is determined based on a second predefined conversion relationship corresponding to the currently selected enable representation type (e.g., implied enable representation)), while maintaining the enable representation appearance value range as the second value range (e.g., when the first criterion is not met, the currently selected enable representation type remains the same as the previously selected enable representation type (e.g., implied enable representation)). For example, this is. Figures 5AD to 5AE Shown in.

[0307] In some embodiments, the range switch criteria include (8008) range switch trigger criteria and range switch completion criteria, the range switch trigger criteria requiring that the change in the appearance of the content includes a first amount of change within a first time period, the first amount of change causing a predefined metric of the content appearance (e.g., a biased running average of a display attribute (such as, an aggregate brightness value of a portion of the content below and below the enable representation)) to exceed a predefined threshold (e.g., the predefined threshold is a first threshold when switching from a bright enable representation to a hint enable representation, and the predefined threshold is a second threshold different from the first threshold when switching from the hint enable representation to the bright enable representation), and the range switch completion criteria requiring that the change in the appearance of the content does not include a second amount of change within a second time period after the first time period, so as to satisfy the range switch trigger criteria again before a predefined transition period (e.g., 5 seconds) expires after the range switch trigger criteria is satisfied. Requiring that the range switching trigger criteria are not met again within a predefined transition period after the range switching trigger criteria are met for the first time so that the enabling representation appearance range switching between two value ranges is completed, the operability of the device is enhanced (for example, by avoiding unnecessary switching of the enabling representation appearance type and avoiding causing interference to the user when the change of the underlying content is instantaneous), and making the user device operation more efficient (for example, by reducing interference to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (for example, reducing the demand on the processor and screen due to range switching).

[0308] In some embodiments, a predefined threshold for a range switching trigger criterion is selected based on (8010) a current value range used as an enable representation appearance value range (e.g., a value range of the currently used enable representation type), including: when the enable representation appearance value range is a second value range, using a first threshold as a predefined threshold; and when the enable representation appearance value range is a third value range, using a second threshold as a predefined threshold. (In some embodiments, when the enable representation appearance value range is between a second value range (e.g., a value range associated with “implied enable representation”) and a third value range (e.g., a value range associated with “bright enable representation”), the predefined threshold is based on any value range most recently selected as the enable representation appearance value range (e.g., once the “bright enable representation” range of the enable representation appearance value is selected as the target range of the enable representation appearance value, the “bright enable representation” threshold is used to determine when to switch back to the “implied enable representation” range of the enable representation appearance value, and once the “implied enable representation” range of the enable representation appearance value is selected as the target range of the enable representation appearance value , the "hinted affordance" threshold is used to determine when to switch back to the "lit affordance" range of affordance appearance values). Using different thresholds for range switching trigger criteria enhances device operability (e.g., by adjusting a preference for the currently selected affordance appearance type and avoiding unnecessary switching of affordance appearance types and avoiding distracting the user when changes in underlying content are instantaneous), and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving device battery life (e.g., reducing demands on the processor and screen due to range switching).

[0309] In some embodiments, changing the appearance of an enable representation in response to detecting a change in the appearance of content displayed as an enable representation includes (8012): based on determining that a range switching trigger criterion is satisfied by a first change amount within a first time period and before a range transition completion criterion is satisfied: shifting the enable representation appearance value range to an intermediate value range that is different from the second value range and the third value range (e.g., an intermediate value range [0.3-0.7], optionally on a black to white scale, where black is 0 and white is 1, the starting value of the intermediate value range is between the starting value of the second value range and the starting value of the third value range, and its ending value is between the ending value of the second value range and the ending value of the third value range); and changing the value of the same display attribute of the enable representation based on the value of the display attribute of the content displayed as the enable representation (determining the value of the display attribute of each pixel of the enable representation based on a first predefined value mapping corresponding to the currently selected enable representation type (e.g., a bright enable representation)), wherein the display attribute of the enable representation is constrained to vary within the enable representation appearance value range. In some embodiments, there are multiple intermediate value ranges between the second value range and the third value range, and the device moves sequentially through each of the multiple intermediate value ranges within a predefined transition period until the range switch completion criteria are met. After the range switch completion criteria are met, when the display attributes of the enable representation are changed according to any additional appearance changes of the content, the display attributes of the enable representation are constrained within the third value range. Shifting the enable representation appearance value range to an intermediate value range that is different from (or between) the appearance range values ​​of two stable enable representation appearance types as a transition during switching between two stable enable representation appearance types enhances the operability of the device (e.g., by making switching the enable representation appearance type more sophisticated and less distracting to the user), and makes user device operation more efficient (e.g., by reducing interference to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0310] In some embodiments, the change in the appearance of the content includes (8014) a third amount of change within a third time period after the first time period, after the range switch trigger criteria are met, and before the range switch completion criteria are met. For example, after the range switch trigger criteria are met, the device selects an intermediate content affordance appearance inversion curve between the content affordance appearance inversion curves of the bright affordance and the implied affordance, and uses the intermediate inversion curve to determine how to change the brightness of the affordance based on the brightness change of the content during the transition period (e.g., within 5 seconds after the range switch trigger criteria are met). When the affordance appearance range shifts from one stable value range to another stable value range over a period of time, allowing the appearance of the affordance to continue to change with the underlying content enhances the operability of the device (e.g., by providing visual continuity of the affordance appearance during switching affordance appearance types), and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0311] In some embodiments, the change in appearance of the content of the display affordance is (8016) caused by scrolling of the content, and the range switching criteria do not require a lack of scrolling of the content in order to meet the range switching criteria. Allowing the appearance of the affordance to change with the underlying content as the underlying content is scrolled enhances the operability of the device (e.g., by providing visual continuity of the affordance appearance as the underlying content is scrolled) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0312] In some embodiments, the change in appearance of the content of the display affordance is (8018) caused by movement of the content below the affordance (e.g., zooming or scrolling of the content), and the range switching criteria requires that the content below the affordance moves less than a predefined amount for at least a predetermined amount of time in order to meet the range switching criteria. For example, in addition to requiring a predefined metric of the content appearance (e.g., a biased running average of a display attribute (such as, an aggregate brightness value of a portion of the content below and below the affordance) to exceed a predefined threshold, the range switching criteria also requires that the content remain substantially stationary for a short period of time near the time when the predefined threshold is exceeded. Requiring the content below the affordance to be substantially stationary (e.g., stop scrolling) in order to meet the range switching criteria enhances the operability of the device (e.g., by avoiding unnecessary switching back and forth of affordance appearance types when the content changes rapidly due to continued scrolling) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0313] In some embodiments, changing the value of the same display attribute of the enable representation based on the value of the display attribute of the content includes (8020): based on determining that the value of the display attribute of the content has decreased (and optionally, the value of the display attribute of the content is within a predefined sub-range of the first value range (for example, outside two special ranges of content values, the two special ranges have discontinuities in the enable representation value)), increasing the value of the display attribute of the enable representation based on the magnitude of the change (for example, decrease) in the value of the display attribute of the content (for example, when content located below the enable representation becomes darker, the enable representation becomes brighter); based on determining that the value of the display attribute of the content has increased (and optionally, the value of the display attribute of the content is within a predefined sub-range of the first value range (for example, outside two special ranges of content values, the two special ranges have discontinuities in the enable representation value)), decreasing the value of the display attribute of the enable representation based on the magnitude of the change (for example, increase) in the value of the display attribute of the content (for example, when content located below the enable representation becomes brighter, the enable representation becomes darker). Using value inversion (e.g., increasing the value of the affordance when decreasing the value of the content, and decreasing the value of the affordance when increasing the value of the content) to determine the value of the same display attribute of the affordance based on the value of the display attribute of the underlying content enhances the operability of the device (e.g., by providing sufficient affordance visibility without being distracting or intrusive) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0314] In some embodiments, a given magnitude of change in the value of the display attribute of the content causes (e.g., 8022) the same magnitude of change in the value of the display attribute of the affordance when the value of the display attribute of the affordance changes within a second value range (e.g., before the range switching criteria are met) and when the value of the display attribute of the affordance changes within a third value range (e.g., after the range switching criteria are met). Figure 5AE and Figure 5AF . In some embodiments, the content affordance appearance reversal curve between the display attribute of the affordance and the same display attribute of the content (e.g., affordance brightness versus background brightness curve) has the same shape of the implied affordance and bright affordance, and for the currently selected affordance type, the brightness value of each pixel of the affordance is looked up from a database of pre-stored brightness value pairs. Requiring the content below the affordance to be substantially stationary (e.g., stop scrolling) in order to meet the range switching criteria enhances the operability of the device (e.g., by avoiding unnecessary switching back and forth of affordance appearance types when the content changes rapidly due to continued scrolling) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0315] In some embodiments, the device transfers the enable indication appearance value range from (8024) a second value range (e.g., an enable indication appearance value range that implies the enable indication) to a third value range (e.g., an enable indication appearance value range that highlights the enable indication), including gradually transferring from the second value range to the third value range over a period of time (e.g., 5 seconds); and the method includes, when gradually transferring the enable indication appearance value range from the second value range to the third value range: detecting additional appearance changes of the content displayed for the enable indication; and in response to detecting additional appearance changes of the content displayed for the enable indication, adjusting the appearance value range of the enable indication according to the enable indication appearance value. The method of changing the appearance of an enable indication over a range (e.g., as the range of the enable indication appearance value gradually shifts over time) includes: starting to gradually shift the enable indication appearance value range back to the second value range (e.g., changing the start and end points of the value range over a period of time (e.g., 1, 2, 3, 4, 5, 10 seconds) based on determining that the change in the appearance of the content satisfies the range switching criteria (e.g., the range switching criteria is met for the second time); and continuing to gradually shift the enable indication appearance value range from the second value range to the third value range based on determining that the change in the appearance of the content does not satisfy the range switching criteria (e.g., the range switching criteria is met for the second time). In some embodiments, as described above, the predefined threshold for determining whether the range switching criteria are met varies based on whether the target value range of the enable indication appearance value range is the second value range (e.g., for a suggestive enable indication) or the third value range (e.g., for a bright enable indication). The method allows the enable indication to gradually switch from one appearance type to another appearance type within a predefined transition period; and during the predefined transition period, the appearance value of the enable indication continues to change based on the change in the appearance value of the underlying content. Additionally, during the transition of the range of appearance values, the range switching criteria may be met again (e.g., due to the continued change in the appearance of the content and the affordances). Thus, depending on whether the range switching criteria are met again during the transition period, the device enables the triggered range switch to be reversed or continued to complete. The mechanism for reversing the range switch during the transition period enhances the operability of the device (e.g., by avoiding unnecessary switching back and forth of affordance appearance types when the content changes rapidly) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0316] In some embodiments, after changing the value of a display attribute of an enable representation within a third value range based on determining that a change in the appearance of the content satisfies a range switching criterion, the device detects (8026) additional appearance changes of the content displayed by the enable representation; and in response to detecting the additional appearance changes of the content displayed by the enable representation, the device changes the appearance of the enable representation, including: changing the value of the same display attribute of the content displayed by the enable representation based on the value of the display attribute of the content displayed by the enable representation (for example, determining the value of the display attribute of each pixel of the enable representation based on a first predefined value mapping corresponding to a currently selected enable representation type (for example, a bright enable representation)) based on determining that the additional appearance changes of the content satisfy the range switching trigger criterion for the first time and do not satisfy the range switching completion criterion, wherein the value of the display attribute of the enable representation The display attribute is constrained to vary within a first intermediate value range between the second value range and the third value range; based on determining that the additional appearance change of the content satisfies the range switch triggering criteria a second time after the first time without satisfying the range switch completion criteria, changing the value of the same display attribute of the affordance based on the value of the display attribute of the content of the affordance, wherein the display attribute of the affordance is constrained to vary within the second intermediate value range between the third value range and the first intermediate value range; and based on determining that the additional appearance change of the content satisfies the range switch triggering criteria only once before satisfying the range switch completion criteria, changing the value of the same display attribute of the affordance based on the value of the display attribute of the content of the affordance, wherein the display attribute of the affordance is constrained to vary within the second value range. After the affordance switches from the first appearance type to the second appearance type (e.g., in response to satisfying the range switch triggering criteria for the first time), the affordance may begin to switch from the second appearance type back to the first appearance type if the range switch triggering criteria are satisfied again. During the predefined transition period, the appearance value range continues to shift from the value range of the second appearance type to the value range of the first appearance type. During the transition of the range of appearance values, the range switch triggering criteria may be satisfied again (e.g., due to the continued change in the appearance of the content and the affordance). Thus, the triggered range switch may be reversed from its current state and continue again toward the value range of the second appearance type. The mechanism for reversing the range switch during the transition period enhances the operability of the device (e.g., by avoiding unnecessary switching back and forth of affordance appearance types when the content changes rapidly) and makes user device operation more efficient (e.g., by reducing distractions to the user and reducing user errors when using or interacting with the device), thereby improving the battery life of the device (e.g., by allowing the user to operate the device more quickly and efficiently).

[0317] In some embodiments, when the home affordance is initially displayed (e.g., a context switch occurs, resulting in a change in the content displayed on the screen), the device has not yet accumulated a sufficient amount of information about the underlying content to affirmatively select between a highlighted affordance and a hinted affordance based on a measure of the content's appearance in the manner described in various embodiments. Therefore, the device optionally sets the value of the affordance's display attribute to a predefined value, such as an intermediate value (e.g., 0.5 or 50% of the full brightness value), and then allows the dynamic algorithm described above to update the affordance appearance over time (e.g., according to both the short-term and long-term strategies described herein). In some embodiments, when the affordance is initially displayed, the rate of progression for switching affordance appearance types is temporarily increased (e.g., reducing the bias toward maintaining the currently selected affordance type), so that the home affordance quickly adapts to a highlighted affordance or a hinted affordance based on the appearance of the underlying content. In some embodiments, the initial display of the home affordance is triggered by a context switch event in the user interface, such as when a new application user interface is displayed (e.g., via selecting a recently opened application in a multitasking user or launching an application from an application icon on the home screen), when a cover user interface (e.g., a system-level information screen) is pulled down from the edge of the display to cover a currently displayed application user interface or home screen, or when the user interface is rotated (e.g., due to rotating the device), etc. In some embodiments, the affordance is initially displayed using an animated transition that includes gradually increasing the opacity of the affordance over time, sliding the affordance onto the screen from the edge of the screen over time, and / or gradually increasing the size of the affordance over time.

[0318] It should be understood that FIG. 8A to FIG. 8F The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., method 6000 and method 7000) are also applicable in a similar manner to the details of the above description of the operations. FIG. 8A to FIG. 8EMethod 8000 described herein. For example, the contacts, gestures, user interface objects, application views, control panels, controls, affordances, position thresholds, directional conditions, inversion curves, filters, value ranges, navigation criteria, movement parameters, focus selectors, and / or animations described above with reference to method 8000 optionally have one or more of the features of the contacts, gestures, user interface objects, application views, control panels, controls, position thresholds, directional conditions, navigation criteria, movement parameters, focus selectors, and / or animations described herein with reference to other methods described herein (e.g., methods 6000 and 7000). For the sake of brevity, these details are not repeated here.

[0319] The operations in the above information processing method are optionally implemented by running one or more functional modules in an information processing device, such as a general-purpose processor (for example, as described above with respect to Figure 1A and Figure 3 described) or application-specific chips.

[0320] References FIG. 8A to FIG. 8F The described operations are optionally performed by Figure 1A to Figure 1B 112. For example, the detection operation, the transfer operation, and the change operation are optionally implemented by the event classifier 170, the event identifier 180, and the event handler 190. The event monitor 171 in the event classifier 170 detects contact on the touch-sensitive display 112, and the event distributor module 174 transmits the event information to the application 136-1. The corresponding event identifier 180 of the application 136-1 compares the event information with the corresponding event definition 186, and determines whether the first contact at the first position on the touch-sensitive surface (or whether the rotation of the device) corresponds to a predefined event or sub-event, such as the selection of an object on the user interface, or the rotation of the device from one orientation to another orientation. When the corresponding predefined event or sub-event is detected, the event identifier 180 activates the event handler 190 associated with the detection of the event or sub-event. The event handler 190 optionally uses or calls the data updater 176 or the object updater 177 to update the application internal state 192. In some embodiments, the event handler 190 accesses the corresponding GUI updater 178 to update the content displayed by the application. Similarly, those skilled in the art will clearly know that based on Figure 1A to Figure 1B How the components depicted in the figure can implement other processes.

[0321] For the purpose of explanation, the foregoing description is described by reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the disclosed precise form. According to the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best illustrate the principles of the present invention and its practical application, so that other persons skilled in the art can best use the present invention and the various described embodiments with various modifications suitable for the specific purposes contemplated.

Claims

1. A method comprising: At a device having a display and one or more input devices: displaying a user interface of an application via the display, wherein the user interface of the application has a first mode and a second mode different from the first mode, and wherein the user interface of the application has different interaction behaviors in the first mode and the second mode; When the user interface of the application is displayed in the first mode via the display, an affordance having a first appearance is displayed on the user interface via the display, wherein: The affordance is displayed on a portion of the user interface, and while displaying the user interface of the application in the first mode, according to a first set of one or more rules, the values ​​of a set of one or more display properties of the affordance having the first appearance change in accordance with changes in the values ​​of a set of one or more display properties of the portion of the user interface located below the affordance; detecting, via the one or more input devices, a request to transition from displaying the user interface in the first mode to displaying the user interface in the second mode while displaying, via the display, the affordance having the first appearance on the portion of the user interface displayed in the first mode; and In response to detecting the request: displaying the user interface of the application in the second mode via the display; and displaying, via the display, the affordance having a second appearance on the portion of the user interface displayed in the second mode, wherein: When the user interface of the application is displayed in the second mode, according to a second set of one or more rules different from the first set of one or more rules, the values ​​of the set of one or more display attributes of the enabling representation having the second appearance change according to changes in the values ​​of the set of one or more display attributes of the portion of the user interface located below the enabling representation.

2. The method according to claim 1, comprising: When displaying the affordance having the second appearance on the portion of the user interface displayed in the second mode after displaying the affordance on the portion of the user interface displayed in the first mode: ceasing to display the affordance on the user interface displayed in the second mode in response to determining that a fade-out criterion is satisfied; and Based on determining that the fade-out criteria are not met, the enable representation having the second appearance remains displayed on the user interface displayed in the second mode.

3. The method according to claim 1, wherein: The user interface of the application displayed in the first mode includes a representation of content that occupies less than all of the display area of ​​the display; and The user interface of the application displayed in the second mode includes a representation of content occupying all of the display area of ​​the display.

4. A method according to claim 1, wherein at least one of the first appearance and the second appearance of the enabling representation is based on an inversion of the portion of the user interface located below the enabling representation.

5. The method according to claim 4, wherein: the first set of one or more rules requiring application of a first inversion amount to the portion of the user interface underlying the affordance to obtain the first appearance of the affordance, The second set of one or more rules requires applying a second inversion amount to the portion of the user interface that is beneath the affordance to obtain the second appearance of the affordance, and The second reversal amount is smaller than the first reversal amount.

6. A method according to any one of claims 1 to 5, wherein at least one of the first appearance and the second appearance of the enabling representation is obtained by blurring the portion of the user interface located below the enabling representation.

7. The method according to claim 6, wherein: the first set of one or more rules requiring application of a first amount of blur to the portion of the user interface that is beneath the affordance to obtain the first appearance of the affordance, The second set of one or more rules requires applying a second amount of blur to the portion of the user interface that is beneath the affordance to obtain the second appearance of the affordance, and The second blur amount is smaller than the first blur amount.

8. A method according to any one of claims 1 to 5, wherein at least one of the first appearance and the second appearance of the enabling representation is obtained by desaturating the portion of the user interface located below the enabling representation.

9. The method according to claim 8, wherein: the first set of one or more rules requiring application of a first amount of desaturation to the portion of the user interface underlying the affordance to obtain the first appearance of the affordance, The second set of one or more rules requires applying a second amount of desaturation to the portion of the user interface underlying the affordance to obtain the second appearance of the affordance, and The second desaturation amount is less than the first desaturation amount.

10. The method according to any one of claims 2 to 5, comprising: while displaying the user interface in the second mode without displaying the affordance, detecting an input that satisfies a first affordance redisplay criterion, wherein the first affordance redisplay criterion is satisfied when the input is detected on the touch-sensitive surface; and In response to detecting the input that satisfies the affordance redisplay criteria, the affordance is redisplayed on the user interface displayed in the second mode.

11. The method according to any one of claims 2 to 5, comprising: When the user interface is displayed in the second mode without displaying the enable indication, detecting an input that satisfies a second enable indication redisplay criterion, wherein the second enable indication redisplay criterion is satisfied when the input is a request to change from displaying the user interface in the second mode to displaying the user interface in the first mode; as well as In response to detecting said input satisfying said second affordance redisplay criterion, redisplaying the user interface in the first mode; and The enable representation is redisplayed on the user interface redisplayed in the first mode.

12. A method according to any one of claims 1 to 5, wherein at least one of the first appearance and the second appearance of the enabling representation is dynamically adjusted based on changes occurring in the portion of the user interface located below the enabling representation.

13. The method according to any one of claims 1 to 5, wherein: generating the first appearance based on a first set of filters applied to the portion of the user interface underlying the affordance, generating the second appearance based on a second set of filters applied to the portion of the user interface beneath the affordance, and For two or more filters in the second set of filters, the first set of filters includes corresponding filters of the same type but with different tuning parameters.

14. The method according to claim 13, comprising: In response to detecting the request: generating one or more intermediate appearances between the first appearance and the second appearance for the affordance; as well as The one or more intermediate appearances of the affordance are displayed as a transition between displaying the affordance having the first appearance and displaying the affordance having the second appearance.

15. A method according to claim 14, wherein generating the one or more intermediate appearances between the first appearance and the second appearance for the enabling representation includes gradually transitioning from the first set of one or more rules to the second set of one or more rules.

16. A method according to any one of claims 1 to 5, wherein the enable representation having the first appearance has a first opacity, and the enable representation having the second appearance has a second opacity, the second opacity being less than the first opacity.

17. A method according to any one of claims 1 to 5, wherein the enabling representation having the first appearance and the enabling representation having the second appearance have the same size and position on the display.

18. A method according to any one of claims 1 to 5, wherein the visual distinction between the enabling representation and the user interface of the application formed by the first group of one or more rules is greater than the visual distinction between the enabling representation and the user interface of the application formed by the second group of one or more rules.

19. An electronic device comprising: monitor; one or more input devices; one or more processors; Memory; and One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors to perform the method according to any one of claims 1 to 18.

20. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions that, when executed by an electronic device having a display and one or more input devices, cause the electronic device to perform the method according to any one of claims 1 to 18.

Images