Secure login with authentication based on visual representation of data

Abstract

The invention relates to a secure login with authentication based on a visual representation of data. Techniques are described for managing secure login with authentication while viewing a unique code. In some examples, a requesting device displays a visual representation of data. An authentication device detects the presence of a visual representation of the data. The authentication device prompts a user to provide authorization information at the authentication device. The authentication device receives a set of one or more inputs. The authentication device transmits information that authorizes access to content on the requesting device.

Description

[0001] Divisional Application Statement

[0002] This application is a divisional application of a Chinese patent application with an application date of January 7, 2019, an invention title of "Secure Login with Authentication Based on Visual Representation of Data", and an application number of 201980009669.8.

[0003] Cross - Reference to Related Applications

[0004] This patent application claims priority to U.S. Provisional Patent Application Serial No. 62 / 620,422, filed on January 22, 2018, entitled "SECURE LOGIN WITH AUTHENTICATION BASED ON A VISUAL REPRESENTATION OF DATA", which is hereby incorporated by reference in its entirety for all purposes. Technical Field

[0005] This disclosure relates generally to computer user interfaces and, more particularly, to techniques for managing secure logins with authentication using visual representations of data (e.g., unique codes). Background Art

[0006] Techniques for logging into a website or service typically rely on a user manually typing a username and password into a text field. Some techniques for logging into a website or service include using another device during the login process. Summary of the Invention

[0007] However, some techniques for managing secure logins with authentication are often cumbersome and inefficient. For example, some prior art uses complex and time-consuming user interfaces that may include multiple button presses or keystrokes. Also, some prior art causes user confusion because the user does not know the current state of the login process. The prior art takes more time than necessary, which results in wasted user time and device energy. This latter consideration is particularly important in battery-powered devices.

[0008] Accordingly, the present technology provides faster and more efficient methods and interfaces for an electronic device to manage secure logins with authentication using visual representations of data (e.g., unique codes). Such methods and interfaces optionally supplement or replace other methods for managing secure logins with authentication. Such methods and interfaces reduce the cognitive burden on the user, improve security, and result in a more effective human-machine interface. For battery-powered computing devices, such methods and interfaces save power and increase the time between battery charges.

[0009] According to some embodiments, a method is described. The method includes: at a first electronic device having a display, one or more cameras, and one or more input devices: detecting, via a first camera of the one or more cameras, the presence of a visual representation of data displayed on a second electronic device; in response to detecting the presence of the visual representation of the data, prompting a user at the first electronic device to provide authorization information for enabling access to content on the second electronic device; after prompting the user to provide the authorization information, receiving a set of one or more inputs; and in response to receiving the set of one or more inputs: transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs includes authorization information that matches stored authorization information; and refraining from transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs does not include authorization information that matches stored authorization information.

[0010] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs executable by one or more processors of a first electronic device having a display, one or more cameras, and one or more input devices, the one or more programs including instructions for performing the following operations: detecting, via a first camera of the one or more cameras, the presence of a visual representation of data displayed on a second electronic device; in response to detecting the presence of the visual representation of the data, prompting a user at the first electronic device to provide authorization information for enabling access to content on the second electronic device; after prompting the user to provide the authorization information, receiving a set of one or more inputs; and in response to receiving the set of one or more inputs: transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs includes authorization information that matches stored authorization information; and refraining from transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs does not include authorization information that matches stored authorization information.

[0011] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs executable by one or more processors of a first electronic device having a display, one or more cameras, and one or more input devices, the one or more programs including instructions for performing the following operations: detecting, via a first camera of the one or more cameras, the presence of a visual representation of data displayed on a second electronic device; in response to detecting the presence of the visual representation of the data, prompting a user at the first electronic device to provide authorization information for enabling access to content on the second electronic device; after prompting the user to provide the authorization information, receiving a set of one or more inputs; and in response to receiving the set of one or more inputs: transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs includes authorization information that matches stored authorization information; and refraining from transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs does not include authorization information that matches stored authorization information.

[0012] According to some embodiments, a first electronic device is described. The first electronic device includes: a display, one or more cameras, one or more input devices, one or more processors, and a memory that stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: detecting, via a first camera of the one or more cameras, the presence of a visual representation of data displayed on a second electronic device; in response to detecting the presence of the visual representation of the data, prompting a user at the first electronic device to provide authorization information for enabling access to content on the second electronic device; after prompting the user to provide the authorization information, receiving a set of one or more inputs; and in response to receiving the set of one or more inputs: transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs includes authorization information that matches stored authorization information; and refraining from transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs does not include authorization information that matches stored authorization information.

[0013] According to some embodiments, a first electronic device is described. The first electronic device includes: a display, one or more cameras, one or more input devices, one or more processors, and means for detecting the presence of a visual representation of data displayed on a second electronic device via a first camera of the one or more cameras; means for prompting a user to provide authorization information at the first electronic device for enabling access to content on the second electronic device in response to detecting the presence of the visual representation of the data; means for receiving a set of one or more inputs after prompting the user to provide the authorization information; and means for performing the following operations in response to receiving the set of one or more inputs: transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs includes authorization information matching stored authorization information; and refraining from transmitting information authorizing access to the content on the second electronic device based on determining that the set of one or more inputs does not include authorization information matching stored authorization information.

[0014] According to some embodiments, a method is described. The method includes: at a first electronic device having a display: displaying a visual representation of data on the display of the first electronic device; receiving first information that the visual representation of the data has been detected by a second electronic device; in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting the user to provide authentication information at the second electronic device for enabling access to restricted content on the first electronic device; receiving second information authorizing access to the restricted content on the first electronic device; and in response to receiving the second information, providing access to the restricted content on the first electronic device.

[0015] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs executable by one or more processors of a first electronic device having a display, the one or more programs including instructions for performing the following operations: displaying a visual representation of data on the display of the first electronic device; receiving first information that the visual representation of the data has been detected by a second electronic device; in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting the user to provide authentication information at the second electronic device for enabling access to restricted content on the first electronic device; receiving second information authorizing access to the restricted content on the first electronic device; and in response to receiving the second information, providing access to the restricted content on the first electronic device.

[0016] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs executable by one or more processors of a first electronic device having a display, the one or more programs including instructions for performing the following operations: displaying a visual representation of data on the display of the first electronic device; receiving first information that the visual representation of the data has been detected by a second electronic device; in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting the user to provide authentication information at the second electronic device to enable access to restricted content on the first electronic device; receiving second information authorizing access to the restricted content on the first electronic device; and in response to receiving the second information, providing access to the restricted content on the first electronic device.

[0017] According to some embodiments, a first electronic device is described. The first electronic device includes: a display, one or more processors, and a memory that stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: displaying a visual representation of data on the display of the first electronic device; receiving first information that the visual representation of the data has been detected by a second electronic device; in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting the user to provide authentication information at the second electronic device to enable access to restricted content on the first electronic device; receiving second information authorizing access to the restricted content on the first electronic device; and in response to receiving the second information, providing access to the restricted content on the first electronic device.

[0018] According to some embodiments, a first electronic device is described. The first electronic device includes: a display, one or more processors, and means for displaying a visual representation of data on the display of the first electronic device; means for receiving first information that the visual representation of the data has been detected by a second electronic device; means for, in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting the user to provide authentication information at the second electronic device to enable access to restricted content on the first electronic device; means for receiving second information authorizing access to the restricted content on the first electronic device; and means for, in response to receiving the second information, providing access to the restricted content on the first electronic device.

[0019] The executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors. The executable instructions for performing these functions are optionally included in a transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0020] Accordingly, the device is provided with a faster and more efficient method and interface for using visual representations of data (e.g., unique codes) to manage authenticated secure logins, thereby enhancing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may supplement or replace other methods for using visual representations of data (e.g., unique codes) to manage authenticated secure logins. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which like reference numerals indicate corresponding parts in all the figures.

[0022] Figure 1A is a block diagram showing a portable multifunctional device having a touch-sensitive display in accordance with some embodiments.

[0023] Figure 1B is a block diagram showing exemplary components for event handling in accordance with some embodiments.

[0024] Figure 2 shows a portable multifunctional device having a touch screen in accordance with some embodiments.

[0025] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface in accordance with some embodiments.

[0026] Figure 4A shows an exemplary user interface for a menu of an application on a portable multifunctional device in accordance with some embodiments.

[0027] Figure 4B shows an exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display in accordance with some embodiments.

[0028] Figure 5A shows a personal electronic device in accordance with some embodiments.

[0029] Figure 5B is a block diagram showing a personal electronic device in accordance with some embodiments.

[0030] Figures 5C - 5DIllustrates exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor, according to some embodiments.

[0031] Figures 5E - 5H Illustrates exemplary components and user interfaces of a personal electronic device, according to some embodiments.

[0032] Figures 6A to 6U Illustrates an exemplary user interface for managing secure logins with authentication using a visual representation of data (e.g., a unique code).

[0033] Figure 7 Is a flowchart showing a method of using a visual representation of data (e.g., a unique code) to manage secure logins with authentication.

[0034] Figure 8 Is a flowchart showing a method of using a visual representation of data (e.g., a unique code) to manage secure logins with authentication. DETAILED DESCRIPTION

[0035] The following description sets forth exemplary methods, parameters, etc. However, it should be recognized that such description is not intended to limit the scope of the disclosure, but rather is provided as a description of exemplary embodiments.

[0036] There is a need to provide an electronic device for using a visual representation of data (e.g., a unique code) to manage secure logins with authentication. For example, a user may be attempting to log in to a website at a personal laptop computer. The user may seek a more efficient method of logging in to the website. Instead of manually typing in a username and password, the user proceeds with an option of an alternative login technique. The alternative login technique enables the user to log in by authenticating at the user's personal smart phone device. The personal laptop computer displays a QR code that can be detected via the camera of the personal smart phone device. Detection of the QR code initiates the process of the alternative login technique. The user's personal smart phone device is a trusted device that has been associated with the user's user account. The user can authenticate at the personal smart phone device, which enables the user to successfully log in to the website at the personal laptop computer. Such techniques can reduce the cognitive burden on the user of using a visual representation of data (e.g., a unique code) to use secure logins with authentication, thereby increasing productivity. In addition, such techniques can reduce the processor power and battery power otherwise wasted on redundant user input.

[0037] Below Figures 1A - 1B 、 Figure 2 、 Figure 3 、 Figures 4A - 4B and Figures 5A - 5H Provides a description of exemplary devices for performing techniques for managing event notifications. Figures 6A to 6UAn exemplary user interface for managing secure login with authentication while viewing a visual representation of data (e.g., a unique code) is shown. Figure 7 and Figure 8 is a flowchart showing a method of managing secure login with authentication using a visual representation of data (e.g., a unique code).

[0038] Although the following description uses terms such as "first" and "second" to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch could be named a second touch and similarly a second touch could be named a first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.

[0039] The terms used in the description of the various described embodiments herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the term "comprises" ("includes", "including", "comprises" and / or "comprising") when used in this specification is specifying the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0040] Depending on the context, the term "if" is optionally interpreted to mean "when", "upon", or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined..." or "if detected [the stated condition or event]" is optionally interpreted to mean "when determining..." or "in response to determining..." or "when detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]".

[0041] Embodiments of electronic devices, user interfaces for such devices, and related processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functions. Exemplary embodiments of the portable multifunctional device include, but are not limited to, those from Apple Inc. (Cupertino, California) Device, iPod Device, and Device. Optionally, other portable electronic devices are used, such as a laptop or tablet computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0042] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0043] The device generally supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, gaming applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0044] The various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports various applications with a user interface that is intuitive and clear to the user.

[0045] Attention is now turned to an embodiment of a portable device having a touch-sensitive display. Figure 1Ais a block diagram showing a portable multifunctional device 100 having a touch-sensitive display system 112 in accordance with some embodiments. The touch-sensitive display 112 is sometimes called a “touch screen” for convenience and is sometimes known as or referred to as a “touch-sensitive display system”. The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, a security component 115, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more contact intensity sensors 165 for detecting the intensity of contacts on the device 100 (e.g., a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100). The device 100 optionally includes one or more tactile output generators 167 for generating tactile outputs on the device 100 (e.g., generating tactile outputs on a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100 or the touchpad 355 of the device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0046] As used in this specification and the claims, the “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on the touch-sensitive surface, or to a surrogate for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values that includes at least four different values and more typically includes hundreds of different values (e.g., at least 256). The intensity of a contact is optionally determined (or measured) using a variety of methods and a variety of sensors or combinations of sensors. For example, one or more force sensors under or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in size of the contact area detected on the touch-sensitive surface, the capacitance and / or change in capacitance of the touch-sensitive surface near the contact, and / or the resistance and / or change in resistance of the touch-sensitive surface near the contact are optionally used as surrogates for the force or pressure of a contact on the touch-sensitive surface. In some embodiments, the surrogate measurements of contact force or pressure are directly used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measurements). In some embodiments, the surrogate measurements of contact force or pressure are converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of a contact as an attribute of user input allows a user to access additional device functions that would otherwise be inaccessible to the user on a smaller device with a limited physical area in some cases, the smaller device being used to (e.g., on a touch-sensitive display) display affordances and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls such as a knob or button).

[0047] As used in this specification and the claims, the term "haptic output" refers to a physical displacement of a device relative to a previous position of the device detected by a user using the user's sense of touch, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device. For example, in the case of contact between the device or a component of the device and a surface of the user sensitive to touch (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a sense of touch corresponding to a perceived change in a physical characteristic of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or a touchpad) is optionally interpreted by the user as a "press click" or "release click" of a physical actuation button. In some cases, the user will feel a sense of touch, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when there is no change in the smoothness of the touch-sensitive surface, movement of the touch-sensitive surface will optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such interpretations of touch by the user will be limited by the user's individual sensory perception, many sensory perceptions of touch are common to most users. Thus, when a haptic output is described as corresponding to a particular sensory perception of the user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated haptic output corresponds to a physical displacement of the device or a component thereof that would generate the described sensory perception of a typical (or ordinary) user.

[0048] It should be understood that device 100 is only one example of a portable multifunctional device, and device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0049] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0050] The peripheral device interface 118 can be used to couple the input and output peripheral devices of the device to the CPU 120 and the memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions of the device 100 and process data. In some embodiments, the peripheral device interface 118, the CPU 120, and the memory controller 122 are optionally implemented on a single chip such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0051] The RF (Radio Frequency) circuit 108 receives and transmits RF signals which are also referred to as electromagnetic signals. The RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with a communication network and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, a memory, and so on. The RF circuit 108 optionally communicates with the network and other devices via wireless communication, and these networks are such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular phone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The RF circuit 108 optionally includes well-known circuits for detecting a near field communication (NFC) field, such as detecting via a short-range communication radio component. The wireless communication optionally uses any one of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed Downlink Packet Access (HSDPA), High-Speed Uplink Packet Access (HSUPA), Evolution-Data Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long-Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), WiMAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols not yet developed as of the date of submission of this document.

[0052] The audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuitry 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into sound waves audible to humans. The audio circuitry 110 also receives the electrical signal converted from sound waves by the microphone 113. The audio circuitry 110 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuitry 108 by the peripheral interface 118. In some embodiments, the audio circuitry 110 also includes an earphone jack (e.g., Figure 2 212 in

[0053] ). The earphone jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device such as an output-only earphone or an earphone having both an output (e.g., a mono or stereo earphone) and an input (e.g., a microphone). Figure 2 Figure 2 Figure 2 Figure 2

[0054] The I / O subsystem 106 couples input / output peripheral devices on the device 100 such as the touch screen 112 and other input control devices 116 to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, a depth camera controller 169, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from / transmit electrical signals to other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller 160 is optionally coupled to (or not coupled to) any of the following: a keyboard, an infrared port, a USB port, and a pointing device such as a mouse. One or more buttons (e.g., Figure 2 Figure 2 Figure 2 Figure 2

[0054] Quickly pressing the depress button optionally disengages the lock of the touch screen 112 or optionally begins the process of unlocking the device using gestures on the touch screen, as described in U.S. Patent Application No. 11 / 322,549, filed on December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image" (i.e., U.S. Patent No. 7,657,849), which is hereby incorporated by reference in its entirety. Long pressing the depress button (e.g., 206) optionally powers on or powers off the device 100. The functionality of one or more buttons is optionally user-customizable. The touch screen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0055] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or sends electrical signals to the touch screen. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0056] The touch screen 112 has a touch-sensitive surface, sensor, or group of sensors that accepts input from the user based on haptic and / or tactile contact. The touch screen 112 and the display controller 156 (along with any associated modules and / or instruction sets in the memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into an interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

[0057] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, but uses other display technologies in other embodiments. The touch screen 112 and the display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof, the variety of touch sensing technologies including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in and iPod The technology used in

[0058] In some embodiments of the touchscreen 112, the touch-sensitive display is optionally similar to the multi-touch touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.) and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is hereby incorporated by reference in its entirety. However, the touchscreen 112 displays the visual output from the device 100, while the touch-sensitive touchpad does not provide a visual output.

[0059] In some embodiments, the touch-sensitive display of the touch screen 112 is as described in the following patent applications: (1) U.S. Patent Application No. 11 / 381,313, entitled "Multipoint Touch Surface Controller," filed on May 2, 2006; (2) U.S. Patent Application No. 10 / 840,862, entitled "Multipoint Touchscreen," filed on May 6, 2004; (3) U.S. Patent Application No. 10 / 903,964, entitled "Gestures For Touch Sensitive Input Devices," filed on July 30, 2004; (4) U.S. Patent Application No. 11 / 048,264, entitled "Gestures For Touch Sensitive Input Devices," filed on January 31, 2005; (5) U.S. Patent Application No. 11 / 038,590, entitled "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices," filed on January 18, 2005; (6) U.S. Patent Application No. 11 / 228,758, entitled "Virtual Input Device Placement On A Touch Screen User Interface," filed on September 16, 2005; (7) U.S. Patent Application No. 11 / 228,700, entitled "Operation Of A Computer With A Touch Screen Interface," filed on September 16, 2005; (8) U.S. Patent Application No. 11 / 228,737, entitled "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard," filed on September 16, 2005; and (9) U.S. Patent Application No. 11 / 367,749, entitled "Multi-Functional Hand-Held Device," filed on March 3, 2006. All of these applications are hereby incorporated by reference in their entirety.

[0060] The touch screen 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen has a video resolution of about 160 dpi. The user optionally uses any suitable object or attachment such as a stylus, finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily through finger-based contact and gestures, which may not be as precise as stylus-based input due to the larger contact area of the finger on the touch screen. In some embodiments, the device converts the finger-based rough input into an accurate pointer / cursor position or command for performing the action desired by the user.

[0061] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device, which, unlike the touch screen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touch screen 112 or an extension of the touch-sensitive surface formed by the touch screen.

[0062] The device 100 also includes a power system 162 for powering various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0063] The device 100 optionally further includes one or more optical sensors 164. Figure 1AAn optical sensor coupled to the optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In combination with the imaging module 143 (also referred to as the camera module), the optical sensor 164 optionally captures a still image or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite the touch screen display 112 on the front of the device, such that the touch screen display can be used as a viewfinder for still image and / or video image capture. In some embodiments, the optical sensor is located on the front of the device such that an image of the user can optionally be obtained for video conferencing while the user views other video conferencing participants on the touch screen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) such that a single optical sensor 164 can be used with the touch screen display for both video conferencing and still image and / or video image capture.

[0064] The device 100 optionally further includes one or more contact intensity sensors 165. Figure 1A A contact intensity sensor coupled to the intensity sensor controller 159 in the I / O subsystem 106 is shown. The contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). The contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., the touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the rear of the device 100, opposite the touch screen display 112 located on the front of the device 100.

[0065] The device 100 optionally further includes one or more proximity sensors 166. Figure 1AA proximity sensor 166 coupled to the peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 is optionally coupled to the input controller 160 in the I / O subsystem 106. The proximity sensor 166 optionally operates as described in the following U.S. patent applications: No. 11 / 241,839, titled "Proximity Detector In Handheld Device"; No. 11 / 240,788, titled "Proximity Detector In Handheld Device"; No. 11 / 620,702, titled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; No. 11 / 586,862, titled "Automated Response To And Sensing Of User Activity In Portable Devices"; and No. 11 / 638,251, titled "Methods And Systems For Automatic Configuration Of Peripherals", which U.S. patent applications are hereby incorporated by reference in their entirety. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call), the proximity sensor turns off and disables the touch screen 112.

[0066] Device 100 optionally further includes one or more haptic output generators 167. Figure 1AIllustrated is a haptic output generator coupled to a haptic feedback controller 161 in an I / O subsystem 106. The haptic output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components; and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert an electrical signal into a haptic output on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., a touch-sensitive display system 112), and optionally generates a haptic output by moving the touch-sensitive surface vertically (e.g., into / out of the surface of the device 100) or laterally (e.g., backward and forward in a plane that is the same as the surface of the device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite a touch screen display 112 located on the front of the device 100.

[0067] The device 100 optionally further includes one or more accelerometers 168. Figure 1A Illustrated is an accelerometer 168 coupled to a peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. The accelerometer 168 optionally operates as described in the following U.S. patent publications: U.S. Patent Publication No. 20050190059, titled "Acceleration-based Theft Detection System for Portable Electronic Devices" and U.S. Patent Publication No. 20060017692, titled "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer", both of which are hereby incorporated by reference in their entireties. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on an analysis of data received from one or more accelerometers. The device 100 optionally further includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to one or more accelerometers 168 for obtaining information about the location and orientation (e.g., portrait or landscape) of the device 100.

[0068] The device 100 optionally further includes one or more depth camera sensors 175. Figure 1AA depth camera sensor coupled to the depth camera controller 169 in the I / O subsystem 106 is shown. The depth camera sensor 175 receives data projected through the sensor from the environment. Optionally, in combination with the imaging module 143 (also referred to as the camera module), the depth camera sensor 175 is used to determine depth maps for different portions of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located in front of the device 100 such that user images with depth information are available for different functions of the device, such as video conferencing, capturing a selfie with depth map data, and authenticating a user of the device. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) such that the depth camera sensor 175 can be used with the touchscreen display for both video conferencing and static image and / or video image capture.

[0069] In some embodiments, the software components stored in the memory 102 include an operating system 126, a communication module (or set of instructions) 128, a touch / motion module (or set of instructions) 130, a graphics module (or set of instructions) 132, a text input module (or set of instructions) 134, a Global Positioning System (GPS) module (or set of instructions) 135, and application programs (or sets of instructions) 136. Additionally, in some embodiments, the memory 102 ( Figure 1A ) or 370 ( Figure 3 ) stores a device / global internal state 157, as Figure 1A and Figure 3 shown. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various regions of the touchscreen display 112; a sensor state, including information obtained from the various sensors and input control devices 116 of the device; and location information regarding the location and / or orientation of the device.

[0070] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware components and software components.

[0071] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, a wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with the 30-pin connector used on (a trademark of Apple Inc.) devices.

[0072] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or a physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of the contact point being represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., “multi-touch” / multiple finger contact). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0073] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determining whether the user has “clicked” an icon). In some embodiments, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the device 100). For example, the mouse “click” threshold of the touchpad or touch screen can be set to any one of a wide range of predefined thresholds without changing the touchpad or touch screen display hardware. Additionally, in some implementations, software settings are provided to the user of the device for adjusting one or more of the intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by using a system-level click on an “intensity” parameter to adjust multiple intensity thresholds at once).

[0074] The touch / motion module 130 optionally detects a user's gesture input. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Thus, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same position (or substantially the same position) as the finger press event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift (lift-off) event.

[0075] The graphics module 132 includes various known software components for presenting and displaying graphics on the touch screen 112 or other display, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual characteristics). As used herein, the term "graphics" includes any object that can be displayed to a user, which non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0076] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application, etc., and also receives coordinate data and other graphic attribute data as necessary, and then generates screen image data for output to the display controller 156.

[0077] The haptic feedback module 133 includes various software components for generating instructions that are used by the haptic output generator 167 to produce haptic output at one or more locations on the device 100 in response to a user's interaction with the device 100.

[0078] The text input module 134, which is optionally a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0079] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., provided to the phone 138 for location-based dialing; provided to the camera 143 as picture / video metadata; and provided to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0080] The application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0081] · A contacts module 137 (sometimes referred to as an address book or contacts list);

[0082] · A phone module 138;

[0083] · A video conferencing module 139;

[0084] · An email client module 140;

[0085] · An instant messaging (IM) module 141;

[0086] · A fitness support module 142;

[0087] · A camera module 143 for still images and / or video images;

[0088] · An image management module 144;

[0089] · A video player module;

[0090] · A music player module;

[0091] · A browser module 147;

[0092] · A calendar module 148;

[0093] · A desktop applet module 149, which optionally includes one or more of the following: a weather desktop applet 149-1, a stock market desktop applet 149-2, a calculator desktop applet 149-3, an alarm clock desktop applet 149-4, a dictionary desktop applet 149-5, and other desktop applets obtained by the user, as well as a user-created desktop applet 149-6;

[0094] · A desktop applet creator module 150 for forming the user-created desktop applet 149-6;

[0095] · A search module 151;

[0096] · A video and music player module 152, which combines the video player module and the music player module;

[0097] · A notepad module 153;

[0098] · A map module 154; and / or

[0099] · An online video module 155.

[0100] Examples of other applications 136 that are optionally stored in the memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, speech recognition, and speech reproduction.

[0101] In conjunction with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the contacts module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contacts module 137 stored in the memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating a phone number, email address, physical address, or other information with a name; associating an image with a name; categorizing and classifying names; providing a phone number or email address to initiate and / or facilitate communication via the phone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0102] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the phone module 138 is optionally used to input a character sequence corresponding to a phone number, access one or more phone numbers in the contacts module 137, modify an entered phone number, dial the corresponding phone number, conduct a session, and disconnect or hang up when the session is complete. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0103] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contacts module 137, and phone module 138, the video conferencing module 139 includes executable instructions to initiate, conduct, and terminate a video conference between the user and one or more other participants according to user instructions.

[0104] In conjunction with the RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the email client module 140 includes executable instructions to create, send, receive, and manage emails in response to user instructions. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send emails with static or video images captured by the camera module 143.

[0105] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying a previously input character, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving an instant message, and viewing the received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant message" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0106] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, the fitness support module 142 includes executable instructions for creating a fitness (e.g., having time, distance, and / or calorie burn goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for the fitness; and displaying, storing, and transmitting fitness data.

[0107] In combination with touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for: capturing still images or video (including video streams) and storing them in memory 102, modifying the characteristics of a still image or video, or deleting a still image or video from memory 102.

[0108] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0109] In combination with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, and linking to attachments and other files of web pages.

[0110] In combination with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and data associated with the calendars (e.g., calendar entries, to-do items, etc.) according to user instructions.

[0111] In combination with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, desktop widget module 149 is a mini-application (e.g., weather desktop widget 149-1, stock market desktop widget 149-2, calculator desktop widget 149-3, alarm clock desktop widget 149-4, and dictionary desktop widget 149-5) optionally downloaded and used by the user or a mini-application created by the user (e.g., user-created desktop widget 149-6). In some embodiments, the desktop widget includes HTML (HyperText Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop widget includes XML (eXtensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop widget).

[0112] In combination with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, desktop widget creator module 150 is optionally used by the user to create desktop widgets (e.g., transforming a user-specified portion of a web page into a desktop widget).

[0113] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, search module 151 includes executable instructions for searching memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0114] In combination with the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132, the audio circuit 110, the speaker 111, the RF circuit 108, and the browser module 147, the video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on the touch screen 112 or on an external display connected via the external port 124). In some embodiments, the device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0115] In combination with the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132, and the text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0116] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132, the text input module 134, the GPS module 135, and the browser module 147, the map module 154 is optionally used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data related to stores and other points of interest at or near a particular location, and other location-based data) according to user instructions.

[0117] In combination with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, e-mail client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing a user to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touch screen or on an external display connected via the external port 124), send an e-mail with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, the instant message module 141 is used instead of the e-mail client module 140 to send a link to a particular online video. Other descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed on Jun. 20, 2007, and U.S. Patent Application No. 11 / 968,067, entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed on Dec. 31, 2007, the contents of both of which are hereby incorporated by reference in their entireties.

[0118] Each of the above modules and applications corresponds to a set of executable instructions for performing one or more of the above functions and the methods described in this patent application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. For example, the video player module is optionally combined with the music player module into a single module (e.g., Figure 1A the video and music player module 152 in). In some embodiments, the memory 102 optionally stores a subset of the above modules and data structures. Additionally, the memory 102 optionally stores additional modules and data structures not described above.

[0119] In some embodiments, device 100 is a device in which the operation of a predefined set of functions on the device is performed exclusively via a touchscreen and / or a touchpad. By using the touchscreen and / or the touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on device 100 is optionally reduced.

[0120] The predefined set of functions performed exclusively via the touchscreen and / or the touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to a main menu, a home menu, or a root menu. In such embodiments, the touchpad is used to implement a "menu button". In some other embodiments, the menu button is a physical push button or other physical input control device rather than the touchpad.

[0121] Figure 1B is a block diagram showing exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in operating system 126) and corresponding application 136-1 (e.g., any one of the foregoing applications 137 to 151, 155, 380 to 390).

[0122] Event classifier 170 receives event information and determines the application 136-1 to which the event information is to be delivered and the application view 191 of application 136-1. Event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates one or more current application views displayed on the touch-sensitive display 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by event classifier 170 to determine which application(s) is / are currently active, and the application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0123] In some embodiments, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready to be displayed by application 136-1, a state queue for enabling a user to return to a previous state or view of application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0124] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (e.g., a user touch on the touch-sensitive display 112, as part of a multi-touch gesture). The peripheral interface 118 transmits information that it receives from the I / O subsystem 106 or sensors such as the proximity sensor 166, one or more accelerometers 168, and / or the microphone 113 (via the audio circuitry 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display 112 or a touch-sensitive surface.

[0125] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when there is a significant event (e.g., a received input that is above a predetermined noise threshold and / or a received input that exceeds a predetermined duration).

[0126] In some embodiments, the event classifier 170 further includes a hit view determination module 172 and / or an active event recognizer determination module 173.

[0127] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where within one or more of the views a sub-event has occurred. Views are composed of controls and other elements that a user can see on the display.

[0128] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct inputs is optionally determined at least in part based on the hit view of the initial touch that begins the touch-based gesture.

[0129] The hit view determination module 172 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest view in the hierarchical structure that should handle the sub-event. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or a potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view generally receives all sub-events related to the same touch or input source for which it is identified as the hit view.

[0130] The active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views and, thus, determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with a particular view, higher views in the hierarchy will still remain as actively participating views.

[0131] The event dispatcher module 174 distributes event information to event recognizers (e.g., event recognizer 180). In embodiments that include the active event recognizer determination module 173, the event dispatcher module 174 delivers the event information to the event recognizer determined by the active event recognizer determination module 173. In some embodiments, the event dispatcher module 174 stores the event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0132] In some embodiments, the operating system 126 includes the event classifier 170. Alternatively, the application 136-1 includes the event classifier 170. In yet another embodiment, the event classifier 170 is an independent module or part of another module (such as the contact / motion module 130) stored in the memory 102.

[0133] In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of an independent module that is a higher-level object such as a user interface toolkit (not shown) or from which application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handlers 190 include one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handlers 190 optionally utilize or invoke data updater 176, object updater 177, or GUI updater 178 to update the application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included within the corresponding application view 191.

[0134] The corresponding event recognizer 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0135] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events such as a touch or a touch movement. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device's attitude).

[0136] Event comparator 184 compares the event information with predefined event or sub - event definitions and determines an event or sub - event based on this comparison, or determines or updates the status of an event or sub - event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains the definition of an event (e.g., a predefined sequence of sub - events), such as event 1 (187 - 1), event 2 (187 - 2), and others. In some embodiments, the sub - events in an event (187) include, for example, touch start, touch end, touch move, touch cancel, and multi - touch. In one example, the definition of event 1 (187 - 1) is a double - tap on a displayed object. For example, a double - tap includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift - off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on the displayed object, and a second lift - off (touch end) of a predetermined duration. In another example, the definition of event 2 (187 - 2) is a drag on a displayed object. For example, a drag includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch - sensitive display 112, and lift - off of the touch (touch end). In some embodiments, an event also includes information for one or more associated event handlers 190.

[0137] In some embodiments, event definition 187 includes the definition of an event for a corresponding user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub - event. For example, in an application view that displays three user interface objects on touch - sensitive display 112, when a touch is detected on touch - sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub - event). If each displayed object is associated with a corresponding event handler 190, event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub - event and the object that triggered the hit test.

[0138] In some embodiments, the definition of a corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub - event sequence does or does not correspond to the event type of the event recognizer.

[0139] When the corresponding event recognizer 180 determines that the sub - event sequence does not match any event in the event definition 186, the corresponding event recognizer 180 enters an event - impossible, event - failed, or event - ended state, after which subsequent sub - events of the touch - based gesture are ignored. In such a case, other event recognizers (if any) that remain active for the hit view continue to track and process the ongoing sub - events of the touch - based gesture.

[0140] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists indicating how the event delivery system should perform sub - event delivery to the active participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists indicating how event recognizers interact with each other or can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists indicating whether sub - events are delivered to different levels in the view or the programmatic hierarchy.

[0141] In some embodiments, when one or more specific sub - events of an event are recognized, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferring sending) sub - events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a token associated with the recognized event, and the event handler 190 associated with the token obtains the token and executes a predefined process.

[0142] In some embodiments, the event delivery instruction 188 includes a sub - event delivery instruction that delivers event information about the sub - event without activating the event handler. Instead, the sub - event delivery instruction delivers the event information to the event handler associated with the sub - event sequence or to the active participating view. The event handler associated with the sub - event sequence or with the active participating view receives the event information and executes a predetermined process.

[0143] In some embodiments, the data updater 176 creates and updates data used in the application 136-1. For example, the data updater 176 updates the phone numbers used in the contact module 137, or stores the video files used in the video player module. In some embodiments, the object updater 177 creates and updates objects used in the application 136-1. For example, the object updater 177 creates new user interface objects or updates the positions of user interface objects. The GUI updater 178 updates the GUI. For example, the GUI updater 178 prepares the display information and sends the display information to the graphics module 132 for display on the touch-sensitive display.

[0144] In some embodiments, the event handler 190 includes or has access to the data updater 176, the object updater 177, and the GUI updater 178. In some embodiments, the data updater 176, the object updater 177, and the GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0145] It should be understood that the above discussion of event handling for user touches on the touch-sensitive display also applies to other forms of user input for operating the multifunctional device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in cooperation with single or multiple keyboard presses or holds; contact movement on a touchpad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; voice commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as inputs for sub-events corresponding to the events to be recognized.

[0146] Figure 2FIG. 0 shows a portable multifunctional device 100 having a touch screen 112, according to some embodiments. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In this and other embodiments described below, the user is able to select one or more of these graphics by making gestures on the graphics, such as by using one or more fingers 202 (not drawn to scale in the figure) or one or more styli 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gestures optionally include one or more taps, one or more swipes (from left to right, right to left, up, and / or down), and / or rolling of a finger that has made contact with the device 100 (from right to left, left to right, up, and / or down). In some implementations or in some cases, inadvertently contacting a graphic does not select the graphic. For example, when the gesture corresponding to selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0147] The device 100 optionally further includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any of a set of applications optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.

[0148] In some embodiments, the device 100 includes a touch screen 112, a menu button 204, a depressible button 206 for powering the device on / off and for locking the device, one or more volume adjustment buttons 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The depressible button 206 is optionally used to power the device on / off by depressing the button and holding the button in the depressed state for a predefined time interval; to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also accepts voice input for activating or deactivating certain functions via a microphone 113. The device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch screen 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0149] Figure 3FIG. 0 is a block diagram of an exemplary multi-functional device having a display and a touch-sensitive surface, in accordance with some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a gaming system, or a control device (e.g., a home or industrial controller). Device 300 generally includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes termed a chipset) that interconnects system components and controls the communication between them. Device 300 includes an input / output (I / O) interface 330 having a display 340, which is typically a touch screen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to one or more of the haptic output generators 167 described above with reference to Figure 1A ), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and / or contact intensity sensors (similar to one or more of the contact intensity sensors 165 described above with reference to Figure 1A ). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, those stored in memory 102 of portable multi-functional device 100 ( Figure 1A ). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multi-functional device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a web site creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while memory 102 of portable multi-functional device 100 ( Figure 1A ) optionally does not store these modules.

[0150] Figure 3Each of the above - mentioned elements in [the above - mentioned] is optionally stored in one or more memory devices of the previously mentioned memory device. Each of the above - mentioned modules corresponds to an instruction set for performing the above - mentioned functions. The above - mentioned modules or programs (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, the memory 370 optionally stores a subset of the above - mentioned modules and data structures. In addition, the memory 370 optionally stores additional modules and data structures not described above.

[0151] Attention is now turned to an embodiment of a user interface optionally implemented on, for example, the portable multifunctional device 100.

[0152] Figure 4A An exemplary user interface of an application menu on the portable multifunctional device 100 according to some embodiments is shown. A similar user interface is optionally implemented on the device 300. In some embodiments, the user interface 400 includes the following elements or a subset or superset thereof:

[0153] · One or more signal strength indicators 402 for one or more wireless communications such as cellular signals and Wi - Fi signals;

[0154] · Time 404;

[0155] · Bluetooth indicator 405;

[0156] · Battery status indicator 406;

[0157] · A tray 408 with icons for common applications, such as:

[0158] ○ An icon 416 marked "Phone" for the phone module 138, and the icon 416 optionally includes an indicator 414 of the number of missed calls or voicemails;

[0159] ○ An icon 418 marked "Mail" for the email client module 140, and the icon 418 optionally includes an indicator 410 of the number of unread emails;

[0160] ○ An icon 420 marked "Browser" for the browser module 147; and

[0161] ○ An icon 422 marked "iPod" for the video and music player module 152 (also known as the iPod (trademark of Apple Inc.)

[0162] module 152); and

[0163] · Icons for other applications, such as:

[0164] ○ The icon 424 of the IM module 141 marked as "Message";

[0165] ○ The icon 426 of the calendar module 148 marked as "Calendar";

[0166] ○ The icon 428 of the image management module 144 marked as "Photo";

[0167] ○ The icon 430 of the camera module 143 marked as "Camera";

[0168] ○ The icon 432 of the online video module 155 marked as "Online Video";

[0169] ○ The icon 434 of the stock market desktop applet 149-2 marked as "Stock Market";

[0170] ○ The icon 436 of the map module 154 marked as "Map";

[0171] ○ The icon 438 of the weather desktop applet 149-1 marked as "Weather";

[0172] ○ The icon 440 of the alarm clock desktop applet 149-4 marked as "Clock";

[0173] ○ The icon 442 of the fitness support module 142 marked as "Fitness Support";

[0174] ○ The icon 444 of the notepad module 153 marked as "Notepad"; and

[0175] ○ The icon 446 of the settings application or module marked as "Settings", which provides access to the settings of the device 100 and its various applications 136.

[0176] It should be noted that Figure 4A The icon labels shown are merely exemplary. For example, the icon 422 of the video and music player module 152 is marked "Music" or "Music Player". Other labels may optionally be used for the various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0177] Figure 4B A device (e.g., Figure 3 is shown having a touch-sensitive surface 451 (e.g., Figure 3Exemplary user interface on device 300). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of a contact on the touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating a tactile output for a user of device 300.

[0178] Although some of the following examples will be given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface separate from the display, as Figure 4B shown. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451 in ) has a major axis (e.g., Figure 4B 452 in ) corresponding to the major axis (e.g., Figure 4B 453 in ) on the display (e.g., 450). According to these embodiments, the device detects contact (e.g., Figure 4B 460 and 462 in ) with the touch-sensitive surface 451 at a location corresponding to a respective location on the display (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, when the touch-sensitive surface (e.g., Figure 4B 451 in ) is separate from the display (e.g., Figure 4B 450 in ) of the multifunctional device, user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0179] Additionally, although the following examples are mainly given with reference to finger input (e.g., finger contact, single-finger tap gesture, finger swipe gesture), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of a contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is above the location of the tap gesture (e.g., instead of detecting a contact, followed by ceasing to detect the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and a finger contact are optionally used simultaneously.

[0180] Figure 5AAn exemplary personal electronic device 500 is shown. The device 500 includes a body 502. In some embodiments, the device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., Figures 1A to 4B ). In some embodiments, the device 500 has a touch-sensitive display screen 504 hereinafter referred to as a touch screen 504. As an alternative or addition to the touch screen 504, the device 500 has a display and a touch-sensitive surface. As in the case of devices 100 and 300, in some embodiments, the touch screen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). One or more intensity sensors of the touch screen 504 (or touch-sensitive surface) may provide output data representative of the intensity of the touch. The user interface of the device 500 may respond to the touch based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0181] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application," published as WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships," published as WIPO Patent Publication No. WO / 2014 / 105276, each of which is hereby incorporated by reference in its entirety.

[0182] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) can be in physical form. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) may allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watchbands, bracelets, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow the user to wear device 500.

[0183] Figure 5B An exemplary personal electronic device 500 is shown. In some embodiments, device 500 may include some or all of the components referred to in Figure 1A , Figure 1B and Figure 3 Device 500 has a bus 512 that operatively couples the I / O section 514 to one or more computer processors 516 and a memory 518. The I / O section 514 may be connected to a display 504, which may have a touch-sensitive component 522 and optionally a force sensor 524 (e.g., a contact force sensor). Additionally, the I / O section 514 may be connected to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular, and / or other wireless communication technologies. Device 500 may include an input mechanism 506 and / or 508. For example, input mechanism 506 is optionally a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 is optionally a button.

[0184] In some examples, input mechanism 508 is optionally a microphone. Personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or a combination thereof, all of which are operatively connected to the I / O section 514.

[0185] The memory 518 of personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, may cause the computer processors to perform the techniques described below, including process 700 ( Figure 7)。A computer-readable storage medium can be any medium that tangibly contains or stores computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transient computer-readable storage medium. Non-transient computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, etc. The personal electronic device 500 is not limited to Figure 5B the components and configurations thereof, but may include other components or additional components in a variety of configurations.

[0186] As used herein, the term "enabling representation" refers to a user-interactive graphical user interface object optionally displayed on the display screen of devices 100, 300, and / or 500 ( Figure 1A 、 Figure 3 and Figures 5A to 5B ). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute an enabling representation.

[0187] As used herein, the term "focus selector" refers to an input element for indicating the current part of the user interface with which the user is interacting. In some specific implementations including a cursor or other position marker, the cursor acts as the "focus selector" such that when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., Figure 3 the touchpad 355 in Figure 4B ) or Figure 1A the touch-sensitive surface 451 in Figure 4AIn some specific implementations of the touch screen 112), the detected contact on the touch screen acts as a "focus selector", such that when an input (e.g., a press input made by the contact) is detected at the position of a specific user interface element (e.g., a button, a window, a slider, or other user interface element) on the touch screen display, the specific user interface element is adjusted according to the detected input. In some specific implementations, the focus moves from one area of the user interface to another area of the user interface without a corresponding movement of the cursor or a movement of the contact on the touch screen display (e.g., moving the focus from one button to another button by using the tab key or arrow keys); in these specific implementations, the focus selector moves according to the movement of the focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is typically a user interface element (or a contact on the touch screen display) that is controlled by the user in order to deliver the interaction with the user interface that the user anticipates (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or a touch screen), the position of the focus selector (e.g., a cursor, a contact, or a selection box) above the corresponding button will indicate that the user desires to activate the corresponding button (as opposed to other user interface elements shown on the device display).

[0188] As used in the specification and claims, the term "feature intensity" of a contact refers to a feature of the contact based on one or more intensities of the contact. In some embodiments, the feature intensity is based on a plurality of intensity samples. The feature intensity is optionally based on a predefined number or set of intensity samples collected during a predefined period of time (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after detecting a contact, before detecting a lift-off of the contact, before or after detecting a start of movement of the contact, before detecting an end of the contact, before or after detecting an increase in the intensity of the contact, and / or before or after detecting a decrease in the intensity of the contact). The feature intensity of a contact is optionally based on one or more of the following: the maximum value of the intensity of the contact, the mean value of the intensity of the contact, the average value of the intensity of the contact, the value at the top 10% of the intensity of the contact, the half-maximum value of the intensity of the contact, the 90% maximum value of the intensity of the contact, etc. In some embodiments, the duration of the contact is used in determining the feature intensity (e.g., when the feature intensity is the average value of the intensity of the contact over time). In some embodiments, the feature intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an operation. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact with a feature intensity not exceeding the first threshold results in a first operation, a contact with a feature intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second operation, and a contact with a feature intensity exceeding the second threshold results in a third operation. In some embodiments, the comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding operation or to forgo performing the corresponding operation) rather than for determining whether to perform a first operation or a second operation.

[0189] Figure 5C Illustrated is the detection of a plurality of contacts 552A - 552E on a touch-sensitive display screen 504 using a plurality of intensity sensors 524A - 524D. Figure 5C Also included is an intensity map that shows the current intensity measurements of the intensity sensors 524A - 524D relative to intensity units. In this embodiment, the intensity measurements of both intensity sensors 524A and 524D are 9 intensity units, and the intensity measurements of both intensity sensors 524B and 524C are 7 intensity units. In some specific implementations, the cumulative intensity is the sum of the intensity measurements of the plurality of intensity sensors 524A - 524D, which is 32 intensity units in this embodiment. In some embodiments, each contact is assigned a corresponding intensity, i.e., a portion of the cumulative intensity. Figure 5DShows the assignment of cumulative intensity to contacts 552A - 552E based on their distance from the center of force 554. In this embodiment, each of contacts 552A, 552B, and 552E is assigned an intensity of 8 intensity units of cumulative intensity, and each of contacts 552C and 552D is assigned an intensity of 4 intensity units of cumulative intensity. More generally, in some embodiments, each contact j is assigned a corresponding intensity Ij according to a predefined mathematical function Ij = A·(Dj / ΣDi), which is a part of the cumulative intensity A, where Dj is the distance of the corresponding contact j from the center of force, and ΣDi is the sum of the distances of all corresponding contacts (e.g., i = 1 to the last) from the center of force. An electronic device similar to or equivalent to devices 100, 300, or 500 can perform the operations referred to Figures 5C - 5D above. In some embodiments, the characteristic intensity of a contact is based on one or more intensities of the contact. In some embodiments, an intensity sensor is used to determine a single characteristic intensity (e.g., a single characteristic intensity of a single contact). It should be noted that the intensity map is not part of the displayed user interface but is included in Figures 5C - 5D to assist the reader.

[0190] In some embodiments, a part of the recognized gesture is used to determine the characteristic intensity. For example, the touch-sensitive surface optionally receives a continuous swipe contact that transitions from a starting position and reaches an ending position where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a part of the continuous swipe contact, rather than the entire swipe contact (e.g., only the part of the swipe contact at the ending position). In some embodiments, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact for the purpose of determining the characteristic intensity.

[0191] Optionally, characterize the contact intensity on the touch-sensitive surface relative to one or more intensity thresholds such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, when a contact is detected with a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold, contacts below the nominal contact detection intensity threshold are no longer detected), the device will move the focus selector based on the movement of the contact on the touch-sensitive surface without performing an operation associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different sets of user interface figures.

[0192] An increase in the contact characteristic intensity from an intensity below the light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in the contact characteristic intensity from an intensity below the deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in the contact characteristic intensity from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in the contact characteristic intensity from an intensity above the contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a lift-off of the contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0193] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or contacts), where the corresponding press input is detected at least in part based on the detected intensity of the contact (or contacts) increasing above a press input intensity threshold. In some embodiments, a corresponding operation is performed in response to detecting an increase in the intensity of the corresponding contact above the press input intensity threshold (e.g., the "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of the contact below the press input intensity threshold, and a corresponding operation is performed in response to detecting a subsequent decrease in the intensity of the corresponding contact below the press input threshold (e.g., the "up stroke" of the corresponding press input).

[0194] Figures 5E - 5HDetection of a gesture is shown, the gesture comprising contact 562 with an intensity from below Figure 5E The light press intensity threshold in L ”) increases to a strength higher than Figure 5H The deep compression intensity threshold in D ”). On the displayed user interface 570 including application icons 572A-572D displayed in the predefined area 574, when the cursor 576 is displayed above the application icon 572B corresponding to application 2, a gesture performed using contact 562 is detected on the touch-sensitive surface 560. In some embodiments, the gesture is detected on the touch-sensitive display 504. The intensity sensor detects the intensity of the contact on the touch-sensitive surface 560. The device determines whether the intensity of the contact 562 is within the deep press intensity threshold (for example, “IT D ”) reaches a peak above. Contact 562 is maintained on touch-sensitive surface 560. In response to detecting the gesture, and according to the intensity increasing to a deep press intensity threshold (e.g., “IT D ”) above contact 562, displays scaled representations 578A-578C (e.g., thumbnails) of documents recently opened for application 2, such as Figures 5F - 5H In some embodiments, the intensity is a characteristic intensity of the contact compared to one or more intensity thresholds. It should be noted that the intensity map for contact 562 is not part of the displayed user interface, but is included in the Figures 5E - 5H To assist the readers.

[0195] In some embodiments, the display of representations 578A-578C includes animation. For example, representation 578A is initially displayed near application icon 572B, such as Figure 5F As the animation progresses, representation 578A moves upward and representation 578B is displayed near application icon 572B, as shown. Figure 5G Then, representation 578A moves upward, 578B moves upward toward representation 578A, and representation 578C is displayed near application icon 572B, as shown in FIG. Figure 5H 578A-578C form an array above icon 572B. In some embodiments, the animation progresses according to the intensity of contact 562, such as Figures 5F - 5G , where representations 578A-578C appear and increase as the strength of contact 562 approaches a deep press strength threshold (eg, “IT D ”) increases and moves upward. In some embodiments, the intensity according to which the animation progresses is the characteristic intensity of the contact. The reference may be performed using an electronic device similar to or equivalent to device 100, 300, or 500. Figures 5E - 5H The operation described.

[0196] In some embodiments, the device employs hysteresis of intensity to avoid unexpected inputs sometimes referred to as “jitter,” where the device defines or selects a hysteresis intensity threshold having a predefined relationship with the press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of that contact to below the hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the hysteresis intensity threshold (e.g., the “upstroke” of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects an increase in contact intensity from an intensity equal to or below the hysteresis intensity threshold to an intensity equal to or above the press input intensity threshold and optionally the contact intensity subsequently decreases to an intensity equal to or below the hysteresis intensity, and a corresponding operation is performed in response to detecting the press input (e.g., depending on the context, an increase in contact intensity or a decrease in contact intensity).

[0197] For ease of explanation, optionally, a description of an operation performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input is triggered in response to detecting any one of the following various situations: an increase in contact intensity to above the press input intensity threshold, an increase in contact intensity from an intensity below the hysteresis intensity threshold to an intensity above the press input intensity threshold, a decrease in contact intensity to below the press input intensity threshold, and / or a decrease in contact intensity to below the hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in an example where an operation is described as being performed in response to detecting a decrease in the intensity of a contact to below the press input intensity threshold, the operation is optionally performed in response to detecting a decrease in the intensity of the contact to below the hysteresis intensity threshold corresponding to and less than the press input intensity threshold.

[0198] As used herein, an “installed application” refers to a software application that has been downloaded onto an electronic device (e.g., devices 100, 300, and / or 500) and is ready to be launched (e.g., become open) on the device. In some embodiments, a downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the operating system of the computer system.

[0199] Attention is now turned to embodiments of a user interface (“UI”) implemented on an electronic device such as the portable multifunctional device 100, device 300, or device 500, and associated processes.

[0200] Figures 6A to 6UAn exemplary user interface for managing secure logins with authentication using a visual representation of data (e.g., a unique code) in accordance with some embodiments is shown. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 7 to 8 which are described below.

[0201] Figure 6A An authentication device 600 (e.g., device 100, device 300, or device 500) and a requesting device 602 (e.g., device 100, device 300, or device 500) are shown. In the Figures 6A to 6U non-limiting exemplary embodiment shown, the authentication device 600 is a smart phone and the requesting device 602 is a personal laptop computer. In some examples, the authentication device 600 and / or the requesting device 602 may be different types of electronic devices, such as wearable devices (e.g., smart watches). The authentication device 600 includes a touch screen display 601 and one or more input devices (e.g., the touch component of the display 601, hardware buttons 607, and one or more biometric sensors (e.g., a face recognition sensor 603 or a fingerprint sensor 611)). In some examples, the face recognition sensor 603 may include a camera, such as an infrared camera, a thermal imaging camera, or a combination thereof. In some examples, the authentication device 600 further includes a lighting device (e.g., a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The lighting device is optionally used to illuminate a biometric feature (e.g., a face) during capture of biometric data of the biometric feature by the one or more biometric sensors. In some examples, the fingerprint sensor 611 is integrated into a physical button (e.g., the home button of the authentication device 600). The requesting device 602 includes a display 605 and one or more input devices (e.g., a keyboard).

[0202] In Figure 6A , a user of the device 602 wishes to access content associated with the website CLOUD.COM, which requires authentication before providing access to the content. The requesting device 602 displays a login web page 606 having a user name field 608 and a password field 610 for logging into the website (e.g., CLOUD.COM) on the display 605. The user can log into the website by entering valid credentials (e.g., via a keyboard) into the user name field 608 and the password field 610. The requesting device 602 displays an enabling representation 614 on the display 605, which, when activated, causes a transition from displaying the login web page 606 to displaying Figure 6B an alternative login web page 618. The requesting device 602 receives an input 612 (e.g., a mouse click event) at the location of the enabling representation 614. In some examples, the user selects the enabling representation 614 via a tap gesture on the display 605 (e.g., a touch screen display).

[0203] In some examples, when the enabling representation 614 is activated, the requesting device 602 receives data (e.g., a cookie) from a website. When the user later attempts to log in to the same website, this data indicates to the web browser to load an alternative login web page 618. In some examples, after receiving data from the website, when the requesting device 602 receives a request to log in to the website, it displays the alternative login web page 618 on the display 605. In some examples, the alternative login web page 618 is displayed instead of the login web page 606.

[0204] As Figure 6A shown, the authentication device 600 displays a home screen 604 including a camera icon 616 on the display 601. When the camera icon 616 is activated, it causes the authentication device 600 to transition from displaying the home screen 604 to launching a camera application. As Figure 6A shown, the authentication device 600 is not being used to access (or request access to) content associated with the website CLOUD.COM.

[0205] At Figure 6B , when the enabling representation 614 of Figure 6A is activated, the requesting device 602 displays an alternative login web page 618 for logging in to the website on the display 605. The alternative login web page 618 enables the user to log in to the website at the requesting device 602 by authenticating the user at the authentication device 600. The user can log in to the website without manually entering user credentials into text fields (e.g., the username field 608 and / or the password field 610). Displaying the alternative login web page 618 includes displaying a computer-readable data representation such as a QR (Quick Response) code 620 and / or user instructions 622. The QR code 620 is a non-text code that represents machine-readable data by encoding the machine-readable data in two dimensions (e.g., along a horizontal direction and a vertical direction). As Figure 6B shown, the QR code 620 is composed of many small viewpoints that are arranged in a specific configuration to form the QR code 620. In some examples, the QR code 620 includes encoded information representing one or more of a Uniform Resource Locator (URL), a session identifier (ID), and a key (e.g., a shared key) or associated with one or more of them. The authentication device 600 can use the URL and / or the session ID to identify the web browser session and / or the website that the user at the requesting device 602 wishes to log in to. The authentication device 600 optionally uses the key as a shared key between the authentication device 600 and the web browser of the requesting device 602. In some examples, the shared key enables the authentication device 600 and the requesting device 602 to (directly or indirectly) transmit data, and an intermediate device (e.g., a server) cannot identify the data being transmitted. The user instructions 622 include text that prompts the user to take an action at the authentication device 600.

[0206] In Figure 6B when the main screen 604 is being displayed, the authentication device 600 receives an input 609 (e.g., a tap gesture) via the touch screen of the display 601 at the location of the camera icon 616. The input 609 at the location of the camera icon 616 causes the activation (or selection) of the camera icon 616.

[0207] In Figure 6C when the activation of the camera icon 616 is detected, the authentication device 600 launches the camera application. Launching the camera application includes stopping the display of the main screen 604 and transitioning to display the user interface 624 of the camera application on the display 601. Displaying the user interface 624 of the camera application includes displaying a representation of at least a portion of the data captured by one or more cameras of the authentication device 600. The authentication device 600 is initially oriented such that the one or more cameras of the authentication device 600 do not fully point at the QR code 620. In this orientation, the authentication device 600 cannot recognize the QR code 620 because a large portion of the QR code 620 is not captured by the one or more cameras of the authentication device 600. In some examples, if the QR code 620 is below a threshold size, the device also does not recognize the QR code or does not react to having recognized the QR code to avoid accidentally prompting the user to perform an operation associated with the QR code when it is not clear whether the user is orienting the camera towards the QR code (e.g., by placing the QR code in the view of the camera at a size greater than the threshold).

[0208] In Figure 6D the user reorients the authentication device 600 such that the QR code 620 is fully within the field of view of the one or more cameras of the authentication device 600. In this new orientation, the authentication device 100 detects (or recognizes) the QR code 620 displayed on the display 605 of the requesting device 602. Detecting the QR code 620 includes capturing the data represented by the QR code 620 via the one or more cameras of the authentication device 600. Detecting the QR code 620 optionally includes interpreting the data represented by the QR code 620. As Figures 6A to 6U shown, in response to detecting the QR code 620, the authentication device 600 proceeds to display Figure 6E the banner 626. In some examples, upon detecting the QR code 620, the authentication device 600 proceeds to display Figure 6H the biometric authentication interface 634 without displaying Figure 6E the banner 626.

[0209] In Figure 6EIn [text not provided], when the QR code 620 is detected, the authentication device 600 displays a banner 626 (e.g., a user interface element) on the display 601. In some examples, displaying the banner 626 includes displaying an animation of the banner 626 such that the banner 626 slides from the top edge of the display 601 of the authentication device 600 into the display (e.g., into the view on the display). The banner 626 includes text prompting the user to log in to a user account associated with the user. In some examples, the banner 626 includes replacing (or obscuring) multiple characters of the text associated with the user's user account, as Figure 6E shown. In some examples, the authentication device 600 stops displaying the banner 626 after a pre-determined amount of inactivity (e.g., the authentication device 600 does not receive input (e.g., a tap gesture) within a pre-determined amount of time).

[0210] At Figure 6F , the authentication device 600 receives an input 625 (e.g., a tap gesture) at the location of the banner 626 via the touch screen of the display 601.

[0211] At Figure 6G , when a selection of the banner 626 is received, the authentication device 600 stops displaying the banner 626. Additionally, when a selection of the banner 626 is received, the authentication device 600 transmits data to a remote device (e.g., the requesting device 602 or a server communicating with the requesting device 602). The data includes data regarding the types of authentication available for the user to authenticate the user at the authentication device 600. For example, some types of authentication include biometric authentication via a biometric sensor (e.g., a face recognition sensor 603 or a fingerprint sensor 611) or non-biometric authentication (e.g., a passphrase, password, or gesture input) via a displayed character input interface (e.g., a virtual keyboard or a virtual numeric keypad). In some examples, the data includes data indicating that the authentication device 600 has detected the QR code 620. In some examples, the data includes data indicating that the authentication device 600 has received a selection of the banner 626.

[0212] In response to the authentication device 600 transmitting data to the remote device, the requesting device 602 receives data regarding the types of authentication available for the user to authenticate the user at the authentication device 600. When the data regarding the type of authentication is received (or when the authentication device 600 receives a selection of the banner 626), the requesting device 602 displays an animation 628 that shows a transition from Figure 6F the QR code 620 to Figure 6H the icon 632. Figures 6F to 6H The animation 628 shown includes rearranging the dots that include the QR code 620 into a new configuration. As Figure 6GAs shown, the animation 628 has started and the dots including the QR code 620 are being rearranged. Additionally, when receiving data regarding the authentication type, the requesting device 602 displays a user instruction 630 on the display 605. The user instruction 630 includes text prompting the user to take an action at the authentication device 600. In some examples, in response to the authentication device 600 detecting the QR code 620, the requesting device 602 displays the animation 628.

[0213] In some examples, the requesting device 602 receives data directly from the authentication device 600. In some examples, the requesting device 602 receives data from an intermediate server that communicates with the authentication device 600.

[0214] At Figure 6H which, the animation 628 has been completed at the requesting device 602. The dots have been rearranged into a new configuration. The requesting device 602 displays the new configuration as an icon 632 on the display 605. The icon 632 is associated with facial recognition authentication. The icon 632 indicates to the user that the user can authenticate at the authentication device 600 using facial recognition authentication. In some examples, the icon 632 is associated with fingerprint authentication or non - biometric (e.g., password, passphrase, or gesture) authentication.

[0215] As Figure 6H shown, the authentication device 600 initiates biometric authentication upon receiving a selection of the Figure 6F banner 626. The authentication device 600 uses a biometric sensor (e.g., the facial recognition sensor 603) to determine whether specific biometric authentication criteria have been met.

[0216] Initiating biometric authentication includes displaying a biometric authentication interface 634 having a biometric authentication symbol 636 on a display 601. The biometric authentication symbol 636 is an imitation of a representation of a biometric characteristic (e.g., face). The biometric authentication interface 634 overlays at least a portion of a user interface 624 of a camera application. In some examples, displaying the biometric authentication interface 634 includes blurring at least a portion of the user interface 624 of the camera application. A user presents their face for biometric authentication. The authentication device 600 captures and processes (e.g., analyzes) biometric data associated with the user's face from a biometric sensor (e.g., a face recognition sensor 603). The face recognition sensor 603 is non-contact, such that the sensor is configured to perform biometric authentication without physical input from the user (e.g., without any additional gestures after input 625). Thus, the authentication device 600 initiates biometric authentication using the face recognition sensor 603 without requiring an explicit request from the user to initiate biometric authentication. In some examples, the authentication device 600 requires an explicit request (e.g., a single press or a double press) from the user via one or more input devices (e.g., a hardware button 607) to initiate biometric authentication. In some examples, the authentication device 600 stops displaying the biometric authentication interface 634 after a pre-determined amount of inactivity (e.g., the authentication device 600 does not receive an input (e.g., a tap gesture) within a pre-determined amount of time).

[0217] At Figure 6I which, the authentication device 600 determines that the captured biometric data meets a specific biometric authentication criterion. As a result of successful user authentication, the authentication device 600 displays the biometric authentication interface 634 with a successful biometric authentication symbol 638. The successful biometric authentication symbol 638 indicates that the user has been successfully authenticated at the authentication device 600. In addition, in response to determining that the captured biometric data meets a specific biometric authentication criterion, the authentication device 600 transmits the data to a remote device (e.g., a requesting device 602 or a server communicating with the requesting device 602). The data indicates that the user has been successfully authenticated at the authentication device 600.

[0218] In addition, at Figure 6I which, in response to the authentication device 600 determining that the captured biometric data meets a specific biometric authentication criterion, the requesting device 602 receives data indicating that the user has been successfully authenticated at the authentication device 600. In response to receiving this indication of successful authentication, the requesting device 602 displays an animation 640 on a display 605, in which dots including an icon 632 are scattered such that the icon 632 stops being displayed due to the animation. In some examples, the animation 640 shows the dots including the icon 632 moving outwards before stopping being displayed.

[0219] In addition, at Figure 6J , in response to receiving data indicating that the user has been successfully authenticated at the authentication device 600, the requesting device 602 prompts the user to provide additional authentication information before logging the user in. The requesting device 602 prompts the user for additional authentication because the website requires two-factor authentication for the user to log in. Prompting the user for additional authentication information includes displaying a user interface 642 having a prompt 644 on a display 605. The prompt 644 includes an input field for receiving input via one or more input devices (e.g., a keyboard). The prompt 644 optionally indicates to the user that an authentication code (e.g., authentication code 646) has been sent to one of the user's devices (e.g., a device that has been associated with the user's user account).

[0220] As Figure 6J shown, the authentication device 600 displays an authentication code 646 that can be used to successfully complete two-factor authentication. In some examples, all devices associated with the user's user account (e.g., trusted devices) are capable of displaying the authentication code 646. In some examples, before displaying the authentication code 646, a trusted device (e.g., the authentication device 600, the requesting device 602, or a third device) displays a map having the location of the device (e.g., the requesting device 602) for logging in to the website or service. Displaying the map includes displaying an enabling representation that is used to allow the trusted device to proceed to display the authentication code (e.g., authentication code 646) for completing two-factor authorization at the requesting device 602. If the trusted device receives an input selecting the enabling representation, the trusted device displays the authentication code.

[0221] In some examples, two-factor authentication is not required. When two-factor authentication is not required, the requesting device 602 proceeds to display Figure 6L the user interface 648 or Figure 6M the user interface 654, without prompting the user for additional authentication information.

[0222] At Figure 6K , the requesting device 602 receives input via an input device (e.g., a keyboard). The requesting device 602 determines whether the received input is a valid authentication code. If the requesting device 602 determines that the received input is a valid authentication code, the requesting device 602 continues to log the user in. In some examples, if the requesting device 602 determines that the received input is not a valid authentication code, the requesting device 602 abandons continuing to log the user in.

[0223] At Figure 6LIn [description], when it is determined that the received input is a valid authentication code, the requesting device 602 displays a user interface 648 including a trust enabling indication 650. The requesting device 602 receives an input 652 (e.g., a mouse click event) at the location of the trust enabling indication 650. The trust enabling indication 650 provides the user with an option to trust a web browser (e.g., Safari) for a pre-determined amount of time (e.g., 30 days). When attempting a future login within the pre-determined amount of time, trusting the web browser eliminates the need for two-factor authentication. Thus, when the user later attempts to log in to a website using the same web browser within the pre-determined amount of time, the user does not need to provide additional authentication information (e.g., authentication code 646). In some examples, when it is determined that the received input is a valid authentication code, the requesting device 602 proceeds to display Figure 6M user interface 654 without displaying user interface 648.

[0224] In Figure 6M [description], when the requesting device 602 receives the input 652 at the location of the trust enabling indication 650, it displays a user account web page 654. The user has successfully logged in to the website. In some examples, the user account web page 654 includes content 655. The content 655 can be restricted content (e.g., content that requires authentication to access). In some examples, the content 655 is remote content (e.g., content received from a remote device).

[0225] In some examples, the animation 628 transitions to display an icon 656 as Figure 6N shown, rather than the animation 628 transitioning to display an icon 632 as Figures 6G to 6H shown. At Figure 6N , the animation 628 has been completed at the requesting device 602. Similar to Figure 6H , the dots including the QR code 620 have been rearranged into a new configuration. The requesting device 602 displays the new configuration of the dots on the display 605 as an icon 656. The icon 656 indicates to the user that the user can authenticate at the authentication device 600 using fingerprint authentication (e.g., via the fingerprint sensor 611). Similar to Figure 6G , when receiving data regarding the authentication type, the requesting device 602 displays a user instruction 658 on the display 605. The user instruction 658 includes text prompting the user to take an action at the authentication device 600.

[0226] In some examples, the authentication device 600 uses fingerprint authentication (e.g., via the fingerprint sensor 611) instead of facial recognition authentication as Figure 6H shown to authenticate the user. The authentication device 600 displays a biometric authentication interface 660 instead of Figure 6HThe biometric authentication interface 634. The biometric authentication interface 660 indicates to the user that the user can perform fingerprint authentication via the fingerprint sensor 611. When the user places their finger on the fingerprint sensor 611, the authentication device 600 performs fingerprint authentication. The authentication device 600 receives (or captures) biometric data corresponding to the user's finger via the fingerprint sensor 611.

[0227] In some examples, contrary to a successful biometric authentication as Figure 6I shown, the authentication device 600 determines that the biometric authentication (e.g., via the face recognition sensor 603) has failed, as Figure 6O shown. In some examples, the authentication device 600 determines that the biometric authentication via the fingerprint sensor 611 has failed. Determining that the biometric authentication has failed includes determining that the captured biometric data does not meet specific biometric authentication criteria. In some examples, the authentication device 600 displays the biometric authentication interface 634 with a failed biometric authentication symbol 662 due to the biometric authentication failure. The failed biometric authentication symbol 662 indicates to the user that the authentication has failed at the authentication device 600.

[0228] In some examples, as Figure 6P shown, when determining that the biometric authentication has failed, the authentication device 600 displays an alternative authentication interface 664. In some examples, determining that the biometric authentication has failed includes determining that the biometric authentication is not available at the authentication device 100. For example, due to repeated failed biometric authentication attempts, because the device has recently been powered cycled (e.g., turned off and then on again) or because the device has been in a locked state for a predetermined amount of time, the biometric authentication may not be available. The alternative authentication interface 664 enables the user to authenticate by entering a password (or passphrase) via one or more input devices (e.g., the touchscreen of the display 601). Additionally, when determining that the biometric authentication has failed, the authentication device 600 transmits data to a remote device (e.g., the requesting device 602 or a server communicating with the requesting device 602). The data indicates a new authentication type (e.g., non - biometric authentication) available for authenticating the user at the authentication device 600.

[0229] In some examples, upon receiving data indicating the new authentication type, the requesting device 602 displays an animation 666 that transitions from displaying the icon 632 to displaying a new icon (e.g., Figure 6Q the icon 670) representing password (or passphrase) authentication. The animation includes rearranging the dots including the icon 670 into a new configuration. As Figure 6PAs shown, the animation has started and the dots are being rearranged. Additionally, when the request device 602 receives data indicating a new authentication type, it displays a user instruction 668 on the display 605. The user instruction 668 includes text prompting the user to take an action at the authentication device 600. In some examples, the request device 602 receives data directly from the authentication device 600. In some examples, the request device 602 receives data from a server that communicates with the authentication device 600.

[0230] In some examples, as Figure 6Q shown, at the request device 602 the animation 666 has completed. The dots including the icon 632 have been rearranged into a new configuration. The request device 602 displays the new configuration of the dots on the display 105 as the icon 670. The icon 670 indicates to the user that the user can authenticate at the authentication device 600 using password (or passphrase) authentication.

[0231] In some examples, the animation 628 transitions as Figure 6Q shown to display the icon 670, rather than the animation 628 transitioning as Figures 6G to 6H shown to display the icon 632. At Figure 6Q the animation 628 has completed at the request device 602. Similar to Figure 6H , the dots including the QR code 620 have been rearranged into a new configuration. The request device 602 displays the new configuration of the dots on the display 605 as the icon 670. The icon 670 indicates to the user that the user can authenticate at the authentication device 600 using password (or passphrase) authentication. Similar to Figure 6G , when the request device 602 receives data regarding the authentication type, it displays the user instruction 668 on the display 605.

[0232] In some examples, the authentication device 600 uses password (or passphrase) authentication to authenticate the user instead of facial recognition authentication as Figure 6H shown. The authentication device 600 displays an alternative authentication interface 664 instead of Figure 6H the biometric authentication interface 634.

[0233] In some examples, similar to Figure 6E , the request device 602 displays an alternative login web page 618 on the display 605 as Figure 6R shown. In some examples, the request device 602 receives an input 674 (e.g., a mouse click event) at the location of the previous page enable indication 674. In some examples, when an input 674 is received at the location of the previous page enable indication 672, the request device 602 transitions to display the previous web page 676 on the display 605 as Figure 6SAs shown. Accordingly, the requesting device 602 transmits data to a remote device (e.g., the authentication device 600 or a server communicating with the authentication device 600). The data indicates that the user has navigated away from the alternative login web page 618.

[0234] In some examples, similar to Figure 6E , the authentication device 600 displays a banner 626 (e.g., a user interface element) on the display 601, as Figure 6R shown. In some examples, when the authentication device 600 receives data indicating that the user has navigated away from the alternative login web page 618, it stops displaying the banner 626, as Figure 6S shown. By stopping the display of the banner 626, the authentication device 600 stops providing the user at the authentication device 600 with the option to log in to the website via authentication (e.g., biometric authentication or non - biometric authentication). In some examples, when the authentication device 600 receives data indicating that the user has navigated away from the alternative login web page 618, it stops displaying the biometric authentication interface 634, as Figures 6T to 6U shown, or the biometric authentication interface 660. In some examples, Figures 6R to 6S the above - described techniques shown in

[0235] can be similarly performed with respect to fingerprint authentication and password (or passphrase) authentication. Figure 6H In some examples, similar to Figure 6T , the requesting device 602 displays a user interface with an icon 632 and / or user instructions 630 on the display 605, as Figure 6U shown. In some examples, the requesting device 602 receives an input 680 (e.g., a mouse click event) at the location of the exit enabling indication 678. Upon receiving the input 680, the requesting device 602 stops displaying the web browser (e.g., Safari) and proceeds to display the desktop 682 on the display 605, as

[0236] shown. Accordingly, the requesting device 602 transmits data indicating that the user has navigated away from the user interface with the icon 632 and / or user instructions 630. This user interface is associated with the technique for logging in to the website via authentication at the authentication device 600. Figure 6U In some examples, when the authentication device 600 receives data indicating that the user has navigated away from the user interface with the icon 632 and / or user instructions 630, it stops displaying the biometric authentication interface 634, as Figures 6R to 6Sas shown. In some examples, Figures 6T to 6U the above-described techniques shown in Figures 6T to 6U can be similarly performed with respect to fingerprint authentication and password (or passphrase) authentication.

[0237] In some examples, the above-described techniques for using a visual representation of data (e.g., a unique code) to manage secure logins with authentication can be used with respect to websites and / or services such as iCloud, Apple Business Manager, and Apple School Manager.

[0238] Figure 7 is a flowchart showing a method for authorizing access to content on a second electronic device (e.g., requesting device 602) using a first electronic device (e.g., the authenticating first electronic device 600) according to some embodiments. Method 700 is performed at a first electronic device (e.g., 100, 300, 500, 600) having a display, one or more cameras, and one or more input devices. Some operations in method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0239] As described below, method 700 provides an intuitive way to use a first electronic device (e.g., the authenticating first electronic device 600) to authorize access to content on a second electronic device (e.g., the requesting device 602). The method reduces the cognitive burden on the user to authorize access to content, thereby creating a more effective human-machine interface. For battery-powered computing devices, enabling the user to authorize content faster and more efficiently saves power and increases the time interval between two battery charges.

[0240] The first electronic device (e.g., the authenticating device 600) detects (702) the presence of a visual representation of data (e.g., 620, a QR code, a barcode, or a visual representation of other information identifying the second electronic device (e.g., the requesting device 602), a server associated with the second electronic device (e.g., the requesting device 602), a session associated with the second electronic device (e.g., the requesting device 602)) displayed on the second electronic device (e.g., the requesting device 602) via a first camera of the one or more cameras.

[0241] In some examples, after detecting (704) the presence of a visual representation (e.g., 620) of data displayed on a display (e.g., 605) of a second electronic device (e.g., requesting device 602) at a first electronic device (e.g., authentication device 600), a first prompt (e.g., 632, 630, an image having accompanying text instructing the user to authenticate at the authentication device via a particular authentication type) is displayed on the display (e.g., 605) of the second electronic device (e.g., requesting device 602). The first prompt (e.g., 632, 630) indicates (706) a first authentication type (e.g., biometric authentication (e.g., fingerprint or facial recognition), password authentication) available at the first electronic device (e.g., authentication device 600). In some examples, the first prompt is selected from a plurality of different prompts based on the authentication types available at the first electronic device (e.g., authentication device 600) (e.g., if the first electronic device (e.g., authentication device 600) is enabled to use facial authentication, the second electronic device (e.g., requesting device 602) displays a visual indication that facial authentication is enabled on the first electronic device (e.g., authentication device 600)). In some examples, if the first electronic device (e.g., authentication device 600) is enabled to use fingerprint authentication, the second electronic device (e.g., requesting device 602) displays a different visual indication (e.g., 656, 658) that fingerprint authentication is enabled on the first electronic device (e.g., authentication device 600). In some examples, if the first electronic device (e.g., authentication device 600) is enabled to use password or passphrase authentication, the second electronic device (e.g., requesting device 602) displays a visual indication (e.g., 670, 668) that password or passphrase authentication is enabled on the first electronic device (e.g., authentication device 600). Providing a prompt indicating the authentication types available at the first electronic device (e.g., authentication device 600) on the second electronic device (e.g., requesting device 602) (e.g., a device requesting access to content) provides the user with feedback regarding the status of both the second electronic device (e.g., requesting device 602) (e.g., waiting for authorization to access content) and the first electronic device (e.g., authentication device 600) (e.g., waiting for a particular type of authorization input). Additionally, this provides the user with further feedback regarding the next steps required to perform an operation (e.g., presenting a biometric characteristic (e.g., face or fingerprint) for biometric authentication on the first electronic device). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0242] In some examples, the first authentication type is biometric authentication via a face recognition sensor (e.g., 603, a thermal, optical, and / or depth-based face recognition sensor), and a first prompt (e.g., 632, 630) displayed on a display (e.g., 605) of a second electronic device (e.g., requesting device 602) indicates that biometric authentication via the face recognition sensor is available at a first electronic device (e.g., authenticating device 600).

[0243] In some examples, the first authentication type is biometric authentication via a fingerprint sensor (e.g., 611), and a first prompt (e.g., 656, 658) displayed on a display of a second electronic device (e.g., requesting device 602) indicates that biometric authentication via the fingerprint sensor is available at a first electronic device (e.g., authenticating device 600).

[0244] In some examples, the first authentication type is non-biometric authentication via a character input interface (e.g., 664, a keypad or keyboard including character input keys for entering a passphrase or password), and a first prompt (e.g., 670, 668) displayed on a display of a second electronic device (e.g., requesting device 602) indicates that non-biometric authentication is available at a first electronic device (e.g., authenticating device 600).

[0245] In some examples, after a first prompt (e.g., 632, 630, 656, 658, 668, 670) is displayed on a display of a second electronic device (e.g., requesting device 602), a first electronic device (e.g., authentication device 600) detects a change in the availability of a first authentication type at the first electronic device (e.g., authentication device 600) (e.g., after more than a predetermined number of biometric authentication failures or after a device reboot, biometric authentication is locked and thus a password / passphrase is required). In some such examples, in response to detecting a change in the availability of the first authentication type at the first electronic device (e.g., authentication device 600), the first electronic device (e.g., authentication device 600) provides (e.g., to the requesting device 602 or to a server communicating with the requesting device 602) information indicating that the first authentication type is no longer available at the first electronic device (e.g., authentication device 600), the information causing the first prompt displayed on the display of the second electronic device (e.g., requesting device 602) to be replaced with a fourth prompt (e.g., 632, 630, 656, 658, 668, 670) displayed on the display of the second electronic device (e.g., requesting device 602). The fourth prompt indicates a second authentication type available at the first electronic device (e.g., authentication device 600), the second authentication type being different from the first authentication type available at the first electronic device (e.g., authentication device 600). Providing a prompt on the second electronic device (e.g., requesting device 602) (e.g., a device requesting access to content) that changes based on a change in the authentication type available at the first electronic device (e.g., authentication device 600) provides the user with feedback regarding the status of both the second electronic device (e.g., requesting device 602) (e.g., waiting for authorization to access content) and the first electronic device (e.g., authentication device 600). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0246] In some examples, replacing a first prompt displayed on a display of a second electronic device (e.g., requesting device 602) with a fourth prompt includes animating (e.g., 666) the transition from the first prompt to the fourth prompt. In some examples, the first prompt (e.g., a prompt to provide authentication via a face recognition sensor) includes a plurality of visual elements (e.g., dots) that are rearranged to form the fourth prompt (e.g., a prompt to provide authentication via a password entry interface). Animating the transition between the prompts displayed on the second device draws the user's attention to the changed authentication requirements on the first electronic device, which improves the user-device interface and reduces the risk of errors and user confusion. Improving the user-device interface and reducing errors enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user understand the state of the device and providing appropriate input, reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0247] In some examples, when a camera application is active on a first electronic device (e.g., authenticating device 600) and when the camera application is operating in a restricted mode (e.g., a mode in which one or more functions of the camera application that are available when the camera is operating in an unrestricted mode (e.g., accessing certain media) are unavailable), the presence of a visual representation of detection data is detected. In some examples, a QR code is detected via a dedicated camera application of the device. In some examples, a QR code is detected via a dedicated camera application of the device when the device is in a locked state (e.g., when in a restricted camera mode).

[0248] In response to detecting the presence of a visual representation of data, a first electronic device (e.g., authentication device 600) prompts (708) (e.g., displays a visual cue, generates a haptic output, generates an audio output) a user to provide authorization information at the first electronic device (e.g., authentication device 600) to enable access to content on a second electronic device (e.g., requesting device 602); (ISE, Additionally, in response to detecting the presence of a visual representation of data (e.g., a QR code), the first electronic device (e.g., authentication device 600) transmits information.) (ISE, The information indicates that the visual representation of data was detected by the first electronic device (e.g., authentication device 600).) (ISE, The information indicates the type of authentication available at the first electronic device (e.g., biometric authentication, password / passphrase entry).) (ISE, The first electronic device (e.g., authentication device 600) transmits to a remote device (e.g., a server, a second electronic device (e.g., requesting device 602).) In some examples, prompting the user includes displaying a banner (e.g., 626) that prompts the user to log in to a service or website. In some examples, in response to selecting the banner via an input device (e.g., a tap gesture (e.g., 625) on a touch-sensitive display (e.g., 601)), the first device (e.g., authentication device 600) transmits information. In some examples, the information indicates that the visual representation of data was detected by the first electronic device (e.g., authentication device 600). In some examples, the information indicates the type of authentication available at the first electronic device (e.g., biometric authentication, password / passphrase entry). In some examples, the information indicates that the first device received an input at the banner. In some examples, the information includes data corresponding to and / or associated with the indication. Prompting the user to provide authorization at the first electronic device (e.g., authentication device 600) in response to detecting data on a second electronic device (e.g., requesting device 602) to enable access to content at a separate second electronic device (e.g., requesting device 602) allows the user to utilize the authorization hardware and software available on the first electronic device (e.g., which may not be available on the second electronic device (e.g., requesting device 602) and may be more secure) for authorization and security functions, while using the hardware and software of the second electronic device (e.g., which may not be available on the first electronic device (e.g., authentication device 600)) to access content. Additionally, doing so provides two-factor authentication with a trusted device (e.g., the first electronic device) and provides the user with clear guidance on what actions need to be taken to access the content.Enhancing the operability of a device by expanding the usability of hardware and software for a user and adding security enhancements, and making the user-device interface more efficient (e.g., by providing enhanced functionality to reduce the burden of interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0249] In some examples, the one or more input devices include biometric sensors (e.g., fingerprint sensors (e.g., 611) or face recognition sensors (e.g., 603)) and non-biometric input devices (e.g., 664, touch panel, physical keyboard, or virtual keyboard). In some such examples, prompting (e.g., visually, haptically, aurally) a user to provide authorization information at a first electronic device (e.g., authentication device 600) to enable access to content on a second electronic device (e.g., requesting device 602) includes one or more of the following. Based on determining that biometric authentication is available at the first electronic device (e.g., authentication device 600), generating a prompt (e.g., visual output, haptic output, audio output) at the first electronic device (e.g., authentication device 600) that prompts the user to provide authorization information at the first electronic device (e.g., authentication device 600) via a biometric sensor (e.g., face recognition sensor or fingerprint sensor). Based on determining that biometric authentication is not available at the first electronic device (e.g., due to repeated failures, power cycling the device, the device having been locked for longer than a predetermined period of time), generating a prompt (e.g., visual output, haptic output, audio output) at the first electronic device (e.g., authentication device 600) that prompts the user to provide authorization information at the first electronic device (e.g., authentication device 600) via a non-biometric input device. Prompting the user to provide authorization via a biometric sensor (e.g., face recognition sensor or fingerprint sensor) when biometric authentication is available, and prompting the user to provide authorization via a non-biometric input device when biometric authentication is not available, provides the user with the ability to authenticate via a biometric option, which is generally more secure and convenient (e.g., requires less input) when available, and via an alternative option when the biometric option is not available. Additionally, this provides the user with feedback regarding the status of the device. Improving security, reducing the number of inputs required to perform an operation, and providing the user with improved feedback enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0250] In some examples, further in accordance with determining that biometric authentication is available at a first electronic device (e.g., authentication device 600), a second prompt is displayed on a display of a second electronic device (e.g., requesting device 602), prompting a user to provide authorization information at the first electronic device (e.g., authentication device 600) via a biometric sensor (e.g., a face recognition sensor or a fingerprint sensor). In some such examples, further in accordance with determining that biometric authentication is not available at the first electronic device (e.g., authentication device 600), a third prompt (e.g., 668, 670) is displayed on a display of the second electronic device (e.g., requesting device 602) (e.g., 605), prompting the user to provide authorization information at the first electronic device (e.g., authentication device 600) via a non-biometric input device. Providing a prompt based on the type of authentication available at the first electronic device (e.g., authentication device 600) on the second electronic device (e.g., requesting device 602) (e.g., the device requesting access to content) provides the user with feedback regarding the status of both the second electronic device (e.g., requesting device 602) (e.g., waiting for authorization to access content) and the first electronic device (e.g., authentication device 600) (e.g., waiting for a particular type of authorization input). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0251] After prompting the user to provide authorization information, the first electronic device (e.g., authentication device 600) receives (710) a set of one or more inputs.

[0252] In some examples, the one or more input devices include physical input mechanisms (e.g., 607, hardware (e.g., non-virtual, e.g., physical) buttons), and the set of one or more inputs includes one or more inputs received via the physical input mechanism (e.g., one or more presses of a hardware button). In some examples, the one or more presses initiate biometric authentication (e.g., authentication via a face recognition sensor (e.g., 603)). Authorizing access to content based on inputs that include inputs via a physical input mechanism (e.g., a button) improves security by leveraging more explicit inputs, which can be useful in cases where one or more of the inputs in the set of inputs include less explicit inputs (e.g., inputs received via a face recognition sensor) that may not have been intentionally provided by the user, thereby enhancing security by avoiding unintentional authorization inputs. Enhanced security enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the occurrence of non-authenticated inputs), which also reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0253] In some examples, the one or more input devices include biometric sensors (e.g., a face recognition sensor (e.g., 603) or a fingerprint sensor (e.g., 611)), and the set of one or more inputs further includes inputs received via the biometric sensor (e.g., a face recognition sensor or a fingerprint sensor) (e.g., presenting a biometric characteristic (e.g., a face) for biometric authentication). In some examples, prompting the user to provide authorization information at a first electronic device (e.g., authentication device 600) includes prompting the user to provide authorization information via the biometric sensor (e.g., a face recognition sensor or a fingerprint sensor).

[0254] In response to receiving (712) the one or more inputs of the set and based on determining that the one or more inputs of the set include authorization information that matches the stored authorization information, a first electronic device (e.g., authentication device 600) transmits (714) information authorizing access to content (e.g., 644) on a second electronic device (e.g., requesting device 602) (e.g., transmits the information to the second electronic device (e.g., requesting device 602) or to a server associated with the second electronic device (e.g., requesting device 602)). Transmitting information authorizing content (e.g., 655) at the second electronic device (e.g., requesting device 602) based on a set of inputs that match the authorization information at the first electronic device (e.g., authentication device 600) enables a user to utilize the hardware and software of the first electronic device (e.g., authentication device 600) to provide authorization and security functions while accessing content using the hardware and software of the second electronic device (e.g., requesting device 602) (e.g., which may not be available on the first electronic device (e.g., authentication device 600)). Additionally, doing so avoids the need to store the authorization information on the second electronic device (e.g., requesting device 602) (which may not be as secure as the first electronic device (e.g., authentication device 600)). Expanding the availability of hardware and software for the user and increasing security enhances the operability of the device and makes the user-device interface more efficient (e.g., by providing enhanced functionality to reduce the burden of interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0255] In response to receiving the one or more inputs of the set and based on determining that the one or more inputs of the set do not include authorization information that matches the stored authorization information, the first electronic device (e.g., authentication device 600) forgoes transmitting (716) information authorizing access to content on the second electronic device (e.g., requesting device, 602).

[0256] In some examples, before detecting the presence of a visual representation of data, a first electronic device (e.g., authentication device 600) displays a preview of image data captured via one or more cameras on a display. In some such examples, prompting a user to provide authorization information at the first electronic device (e.g., authentication device 600) includes visually obscuring (e.g., blurring, dimming, desaturating, reducing contrast) at least a portion of the preview of the image data (e.g., 624). When at least a portion of the preview of the image data from the camera is visually obscured when prompting the user to provide authorization at the first electronic device (e.g., authentication device 600) in response to detecting the presence of a visual representation of data, this provides the user with additional feedback as to what the electronic device has detected via one or more of its cameras and draws the user's attention to the prompt. Providing the user with improved visual feedback enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and effectively.

[0257] In some examples, the first electronic device (e.g., authentication device 600) receives an indication that a visual representation of data has stopped being displayed on a second electronic device (e.g., requesting device 602) (e.g., closing a web browser (e.g., 680), closing a web browser window, closing an application, navigating back to an application (e.g., 674), navigating to a new page). In some such examples, in response to receiving the indication, the first electronic device (e.g., authentication device 600) stops prompting the user to provide authorization information at the first electronic device (e.g., authentication device 600). In some such examples, the authentication device stops displaying a banner (e.g., 626) that prompts the user to authenticate. In some such examples, the authentication device stops displaying a prompt (e.g., 634, 636) that prompts the user to authenticate via a particular authentication type.) When the visual representation of data is no longer being displayed on the second electronic device (e.g., requesting device 602), stopping the prompting of the user (at the first electronic device, e.g., authentication device 600) to provide authorization at the first electronic device (e.g., authentication device 600) provides the user with feedback as to the state of both the second electronic device (e.g., requesting device 602) (e.g., no longer attempting to access content that requires authorization) and the first electronic device (e.g., authentication device 600) (e.g., not waiting for a particular type of authorization input). Providing the user with improved feedback enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating the device / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0258] Note that the details of the processes described above with respect to method 700 (e.g., Figure 7 ) also apply in a similar manner to the methods described below / above. For example, method 800 optionally includes one or more features of the various methods described above with reference to method 700. For example, the second electronic device (authentication device 600) of method 800 can be the first electronic device (e.g., authentication device 600) of method 700. As another example, the first electronic device (e.g., requesting device 602) of method 800 can be the second electronic device (e.g., requesting device 602) of method 700. For the sake of brevity, these details are not repeated below.

[0259] Figure 8 is a flowchart showing a method for using a second electronic device (e.g., an authentication device) to authorize access to content on an electronic device (e.g., a first electronic device as requesting device 602) according to some embodiments. Method 800 is performed at a first electronic device (e.g., 100, 300, 500) having a display. Some operations in method 800 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0260] As described below, method 800 provides an intuitive way to use a second electronic device (e.g., an authentication device) to authorize access to content on an electronic device (e.g., a first electronic device as requesting device 602). The method reduces the cognitive burden on the user to authorize access to content, thereby creating a more effective human-machine interface. For battery-powered computing devices, enabling the user to authorize content faster and more efficiently saves power and increases the time interval between two battery charges.

[0261] The first electronic device (e.g., requesting device 602) displays (802) a visual representation (e.g., 620) of data on a display of the first electronic device (e.g., requesting device 602).

[0262] In some examples, the visual representation of the data (e.g., 620) is a two-dimensional computer-readable pattern (e.g., a QR code, a non-text code encoding data along two dimensions (e.g., two dimensions orthogonal to each other)).

[0263] In some examples, the visual representation of the data includes one or more of the following: a Uniform Resource Locator (URL), a session identifier (ID), and a key.

[0264] The first electronic device (e.g., the requesting device 602) receives (804) a visual representation of data that has been detected by a second electronic device (e.g., the authentication device 600) (e.g., a QR code, a barcode, or a visual representation of other information identifying the second electronic device (e.g., the authentication device 600), a server associated with the second electronic device (e.g., the authentication device 600), a session associated with the second electronic device (e.g., the authentication device 600)).

[0265] In some examples, the first information indicates (806) a first authentication type available at the second electronic device (e.g., the authentication device 600). In some such examples, the first authentication type is biometric authentication via a face recognition sensor (e.g., 603). In some such examples, the first authentication type is biometric authentication via a fingerprint sensor (e.g., 611). In some such examples, the first authentication type is non - biometric authentication (e.g., a password, passphrase, or pattern entered via a character input interface (e.g., 664) or a touch - sensitive surface (e.g., 601)).

[0266] In some examples, after a first prompt (e.g., 632, 630, 656, 658, 668, 670) is displayed on a display (e.g., 605) of a first electronic device (e.g., requesting device 602) and after the availability of a first authentication type at a second electronic device (e.g., authentication device 600) changes (e.g., biometric authentication is locked, so a password / passphrase or gesture is required), the first electronic device (e.g., requesting device 602) replaces the first prompt (e.g., a prompt for biometric authentication) displayed on the display of the first electronic device (e.g., requesting device 602) with a second prompt (e.g., 632, 630, 656, 658, 668, 670) displayed on the display of the first electronic device (e.g., requesting device 602) (e.g., a prompt for password / passphrase or gesture authentication). In some such examples, the second prompt indicates a fourth authentication type available at the second electronic device (e.g., authentication device 600), which is different from the first authentication type available at the second electronic device (e.g., authentication device 600). Providing an updated prompt based on a change in the authentication type available at a second electronic device (e.g., authentication device 600) on a first electronic device (e.g., requesting device 602) (e.g., a device requesting access to content) provides the user with feedback regarding the status of both the first electronic device (e.g., requesting device 602) and the second electronic device (e.g., authentication device 600) (e.g., which is waiting for authorization before being able to access content). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0267] In some examples, replacing a first prompt displayed on a display of a first electronic device (e.g., requesting device 602) with a second prompt includes the first electronic device (e.g., requesting device 602) animating a transition from the first prompt to the second prompt (e.g., 666). In some examples, the first prompt (e.g., a prompt to provide authentication via a facial recognition sensor) includes a plurality of visual elements (e.g., dots) that are rearranged to form the second prompt (e.g., a prompt to provide authentication via a password entry interface). Animating the transition between prompts displayed on the first device draws the user's attention to the changed authentication requirements on the first electronic device, which improves the user-device interface and reduces the risk of errors and user confusion. Improving the user-device interface and reducing errors enhances the device's operability and makes the user-device interface more efficient (e.g., by helping the user be more aware of the device's state and provide appropriate input while also reducing user errors when operating the device / interacting with the device), which additionally reduces power usage and improves the device's battery life by enabling the user to use the device more quickly and effectively.

[0268] In response to receiving a first message (e.g., a message including information about the types of authentication available at authentication device 600), a first electronic device (e.g., requesting device 602) displays (808) a first prompt (e.g., 632, 630, 656, 658, 668, 670, an image with accompanying text indicating that the user should authenticate at the authentication device via a particular type of authentication) on a display of the first electronic device (e.g., requesting device 602), prompting the user to provide authentication information at a second electronic device (e.g., authentication device 600) to enable access to restricted content on the first electronic device (e.g., requesting device 602). Displaying a prompt at the first electronic device (e.g., requesting device 602) to provide authentication at the second electronic device (e.g., authentication device 600) in response to the second electronic device (e.g., authentication device 600) detecting a visual representation of data displayed at the first electronic device (e.g., requesting device 602) provides the user with feedback on the status of both the first electronic device (e.g., requesting device 602) (e.g., having received an indication of activity of the second electronic device (e.g., authentication device 600)) and the second electronic device (e.g., authentication device 600) (e.g., waiting for an authorization input). Providing the user with improved feedback enhances the device's operability and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors when operating the device / interacting with the device), which additionally reduces power usage and improves the device's battery life by enabling the user to use the device more quickly and effectively.

[0269] In some examples, displaying a first prompt on a display (e.g., 605) of a first electronic device (e.g., requesting device 602) includes one or more of the following. Based on determining that the first information indicates that a second authentication type (e.g., authentication via a face recognition sensor (e.g., 603), fingerprint sensor (e.g., 611), or password / passphrase input via a character input interface (e.g., 664)) is available at a second electronic device (e.g., authentication device 600), the first electronic device (e.g., requesting device 602) displays a first type of prompt (e.g., an image with accompanying text instructing the user to authenticate at the second electronic device (e.g., authentication device 600) via a particular authentication type) on the display of the first electronic device (e.g., requesting device 602). Based on determining that the first information indicates that a third authentication type is available at the second electronic device (e.g., authentication device 600), the first electronic device (e.g., requesting device 602) displays a second type of prompt on the display of the first electronic device (e.g., requesting device 602). Providing prompts that vary based on the authentication types available at the second electronic device (e.g., authentication device 600) provides the user with feedback on the state of the second electronic device (e.g., which input devices are available for authorization at the second electronic device (e.g., authentication device 600), and the second electronic device (e.g., authentication device 600) is waiting for an authorization input). Additionally, this provides the user with guidance on how to proceed with an operation (e.g., accessing content). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively. In some such examples, the second authentication type is different from the third authentication type. In some such examples, the first type of prompt is different from the second type of prompt.

[0270] In some examples, the first type of hint indicates a second authentication type available at a second electronic device (e.g., authentication device 600). In some such examples, the second type of hint indicates a third authentication type available at the second electronic device (e.g., authentication device 600). Providing a hint indicating the authentication types available at the second electronic device (e.g., authentication device 600) provides the user with feedback on the state of the second electronic device (e.g., which input devices are available for authorization at the second electronic device (e.g., authentication device 600), and the second electronic device (e.g., authentication device 600) is waiting for an authorization input). Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which in turn reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0271] In some examples, the restricted content (e.g., 655) includes content stored remotely (e.g., content not stored at the first electronic device (e.g., requesting device 602)).

[0272] In some examples, the restricted content includes a web page that requires authentication to access.

[0273] The first electronic device (e.g., requesting device 602) receives (810) second information (e.g., information authorizing access to restricted content on the first electronic device (e.g., requesting device 602) from the second electronic device (e.g., authentication device 600) or a server associated with the second electronic device (e.g., authentication device 600)).

[0274] In some examples, the second information includes authentication information input at a first electronic device (e.g., the requesting device 602) (e.g., via a character input key). In some such examples, the second information includes data indicating that a successful authentication has occurred at a second electronic device (e.g., the authentication device 600) (e.g., received from the second electronic device (e.g., the authentication device 600) or a device communicating with the second electronic device (e.g., an authentication server)). In some such examples, for the purpose of completing two-factor authentication, the second information is a set of information that includes, in addition to a password / passphrase input via a character input key at the first electronic device (e.g., the requesting device 602), data indicating a successful authentication at the authentication device. In some such examples, the authentication information corresponds to a plurality of characters (e.g., 646) displayed on a display of a third device, and the third device is associated with the user's user account. In some such examples, the third device is selected from: the first electronic device (e.g., the requesting device 602), the second electronic device (e.g., the authentication device 600), and a device different from the first and second electronic devices.

[0275] In response to receiving the second information, the first electronic device (e.g., the requesting device 602) provides (812) access to restricted content on the first electronic device (e.g., the requesting device 602).

[0276] In some examples, before displaying a visual representation of data on a display of the first electronic device (e.g., the requesting device 602), the first electronic device displays, on the display of the first electronic device (e.g., the requesting device 602), a user interface (e.g., 606) that includes a first affordance representation (e.g., 614) for accessing content via the visual representation of the data. The first electronic device also receives, at an input device of the first electronic device (e.g., the requesting device 602), user input (e.g., 612) corresponding to a selection of the first affordance representation. In some such examples, in response to receiving the user input, the visual representation of the data is displayed. In some such examples, the user interface includes one or more fillable fields (e.g., 608, 610) for inputting credentials (e.g., fillable via a character input key).

[0277] In some examples, when the electronic device receives credential information that meets the authorization criteria via the one or more fillable fields, access to restricted data is granted. In some such examples, when a visual representation of data is displayed on a display of a first electronic device (e.g., requesting device 602), the first electronic device simultaneously displays on the display of the first electronic device (e.g., requesting device 602) a second affordance for navigating to a user interface that includes a first affordance representation. In some such examples, in response to receiving user input corresponding to a selection of the second affordance, the first electronic device (e.g., requesting device 602) stops displaying the visual representation of the data and instead displays a user interface (e.g., 606) that includes the first affordance representation. In cases where the second electronic device (e.g., authentication device 600) is unavailable to the user, displaying an affordance for accessing a user interface that provides an alternative authentication method provides the user with additional input options and provides an indication of the steps that the user can take to perform an operation. Providing the user with an alternative authorization method enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user provide appropriate input and reducing user errors and frustrations when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0278] In some examples, in response to receiving the first information, the first electronic device (e.g., requesting device 602) replaces the visual representation of the data displayed on the display of the first electronic device (e.g., requesting device 602) with a first prompt, where replacing the visual representation of the data with the first prompt includes displaying an animated transition (e.g., 628) from the visual representation of the data to the first prompt. In some such examples, the visual representation of the data (e.g., QR code) includes a plurality of visual elements (e.g., dots), which are rearranged to form the first prompt (e.g., a prompt to provide authentication via a facial recognition sensor). Displaying the transition between the visual representation of the data and the prompt on the first electronic device (e.g., requesting device 602) in an animated manner draws the user's attention to the changed authentication requirements on the first electronic device, which improves the user-device interface and reduces the risk of errors and user confusion. Improving the user-device interface and reducing errors enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user understand the state change of the device and provide appropriate input, thereby reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and effectively.

[0279] Note that the above refers to method 800 (e.g., Figure 8)Details of the processes described above apply to the methods described above in a similar manner. For example, method 700 optionally includes one or more features of the various methods described above with reference to method 800. For example, the second electronic device of method 700 (e.g., requesting device 602) may be the first electronic device of method 800 (e.g., requesting device 602). For example, the first electronic device of method 700 (e.g., authenticating device 600) may be the second electronic device of method 800 (e.g., authenticating device 600). For the sake of brevity, these details are not repeated hereinafter.

[0280] For purposes of explanation, the foregoing description has been described by reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the techniques and their practical applications. Other technicians in the art will thus be able to best utilize the techniques and various embodiments with various modifications suited to the particular uses contemplated.

[0281] Although the present disclosure and examples have been described in full with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of the present disclosure and examples as defined by the claims.

[0282] As described above, one aspect of the disclosed technology is the collection and use of data obtained from various sources to improve the delivery of heuristic content or any other content that a user may be interested in. The present disclosure contemplates that in some instances, the collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifying information.

[0283] The present disclosure recognizes that the use of such personal information data in the disclosed technology can be used to benefit users. For example, the personal information data can be used to deliver targeted content that the user is more interested in. Thus, the use of such personal information data enables a planned control of the content delivered. In addition, the present disclosure also contemplates other uses of personal information data that benefit users.

[0284] The present disclosure also contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Additionally, such collection should only occur after the user's informed consent. Further, such entities should take any necessary steps to safeguard and protect access to such personal information data and ensure that others who have access to the personal information data comply with their privacy policies and procedures. Additionally, such an entity may subject itself to third-party assessments to demonstrate its compliance with widely accepted privacy policies and practices.

[0285] Regardless of the foregoing, the present disclosure also anticipates embodiments in which users selectively block the use or access of personal information data. That is, the present disclosure anticipates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, with respect to an advertising delivery service, the techniques of the present disclosure may be configured to allow a user to select to "opt-in" or "opt-out" of participating in the collection of personal information data during the registration for the service. As another example, a user may choose not to provide location information for a targeted content delivery service. As yet another example, a user may choose not to provide precise location information but permit the transmission of location area information.

[0286] Accordingly, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also anticipates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the techniques of the present disclosure will not be rendered inoperative due to the lack of all or a portion of such personal information data. For example, preferences may be inferred based on non-personal information data or an absolute minimum amount of personal information such as the content requested by a device associated with the user, other non-personal information available to the content delivery service, or publicly available information, and content may be selected and delivered to the user accordingly.

Claims

1. A method, comprising: at a computer system having a display device, one or more cameras, one or more sensors, and an input device: displaying, via the display device, a representation of the field of view of the one or more cameras; while displaying the representation of the field of view of the one or more cameras: detecting, in the field of view of the one or more cameras, a computer-readable code; and in response to detecting the computer-readable code, initiating a process of capturing biometric information via the one or more sensors that are part of the computer system, the computer system displaying the representation of the field of view of the one or more cameras that includes the computer-readable code.

2. The method according to claim 1, wherein initiating the process of capturing biometric information via the one or more sensors includes using at least one of the one or more cameras to capture the biometric information.

3. The method according to claim 2, wherein: detecting the computer-readable code includes detecting the computer-readable code via a first camera of the one or more cameras, wherein the first camera has a first field of view, and wherein the first camera is located on a first side of the computer system; and the process of capturing biometric information includes using a second camera of the one or more cameras to capture the biometric information, wherein the second camera has a second field of view different from the first field of view, and wherein the second camera is located on a second side of the computer system, the second side of the computer system being different from the first side of the computer system.

4. The method according to any one of claims 1 to 3, wherein the process of capturing biometric information includes capturing facial features of a user of the computer system.

5. The method according to any one of claims 1 to 3, wherein the process of capturing biometric information includes: detecting, via the input device, user hand input before capturing the biometric information.

6. The method according to any one of claims 1 to 3, wherein the input device is a button, and wherein the process of capturing biometric information includes: detecting activation of the button before capturing the biometric information.

7. The method according to any one of claims 1 to 3, wherein initiating the process of capturing biometric information includes displaying, via the display device, one or more user interface objects generated simultaneously with the representation of the field of view of the one or more cameras.

8. The method according to any one of claims 1 to 3, wherein initiating the process of capturing biometric information includes visually blurring at least a portion of the representation of the field of view of the one or more cameras.

9. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system having a display device, one or more cameras, one or more sensors, and an input device, the one or more programs including instructions for performing the method according to any one of claims 1 to 8.

10. A computer system having a display device, one or more cameras, one or more sensors, and an input device, the computer system comprising: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 8.

11. A computer system having a display device, one or more cameras, one or more sensors, and an input device, the computer system comprising: means for performing the method according to any one of claims 1 to 8.

12. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system having a display device, one or more cameras, one or more sensors, and an input device, the one or more programs including instructions for performing the method according to any one of claims 1 to 8.

13. A method comprising: at a first electronic device having a display: displaying a visual representation of data on the display of the first electronic device; receiving first information that the visual representation of the data has been detected by a second electronic device; in response to receiving the first information, displaying a first prompt on the display of the first electronic device, the first prompt prompting a user to provide authentication information at the second electronic device for enabling access to restricted content on the first electronic device; receiving second information authorizing access to the restricted content on the first electronic device; and in response to receiving the second information, providing access to the restricted content on the first electronic device.

14. The method according to claim 13, wherein the first information indicates a first authentication type available at the second electronic device.

15. The method according to claim 14, wherein the first authentication type is biometric authentication via a facial recognition sensor.

16. The method according to claim 14, wherein the first authentication type is biometric authentication via a fingerprint sensor.

17. The method according to claim 14, wherein the first authentication type is non-biometric authentication.

18. The method according to claim 14, wherein: displaying the first prompt on the display of the first electronic device comprises: displaying a first type of prompt on the display of the first electronic device based on determining that the first information indicates a second authentication type available at the second electronic device; and Based on determining that the first information indicates that a third authentication type is available at the second electronic device, display a second type of prompt on the display of the first electronic device, wherein the second authentication type is different from the third authentication type, and the first type of prompt is different from the second type of prompt.

19. The method according to claim 18, wherein: the first type of prompt indicates the second authentication type available at the second electronic device, and the second type of prompt indicates the third authentication type available at the second electronic device.

20. The method according to claim 14, further comprising: after displaying the first prompt on the display of the first electronic device and after the availability of the first authentication type at the second electronic device changes, replacing the first prompt displayed on the display of the first electronic device with a second prompt displayed on the display of the first electronic device, wherein the second prompt indicates a fourth authentication type available at the second electronic device, and the fourth authentication type is different from the first authentication type available at the second electronic device.

21. The method according to claim 20, replacing the first prompt displayed on the display of the first electronic device with the second prompt includes animating a transition from the first prompt to the second prompt.

22. The method according to claim 13, further comprising: before displaying a visual representation of the data on the display of the first electronic device, displaying a user interface on the display of the first electronic device, the user interface including a first affordance for accessing content via the visual representation of the data; and receiving, at an input device of the first electronic device, user input corresponding to a selection of the first affordance, wherein the visual representation of the data is displayed in response to receiving the user input.

23. The method according to claim 22, wherein the user interface includes one or more fillable fields for entering credentials.

24. The method according to claim 22, further comprising: when displaying the visual representation of the data on the display of the first electronic device, simultaneously displaying, on the display of the first electronic device, a second affordance for navigating to the user interface including the first affordance.

25. The method according to any one of claims 13 to 24, wherein the visual representation of the data is a two-dimensional computer-readable pattern.

26. The method according to any one of claims 13 to 24, wherein the visual representation of the data includes one or more of the following: a Uniform Resource Locator (URL), a session identifier (ID), and a secret key.

27. The method according to any one of claims 13 to 24, wherein the restricted content includes remotely stored content.

28. The method according to any one of claims 13 to 24, wherein the restricted content includes a web page, and accessing the web page requires authentication.

29. The method according to any one of claims 13 to 24, further comprising: In response to receiving the first information, replacing a visual representation of the data displayed on the display of the first electronic device with the first prompt displayed on the display of the first electronic device, wherein replacing the visual representation of the data with the first prompt includes displaying an animated transition from the visual representation of the data to the first prompt.

30. The method according to any one of claims 13 to 24, wherein the second information includes authentication information input at the first electronic device.

31. The method according to claim 30, wherein: The authentication information corresponds to a plurality of characters displayed on a display of a third device, and The third device is associated with the user's user account.

32. The method according to claim 31, wherein the third device is selected from the group consisting of: the first electronic device, the second electronic device, and a device different from the first electronic device and the second electronic device.

33. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first electronic device having a display, the one or more programs including instructions for performing the method according to any one of claims 13 to 32.

34. A first electronic device having a display, comprising: One or more processors; And A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 13 to 32.

35. A first electronic device having a display, the first electronic device comprising: Means for performing the method according to any one of claims 13 to 32.

36. A computer program product comprising one or more programs configured to be executed by one or more processors of a first electronic device having a display, the one or more programs including instructions for performing the method according to any one of claims 13 to 32.

Images