User interface for device communication

Abstract

The basic console user interface includes an indication that the computer system is attempting to connect to one or more external devices. If the connection is successfully established before a first threshold duration is reached, the display updates to the extended console user interface. If the connection fails, the user is presented with the option to switch to the vehicle's native user interface.

Description

Technical Field

[0001] (Cross - reference to Related Applications) This application claims priority to U.S. Provisional Patent Application No. 18 / 605,747, titled "USER INTERFACES FOR DEVICE COMMUNICATIONS," filed on March 14, 2024, and U.S. Patent Application No. 63 / 459,979, titled "USER INTERFACES FOR DEVICE COMMUNICATIONS," filed on April 17, 2023. The entire content of each of these applications is hereby incorporated by reference into this specification.

[0002] This disclosure generally relates to computer user interfaces and, more specifically, to techniques for managing communication with external devices.

Background Art

[0003] A computer system includes a user interface to enable a user to interact with the computer system. The computer system is also connected to other electronic devices. For example, the computer system optionally connects to a user's phone and creates a communication channel that enables the user to initiate and participate in an audio call using the computer system via a call initiated and processed by the phone.

Summary of the Invention

[0004] However, some techniques for using an electronic device to manage communication with an external device are generally cumbersome and inefficient. For example, some existing techniques use complex and time - consuming user interfaces that may include multiple key presses or keystrokes. Existing techniques take more time than necessary, wasting the user's time and the device's energy. This latter consideration is particularly important in battery - operated devices.

[0005] Therefore, this technology provides electronic devices with faster and more efficient methods and interfaces for managing communication with external devices. Such methods and interfaces optionally complement or replace other methods for managing communication with external devices. Such methods and interfaces reduce the cognitive burden on the user and create a more efficient human-machine interface. In the case of battery-operated computing devices, such methods and interfaces conserve power and increase the interval between battery charges.

[0006] A method is disclosed. The method relates to a vehicle computer system, wherein the computer system is communicating with a first display generation component, and via the first display generation component, displays an indication that the computer system is attempting to connect to one or more external devices and a first set of information, without displaying a second set of information; displays a second set of information via the first display generation component in accordance with a determination that a first set of criteria, including connection establishment criteria, is met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached; and displays selectable options via the first display generation component, without displaying a native user interface and without displaying a second set of information, in accordance with a determination that a second set of criteria, including connection failure criteria, is met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached.

[0007] A non-temporary computer-readable storage medium is disclosed. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a vehicle's computer system, the computer system communicates with a first display-generating component, and one or more programs, via the first display-generating component, display an indication that the computer system is attempting to connect to one or more external devices, and a first set of information without displaying a second set of information; via the first display-generating component, displaying a second set of information in accordance with a determination that a first set of criteria, including connection establishment criteria, is met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached; and via the first display-generating component, displaying selectable options for switching to the vehicle's native user interface without displaying a native user interface and without displaying a second set of information, in accordance with a determination that a second set of criteria, including connection failure criteria, is met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached.

[0008] A temporary computer-readable storage medium is disclosed. The temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a vehicle's computer system, and the computer system communicates with a first display generating component, and one or more programs, via the first display generating component, display an indication that the computer system is attempting to connect to one or more external devices, and a first set of information without displaying a second set of information; via the first display generating component, displaying a second set of information in accordance with a determination that a first set of criteria, including connection establishment criteria, is met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached; and via the first display generating component, displaying selectable options for switching to the vehicle's native user interface without displaying a native user interface and without displaying a second set of information, in accordance with a determination that a second set of criteria, including connection failure criteria, is met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached.

[0009] A computer system is disclosed. The computer system is configured to communicate with a first display generation component. The computer system comprises one or more processors and a memory that stores one or more programs configured to run by the one or more processors, the one or more programs including instructions that, via the first display generation component, display an indication that the computer system is attempting to connect to one or more external devices, and a first set of information without displaying a second set of information; that, in accordance with a determination that a first set of criteria is met, which includes connection establishment criteria that are met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached, display a second set of information via the first display generation component, without displaying a native user interface and without displaying a second set of information; and that, in accordance with a determination that a second set of criteria is met, which includes connection failure criteria that are met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached, display selectable options for switching to the vehicle's native user interface via the first display generation component, without displaying a native user interface and without displaying a second set of information.

[0010] A computer system is disclosed. The computer system is configured to communicate with a first display generation component. The computer system includes means for displaying an indication that the computer system is attempting to connect to one or more external devices and a first set of information without displaying a second set of information; means for displaying a second set of information via the first display generation component in accordance with a determination that a first set of criteria is met, which includes connection establishment criteria that are met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached; and means for displaying selectable options for switching to the vehicle's native user interface via the first display generation component, without displaying a native user interface and without displaying a second set of information, in accordance with a determination that a second set of criteria is met, which includes connection failure criteria that are met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached.

[0011] A computer program product is disclosed. The computer program product includes one or more programs configured to run on one or more processors of a vehicle's computer system, the computer system communicating with a first display generating component, and one or more programs including instructions to display, via the first display generating component, an indication that the computer system is attempting to connect to one or more external devices, and a first set of information without displaying a second set of information; to display, via the first display generating component, a second set of information, in accordance with a determination that a first set of criteria, including connection establishment criteria, is met when a connection between the computer system and the external devices is successfully established before a first threshold duration is reached; and to display, via the first display generating component, selectable options for switching to the vehicle's native user interface without displaying a native user interface and without displaying a second set of information, in accordance with a determination that a second set of criteria, including connection failure criteria, is met when a connection between the computer system and the external devices is not successfully established before a second threshold duration is reached.

[0012] The executable instructions that perform these functions are optionally contained within a non-temporary computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0013] Therefore, devices are provided with faster, more efficient methods and interfaces for managing communication with external devices, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can complement or replace other methods for managing communication with external devices.

[0014] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings, and similar reference numbers throughout the following drawings refer to the corresponding parts. [Brief explanation of the drawing]

[0015] [Figure 1A] This is a block diagram showing a portable multifunctional device with a touch-sensitive display, according to several embodiments.

[0016] [Figure 1B] This is a block diagram illustrating exemplary components for event processing according to several embodiments.

[0017] [Figure 2] Several embodiments of a portable multifunctional device having a touchscreen are shown.

[0018] [Figure 3] This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, according to several embodiments.

[0019] [Figure 4A] This document illustrates an exemplary user interface for an application menu on a portable multifunction device, according to several embodiments.

[0020] [Figure 4B] This document illustrates exemplary user interfaces for multifunctional devices having a touch-sensitive surface separate from the display, according to several embodiments.

[0021] [Figure 5A] This document describes personal electronic devices according to several embodiments.

[0022] [Figure 5B]A block diagram showing a personal electronic device according to some embodiments.

[0023] [Figure 6A] An exemplary user interface for managing communication with an external device according to some embodiments is shown. [Figure 6B] An exemplary user interface for managing communication with an external device according to some embodiments is shown. [Figure 6C] An exemplary user interface for managing communication with an external device according to some embodiments is shown. [Figure 6D] An exemplary user interface for managing communication with an external device according to some embodiments is shown. [Figure 6E] An exemplary user interface for managing communication with an external device according to some embodiments is shown. [Figure 6F] An exemplary user interface for managing communication with an external device according to some embodiments is shown.

[0024] [Figure 7] A flowchart showing a method for managing communication with an external device according to some embodiments.

Mode for Carrying Out the Invention

[0025] The following description describes exemplary methods, parameters, etc. However, it should be recognized that such descriptions are not intended as limitations on the scope of the present disclosure, but rather as descriptions of exemplary embodiments.

[0026] There is a need for electronic devices that provide efficient methods and interfaces for managing communication with external devices. For example, a method that enables a computer system to display various user interfaces and information based on its connection to external devices. Such technology can reduce the cognitive burden on users who rely on computer systems communicating with external devices, thereby improving productivity. Furthermore, such technology can reduce the power consumption of the processor and battery that would normally be wasted on redundant user input.

[0027] Figures 1A-1B, 2, 3, 4A-4B, and 5A-5B below provide a description of exemplary devices for performing techniques for managing event notifications. Figures 6A-6F show exemplary user interfaces for managing communication with external devices in several embodiments. Figure 7 is a flowchart showing a method for managing communication with external devices in several embodiments. The user interfaces in Figures 6A-6F are used to illustrate processes described later, including the process in Figure 7.

[0028] The processes described below enhance the usability of the device and streamline the user-device interface by various techniques, including providing users with improved visual feedback, reducing the number of inputs required to perform actions, offering additional control options without cluttering the user interface with additionally displayed controls, performing actions without requiring further user input when a set of conditions is met, and / or other techniques. These techniques also reduce power consumption and improve the device's battery life by enabling users to use the device more quickly and efficiently.

[0029] Furthermore, in any method described herein that is conditional on one or more conditions being met in one or more steps, it should be understood that the method described can be repeated in multiple iterations such that all the conditions that the steps of the method are conditional on are met in different iterations of the method. For example, if a method requires that a first step be performed if a condition is met, and a second step be performed if the condition is not met, a person skilled in the art will understand that the steps described in the claim are repeated in an unspecified order until the conditions are met and then not met. Thus, a method described in one or more steps that depends on one or more conditions being met can be rewritten as a method that is repeated until each of the conditions described in the method is met. However, this is not required for a claim of a system or computer-readable medium that includes instructions that perform a conditional action based on the satisfaction of the corresponding one or more conditions, and thus can determine whether a contingency has been met without explicitly repeating the steps of the method until all the conditions that the steps of the method are conditional on are met. Those skilled in the art will also understand that, as with a method having conditional steps, a system or computer-readable storage medium may repeat the steps of the method as many times as necessary to ensure that all of the conditional steps have been performed.

[0030] In the following description, terms such as “first,” “second,” etc., are used to describe various elements, but these elements should not be limited by these terms. In some embodiments, these terms are used to distinguish one element from another. For example, without departing from the scope of the various embodiments described, the first touch may be called the second touch, and similarly, the second touch may be called the first touch. In some embodiments, the first touch and the second touch are two distinct references to the same touch. In some embodiments, both the first touch and the second touch are touches, but they are not the same touch.

[0031] The terminology used in the descriptions of the various embodiments described herein is intended solely to describe specific embodiments and is not intended to be limiting. In the descriptions of the various embodiments and the accompanying claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless otherwise explicitly stated in the context. Furthermore, it should be understood that, as used herein, the term “and / or” refers to and includes any and all possible combinations of one or more of the enumerated items relating to the invention. It will be further understood that, as used herein, the terms “includes,” “comprises,” and / or “comprising,” specify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0032] The phrase "if" can be interpreted, at will, depending on the context, as "when" or "upon," or "in response to determining" or "in response to detecting." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" can be interpreted, at will, depending on the context, as "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."

[0033] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Optionally, other portable electronic devices such as laptop computers or tablet computers having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad) are also used. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad). In some embodiments, the electronic device is a computer system communicating (e.g., via wired communication, via wireless communication) with a display-generating component. The display-generating component is configured to provide a visual output, such as a display via a CRT display, a display via an LED display, or a display via image projection. In some embodiments, the display-generating component is integrated with the computer system. In some embodiments, the display generation component is separate from the computer system. As used herein, "display" content includes displaying content (e.g., video data rendered or decoded by the display controller 156) by transmitting data (e.g., image data or video data) via a wired or wireless connection to an integrated or external display generation component in order to visually generate the content.

[0034] The following discussion describes electronic devices including displays and touch-sensitive surfaces. However, it should be understood that electronic devices optionally include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

[0035] The device typically supports a variety of applications, including drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0036] Various applications running on this device optionally utilize at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or modified on an application-by-application basis and / or within individual applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with intuitive and transparent user interfaces for the user.

[0037] Here, we turn our attention to embodiments of portable devices equipped with touch-sensitive displays. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112 according to several embodiments. The touch-sensitive display 112 may be referred to for convenience as a “touchscreen” and may be known or referred to as a “touch-sensitive display system”. Device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 (e.g., touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100) that detect the intensity of contact on Device 100. Device 100 optionally includes one or more tactile output generators 167 that generate tactile outputs on Device 100 (for example, on touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0038] As used herein and in the claims, the term “strength” of contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of contact on the touch-sensitive surface (e.g., finger contact), or a proxy for the force or pressure of contact on the touch-sensitive surface. The strength of contact has a range of values, including at least four distinct values, and more typically, including several hundred (e.g., at least 256) distinct values. The strength of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged) to determine the estimated force of contact. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or modification of the contact area detected on the touch-sensing surface, the capacitance and / or modification of the touch-sensing surface adjacent to the contact, and / or the resistance and / or modification of the touch-sensing surface adjacent to the contact may optionally be used as a substitute for the force or pressure of the contact on the touch-sensing surface. In some implementations, the substitute measurement of the contact force or pressure is used directly to determine whether or not an intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some implementations, the substitute measurement of the contact force or pressure is converted into an estimate of the force or pressure, and this estimate is used to determine whether or not an intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). By using the intensity of contact as an attribute of user input, it becomes possible for users to access additional device functions that may otherwise be inaccessible (e.g., on a touch-sensitive display) and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical control such as a knob or button) on reduced-size devices where the implementation area for displaying affordances is limited.

[0039] As used herein and in the claims, the term “tactile output” means a physical displacement of the device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by the user through the user’s sense of touch. For example, in a situation where the device or a component of the device is in contact with the touch-sensitive surface of the user (e.g., the user’s fingers, palm, or other part of their hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) may be optionally interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may feel a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user’s action. In another embodiment, movement of a touch-sensitive surface may be optionally interpreted or perceived by the user as "roughness" of the touch-sensitive surface, even if there is no change in the smoothness of the touch-sensitive surface. Such user interpretations of touch depend on the user's personal sensory perception, but there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.

[0040] It should be understood that device 100 is merely an example of a portable multifunction device, and that device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of those components. The various components shown in Figure 1A are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application-specific integrated circuits.

[0041] Memory 102 optionally includes high-speed random-access memory and optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0042] The peripheral interface 118 can be used to connect the device's input and output peripherals to the CPU 120 and memory 102. One or more processors 120 operate or execute various software programs (such as computer programs (including instructions)) and / or instruction sets stored in memory 102 to perform various functions for device 100 and process data. In some embodiments, the peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0043] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, which include, but are not limited to, antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets, subscriber identity module (SIM) cards, and memory. The RF circuit 108 optionally communicates wirelessly with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), as well as with other devices. The RF circuit 108 optionally includes a well-known circuit for detecting a near-field communication (NFC) field using a short-range communication radio. Wireless communication is not limited to this, but optionally includes Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), and long-term evolution.Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, Email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant messaging (e.g., Extensible Messaging and Presence Protocol) Using any of several communication standards, protocols, and technologies, including the XMPP protocol, the Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), the Instant Messaging and Presence Service (IMPS), and / or the Short Message Service (SMS), or any other suitable communication protocol, including a communication protocol not yet developed as of the filing date of this specification.

[0044] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts this audio data into an electrical signal, and transmits this electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits this audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or RF circuit 108 by the peripheral interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212 in Figure 2). The headset jack provides an interface between the audio circuit 110 and detachable audio input / output peripherals such as output-only headphones or headsets that have both output (e.g., headphones for one or both ears) and input (e.g., a microphone).

[0045] The I / O subsystem 106 connects input / output peripherals on device 100, such as the touchscreen 112 and other input control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, a depth camera controller 169, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from / transmit electrical signals to other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some embodiments, one or more input controllers 160 are optionally connected to (or not connected to) one of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. One or more buttons (e.g., 208 in Figure 2) optionally include up / down buttons for volume control of speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (e.g., 206 in Figure 2). In some embodiments, the electronic device is a computer system communicating with one or more input devices (e.g., via wireless communication over wired communication). In some embodiments, one or more input devices include a touch-sensitive surface (e.g., a trackpad as part of a touch-sensitive display). In some embodiments, one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), for example, to track user gestures as input (e.g., hand gestures and / or air gestures). In some embodiments, one or more input devices are integrated with the computer system. In some embodiments, one or more input devices are separate from the computer system.In some embodiments, an air gesture is a gesture detected without the user touching (or independently of) an input element that is part of the device, and is based on detected movement of a part of the user's body in the air, including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground, or the distance of the user's hand relative to the ground), movement of the user's body relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of the user's other hand relative to one hand, and / or movement of the user's fingers relative to another finger or part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tap gesture involving movement of the hand in a predetermined pose by a predetermined amount and / or speed, or a shake gesture involving rotation of a part of the user's body by a predetermined speed or amount).

[0046] As described in U.S. Patent Application No. 11 / 322,549, “Unlocking a Device by Performing Gestures on an Unlock Image,” filed December 23, 2005, U.S. Patent No. 7,657,849, which is incorporated herein by reference in its entirety, a quick press of a push button optionally releases the lock on the touchscreen 112, or optionally initiates a process to unlock the device using gestures on the touchscreen. A longer press of a push button (e.g., 206) optionally turns power on or off the device 100. The functionality of one or more of the buttons is optionally customizable by the user. The touchscreen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

[0047] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. This visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively, “graphics”). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0048] The touchscreen 112 has a touch-sensing surface, sensor, or set of sensors that accept user input based on touch and / or tactile contact. The touchscreen 112 and the display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of contact) on the touchscreen 112 and translate the detected contact into interaction with user interface objects displayed on the touchscreen 112 (e.g., one or more soft keys, icons, web pages, or images). In an exemplary embodiment, the point of contact between the touchscreen 112 and the user corresponds to the user's finger.

[0049] The touchscreen 112 optionally uses LCD (liquid crystal display) technology, LPD (polymer light-emitting display) technology, or LED (light-emitting diode) technology, but other display technologies may also be used in other embodiments. The touchscreen 112 and the display controller 156 optionally, but not limited to, use any of a number of currently known or future-developed touch sensing technologies, including capacitive, resistive, infrared, and surface acoustic technologies, as well as other proximity sensor arrays or other elements that determine one or more points of contact with the touchscreen 112, to detect contact and any movement or interruption thereof. In exemplary embodiments, projected mutual capacitive sensing technology is used, such as that found in the iPhone® and iPod Touch® from Apple Inc. of Cupertino, California.

[0050] The touch-sensitive displays in some embodiments of the touchscreen 112 are optionally similar to the multi-touch-sensitive touchpads described in U.S. Patent No. 6,323,846 (Westerman et al.), No. 6,570,557 (Westerman et al.), and / or No. 6,677,932 (Westerman), and / or U.S. Patent Application Publication 2002 / 0015024(A1), which are each incorporated herein by reference in their entirety. However, the touchscreen 112 displays visual output from device 100, whereas the touch-sensitive touchpad does not provide visual output.

[0051] The touch-sensitive displays in some embodiments of the touchscreen 112 are described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, "Multipoint Touch Surface Controller"; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, "Multipoint Touchscreen"; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, "Gestures For Touch Sensitive Input Devices"; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, "Gestures For Touch Sensitive Input Devices"; (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, "Mode-Based Graphical User Interfaces For Touch Sensitive Input These are described in (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, "Virtual Input Device Placement On A Touch Screen User Interface", (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, "Operation Of A Computer With A Touch Screen Interface", (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard", and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, "Multi-Functional Hand-Held Device". All of these applications are incorporated herein by reference in their entirety.

[0052] The touchscreen 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. The user optionally touches the touchscreen 112 using any suitable object or attachment such as a stylus or finger. In some embodiments, the user interface is designed to operate primarily using finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​the finger on the touchscreen. In some embodiments, the device translates coarse finger input into a precise pointer / cursor position or command to perform an action desired by the user.

[0053] In some embodiments, in addition to the touchscreen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touchscreen 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0054] Device 100 also includes a power system 162 that supplies power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with generating, managing, and distributing power within the portable device.

[0055] The device 100 also optionally includes one or more optical sensors 164. Figure 1A shows an optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light from the environment projected through one or more lenses and converts that light into data representing an image. The optical sensor 164 works in conjunction with an imaging module 143 (also called a camera module) to optionally capture still images or video. In some embodiments, the optical sensor is located on the back of the device 100 opposite the touchscreen display 112 on the front of the device, so that the touchscreen display can be used as a viewfinder for acquiring still images and / or video images. In some embodiments, the optical sensor is located on the front of the device so that the user's image is optionally acquired for video conferencing while the user is viewing other video conferencing participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (for example, by rotating the lens and sensor within the device housing), so that a single optical sensor 164 can be used for both video conferencing and the acquisition of still and / or video images, together with the touchscreen display.

[0056] Device 100 also optionally includes one or more depth camera sensors 175. Figure 1A shows a depth camera sensor coupled to a depth camera controller 169 in the I / O subsystem 106. The depth camera sensor 175 receives data from the environment to create a three-dimensional model of an object in the scene (e.g., a face) from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with an imaging module 143 (also called a camera module), the depth camera sensor 175 is optionally used to determine the depth map of different parts of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is positioned on the front of Device 100 to optionally acquire an image of the user with depth information for video conferencing while the user is viewing other video conferencing participants on a touchscreen display, and also to capture a selfie image with depth map data. In some embodiments, the depth camera sensor 175 is positioned on the back of the device, or on both the back and front of Device 100. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (for example, by rotating the lens and sensor within the device housing), so that the depth camera sensor 175, together with the touchscreen display, can be used for both video conferencing and the acquisition of still and / or video images.

[0057] Device 100 also optionally includes one or more contact intensity sensors 165. Figure 1A shows a contact intensity sensor coupled to an intensity sensor controller 159 in the I / O subsystem 106. The contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact intensity sensor 165 receives contact intensity information (e.g., pressure information, or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is positioned juxtaposed with or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112). In some embodiments, at least one contact intensity sensor is located on the back of Device 100, opposite the touchscreen display 112 located on the front of Device 100.

[0058] Furthermore, device 100 optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is optionally coupled to an input controller 160 in the I / O subsystem 106. The proximity sensor 166 optionally functions as described in U.S. Patent Applications 11 / 241,839, “Proximity Detector In Handheld Device,” 11 / 240,788, “Proximity Detector In Handheld Device,” 11 / 620,702, “Using Ambient Light Sensor To Augment Proximity Sensor Output,” 11 / 586,862, “Automated Response To And Sensing Of User Activity In Portable Devices,” and 11 / 638,251, “Methods And Systems For Automatic Configuration Of Peripherals,” which are all incorporated herein by reference. In some embodiments, if the multifunction device is placed near the user's ear (for example, when the user is making a phone call), the proximity sensor turns off and disables the touchscreen 112.

[0059] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices such as a speaker or other audio component, and / or electromechanical devices that convert energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). The contact intensity sensor 165 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on device 100 that can be sensed by the user of device 100. In some embodiments, at least one tactile output generator is positioned alongside or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112) and optionally generates a tactile output by moving the touch-sensing surface vertically (e.g., inward / outward from the surface of device 100) or horizontally (e.g., forward / backward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite the touchscreen display 112 located on the front of device 100.

[0060] The device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. The accelerometer 168 optionally functions as described in U.S. Patent Application Publication 20050190059, "Acceleration-based Theft Detection System for Portable Electronic Devices," and U.S. Patent Application Publication 20060017692, "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape orientation based on an analysis of data received from one or more accelerometers. Device 100 optionally includes, in addition to one or more accelerometers 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information regarding the location and orientation of Device 100 (e.g., longitudinal or transverse).

[0061] In some embodiments, the software components stored in memory 102 include an operating system 126, a communications module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 (Figure 1A) or 370 (Figure 3) stores device / global internal state 157, as shown in Figures 1A and 3. The device / global internal state 157 includes one or more of the following: active application state, indicating which application is active if there is an application currently active; display state, indicating which application, view, or other information occupies various areas of the touchscreen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and location information relating to the device's location and / or orientation.

[0062] An operating system 126 (for example, an embedded operating system such as Darwin, RTXC, LINUX, UNIX®, OS X, iOS, WINDOWS®, or VxWorks) includes various software components and / or drivers that control and manage general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitate communication between various hardware components and software components.

[0063] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire®, etc.) are adapted to connect to other devices directly or indirectly via a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors that are the same as and / or compatible with the 30-pin connector used on iPod® (a trademark of Apple Inc.) devices.

[0064] The contact / motion module 130 optionally detects contact with the touchscreen 112 and other touch-sensitive devices (e.g., a touchpad or physical click wheel) (in cooperation with the display controller 156). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger down event), determining the intensity of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more events of a finger dragging), and determining whether contact has been terminated (e.g., detecting a finger up event or interruption of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point, represented by a series of contact data, optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions can be optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0065] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (for example, to determine whether a user has "clicked" on an icon). In some embodiments, at least a subset of the intensity thresholds is determined according to software parameters (for example, the intensity thresholds can be adjusted without modifying the physical hardware of device 100, rather than being determined by the activation threshold of a particular physical actuator). For example, the mouse "click" threshold for a trackpad or touchscreen display can be set to one of a range of default thresholds without modifying the trackpad or touchscreen display hardware. In addition, some implementations provide the user of the device with software settings to adjust one or more of the set of intensity thresholds (for example, by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using a system-level click "intensity" parameter).

[0066] The contact / motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motion, timing, and / or intensity of the detected contact). Therefore, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture involves detecting a finger down event, followed by a finger up (lift-off) event at the same position (or substantially the same position) as the finger down event (e.g., the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger down event, followed by one or more finger drag events, and then a finger up (lift-off) event.

[0067] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components that modify the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). In this specification, the term “graphics” includes, but is not limited to, any object that can be displayed to the user, including characters, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0068] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, as needed, specifying the graphics to be displayed, along with coordinate data and other graphic property data, and then generates screen image data to output to the display controller 156.

[0069] The haptic feedback module 133 includes various software components for generating commands used by a tactile output generator(s) 167, and generates tactile outputs at one or more locations on the device 100 in response to the user's interaction with the device 100.

[0070] The text input module 134 is optionally a component of the graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other applications that require text input).

[0071] The GPS module 135 determines the device's location and provides this information for use in various applications (for example, to the phone 138 for use in location-based dialing, to the camera 143 as picture / video metadata, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0072] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: • Contact module 137 (sometimes called the address book or contact list), • Telephone module 138, • Video conferencing module 139, • Email client module 140, • Instant messaging (IM) module 141, • Training support module 142, • Camera module 143 for still images and / or video images, Image management module 144, • Video player module, • Music player module, • Browser module 147, Calendar module 148, A widget module 149 optionally includes one or more of the following: weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, other widgets obtained by the user, and user-created widgets 149-6. • Widget creator module 150 for creating user-created widgets 149-6. • Search module 151, • Video and music player module 152, which integrates a video player module and a music player module. • Memo module 153, • Map module 154, and / or, • Online video module 155.

[0073] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java®-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0074] Together with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (stored, for example, in the application internal state 192 of the contact module 137 in memory 102 or memory 370), which includes adding names(s) to the address book, deleting names(s) from the address book, associating telephone numbers(s) to names, email addresses(s) to names, addresses(s) to names, or other information, associating images to names, categorizing and sorting names, and providing telephone numbers or email addresses to initiate and / or facilitate communication via telephone 138, video conferencing module 139, email 140, or IM 141.

[0075] The telephone module 138 works in conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134 to optionally input character sequences corresponding to telephone numbers, access one or more telephone numbers in the contact module 137, modify entered telephone numbers, dial individual telephone numbers, make calls, and disconnect and terminate calls at the end of a call. As previously mentioned, wireless communication may optionally use any of several communication standards, protocols, and technologies.

[0076] The video conferencing module 139 works in conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138 to include executable commands for starting, running, and ending video conferences between the user and one or more other participants in accordance with the user's commands.

[0077] The email client module 140, in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, includes executable commands for creating, sending, receiving, and managing emails in response to user commands. In conjunction with the image management module 144, the email client module 140 makes it extremely easy to create and send emails containing still or video images captured by the camera module 143.

[0078] The instant messaging module 141, in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, includes executable commands for inputting character sequences corresponding to instant messages, modifying previously entered characters, sending individual instant messages (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or Enhanced Messaging Service (EMS). In this specification, “instant messaging” refers to both telephone-based messaging (e.g., messages sent using SMS or MMS) and internet-based messaging (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0079] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, the training support module 142 includes executable commands, which create training (e.g., having time, distance, and / or calorie burn goals), communicate with training sensors (sports devices), receive training sensor data, calibrate sensors used to monitor training, select and play music for training, and display, store, and transmit training data.

[0080] The camera module 143 works in conjunction with the touchscreen 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144 to include executable commands for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0081] The image management module 144 works in conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143 to include executable commands for arranging, modifying (e.g., editing), or otherwise manipulating still and / or video images, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing them.

[0082] The browser module 147 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134 to include executable commands for browsing the internet according to user commands, including searching, linking, receiving, and displaying web pages or parts thereof, as well as attachments and other files linked to web pages.

[0083] The calendar module 148 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147 to include executable commands for creating, displaying, modifying, and storing a calendar and data associated with the calendar (e.g., calendar entries, to-do lists, etc.) in accordance with user commands.

[0084] The widget module 149 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147 to optionally download and use mini-applications (e.g., weather widget 149-1, stock price widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or mini-applications created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widgets®).

[0085] The widget creator module 150 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147 to be used by the user to optionally create widgets (for example, to turn a user-specified portion of a web page into a widget).

[0086] The search module 151 works in conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134 to include executable commands for searching for characters, music, sounds, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user commands.

[0087] The video and music player module 152 works in conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147 to include executable commands that allow the user to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable commands for displaying, presenting, or otherwise playing video (for example, on the touchscreen 112 or on an external display connected via the external port 124). In some embodiments, the device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0088] The memo module 153 works in conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134 to include executable commands for creating and managing memos, to-do lists, etc., according to user commands.

[0089] The map module 154 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147 to optionally receive, display, modify, and store maps and map-related data (e.g., driving directions, data on shops and other points of interest at a specific location or nearby, and other location-based data) in accordance with user commands.

[0090] The online video module 155, in conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, includes instructions that enable the user to access a specific online video, browse a specific online video, receive it (e.g., by streaming and / or downloading), play it (e.g., on the touchscreen or on an external display connected via external port 124), send an email with a link to a specific online video, and perform other management of online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send a link to a specific online video. Further descriptions of online video applications can be found in U.S. Provisional Patent Application No. 60 / 936,562, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed June 20, 2007, and U.S. Patent Application No. 11 / 968,067, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed December 31, 2007, the entirety of which is incorporated herein by reference.

[0091] Each of the modules and applications identified above corresponds to a set of executable instructions that perform one or more of the functions described above and the methods described in this application (e.g., the computer methods and other information processing methods described herein). These modules (e.g., instruction sets) do not need to be implemented as separate software programs (e.g., computer programs containing instructions), procedures, or modules, and therefore in various embodiments, various subsets of these modules are optionally combined or otherwise reconfigured. For example, a video player module is optionally combined with a music player module to form a single module (e.g., the video and music player module 152 in Figure 1A). In some embodiments, memory 102 optionally stores subsets of the modules and data structures identified above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0092] In some embodiments, device 100 is a device in which the operation of a default set of functions in the device is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for device 100 to operate, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0093] A default set of functions, which are performed only through the touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0094] Figure 1B is a block diagram showing exemplary components for event processing according to several embodiments. In some embodiments, memory 102 (Figure 1A) or 370 (Figure 3) includes an event sorting unit 170 (e.g., within the operating system 126) and individual applications 136-1 (e.g., any of the aforementioned applications 137-151, 155, 380-390).

[0095] The event sorting unit 170 receives event information and determines the application 136-1 that distributes the event information, and the application view 191 of application 136-1. The event sorting unit 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view(s) displayed on the touch-sensitive display 112 when the application is active or running. In some embodiments, a device / global internal state 157 is used by the event sorting unit 170 to determine which application(s) are currently active, and the application internal state 192 is used by the event sorting unit 170 to determine the application view(s) to which the event information is distributed.

[0096] In some embodiments, the application internal state 192 includes additional information such as resume information to be used when the application 136-1 resumes execution, user interface state information that indicates or is ready to display information displayed by the application 136-1, a state queue that allows the user to return to a previous state or view of the application 136-1, and one or more redo / undo queues of previous actions performed by the user.

[0097] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, user touch as part of a multi-touch gesture on the touch-sensitive display 112). The peripheral interface 118 transmits information received from the I / O subsystem 106, or from sensors such as the proximity sensor 166, one or more accelerometers 168, and / or the microphone 113 (via the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display 112 or the touch-sensitive surface.

[0098] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when there is a significant event (e.g., reception of input exceeding a predetermined noise threshold and / or exceeding a predetermined duration).

[0099] In some embodiments, the event sorting unit 170 also includes a hit view determination module 172 and / or an active event recognition determination module 173.

[0100] The hit view determination module 172 provides a software procedure for determining where in one or more views a sub-event occurred when the touch-sensitive display 112 is displaying two or more views. A view consists of control devices and other elements that the user can see on the display.

[0101] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of an individual application) in which a touch is detected optionally corresponds to the program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected optionally refers to a hit view, and the set of events recognized as appropriate input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch gesture.

[0102] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views arranged in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy from which sub-events should be processed. In most situations, the hit view is the lowest-level view from which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source identified as the hit view.

[0103] The active event recognition determination module 173 determines which view(s) in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-event, are actively involved views, and therefore all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is entirely confined to an area associated with one particular view, higher-level views in the hierarchy still remain actively involved views.

[0104] The event dispatcher module 174 dispatches event information to an event recognition unit (e.g., an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores event information retrieved by individual event receiving units 182 in an event queue.

[0105] In some embodiments, the operating system 126 includes an event sorting unit 170. Alternatively, application 136-1 includes an event sorting unit 170. In yet another embodiment, the event sorting unit 170 is a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.

[0106] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for handling touch events occurring within a separate view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Typically, a separate application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a separate event processing unit 190 includes one or more event data 179 received from a data update unit 176, an object update unit 177, a GUI update unit 178, and / or an event sorting unit 170. The event processing unit 190 optionally uses or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Alternatively, one or more application views 191 include one or more event processing units 190. In some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in individual application views 191.

[0107] Each individual event recognition unit 180 receives event information (e.g., event data 179) from the event sorting unit 170 and identifies events from the event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes at least a subset of metadata 183 and event distribution commands 188 (optionally including sub-event distribution commands).

[0108] The event receiving unit 182 receives event information from the event sorting unit 170. The event information includes information about sub-events, such as touches or the movement of touches. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes corresponding information about the device's current orientation (also called the device's orientation).

[0109] The event comparison unit 184 compares event information with a definition of a default event or sub-event, and based on the comparison, determines the event or sub-event, or determines or updates the state of the event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes definitions of events (e.g., a default sequence of default sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, sub-events within an event (187-1 and / or 187-2) include, for example, a touch start, a touch end, a touch movement, a touch cancellation, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch on the displayed object for a predetermined stage (touch start), a first lift-off for the predetermined stage (touch end), a second touch on the displayed object for the predetermined stage (touch start), and a second lift-off for the predetermined stage (touch end). In another example, the definition of event 2(187-2) is a drag on a displayed object. A drag includes, for example, a touch (or contact) on the displayed object to a predetermined stage, movement of the touch across the touch-sensitive display 112, and lift-off of the touch (end of touch). In some embodiments, the event also includes information about one or more associated event processing units 190.

[0110] In some embodiments, the event definition 186 includes event definitions for individual user interface objects. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display 112, when a touch is detected on the touch-sensitive display 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with an individual event processing unit 190, the event comparison unit uses the results of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects the sub-event and the event processing unit associated with the object that triggers the hit test.

[0111] In some embodiments, the definition of an individual event 187 also includes a delay action that delays the delivery of event information until it is determined whether the sequence of sub-events corresponds to the event type of the event recognition unit.

[0112] If an individual event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, the individual event recognition unit 180 enters an event impossible, event failed, or event terminated state, and thereafter ignores subsequent sub-events of the touch-based gesture. In this situation, if there are other event recognition units that remain active for the hit view, those event recognition units continue to track and process the sub-events of the ongoing touch-based gesture.

[0113] In some embodiments, an individual event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition unit how the event distribution system should perform sub-event distribution. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how the event recognition units interact with each other, or how they can interact with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.

[0114] In some embodiments, an individual event recognition unit 180 activates an event processing unit 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, the individual event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processing unit 190 is separate from sending (and delaying the sending of) sub-events to individual hit views. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event processing unit 190 associated with that flag captures the flag and executes a default process.

[0115] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event processing unit. Instead, the sub-event distribution command distributes event information to an event processing unit associated with a set of sub-events, or to an actively involved view. The event processing unit associated with the set of sub-events or the actively involved view receives the event information and executes a predetermined process.

[0116] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video player module. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the positions of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.

[0117] In some embodiments, the event processing unit(s) 190 includes or has access to a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are contained in a single module of an individual application 136-1 or application view 191. In other embodiments, they are contained in two or more software modules.

[0118] The foregoing description regarding the handling of user touch events on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using input devices, but it should be understood that not all of these begin on the touchscreen. For example, mouse movement and mouse button presses, touch movements such as taps, drags, and scrolls on a touchpad, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof may be optionally used as inputs corresponding to sub-events that define the events to be recognized.

[0119] Figure 2 shows a portable multifunction device 100 having a touchscreen 112 according to several embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment, and in other embodiments described below, the user can select one or more of the graphics by performing gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics is performed when the user interrupts contact with that one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and / or downward) and / or rolling (from right to left, left to right, upward and / or downward) with a finger in contact with the device 100. In some implementations or situations, accidental contact with a graphic does not constitute a selection of that graphic. For example, if the gesture corresponding to selection is a tap, a swipe gesture sweeping over an application icon does not arbitrarily select the corresponding application.

[0120] Device 100 also optionally includes one or more physical buttons, such as a "Home" button or a menu button 204. As previously mentioned, the menu button 204 is optionally used to navigate to any application 136 within the set of applications running on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touchscreen 112.

[0121] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push button 206 for turning the device on / off and locking the device, one or more volume buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external port 124 for docking / charging. The push button 206 is optionally used to turn the device on / off by pressing down and holding the button down for a predetermined period of time, to lock the device by pressing down and releasing the button before a predetermined period of time has elapsed, and / or to unlock the device or initiate an unlocking process. In alternative embodiments, device 100 also accepts verbal input via a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.

[0122] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to several embodiments. Device 300 does not need to be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 that interconnect these components. The communication buses 320 optionally include circuitry (sometimes referred to as a chipset) that interconnects and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 including a display 340, the display 340 is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 that generates tactile output on device 300 (for example, similar to the tactile output generator(s) 167 described above with reference to Figure 1A), and a sensor 359 (for example, light, acceleration, proximity, touch sensing, and / or a contact intensity sensor similar to the contact intensity sensor(s) 165 described above with reference to Figure 1A). The memory 370 includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310.In some embodiments, memory 370 stores programs, modules, and data structures similar to, or subsets thereof, that are stored in memory 102 of the portable multifunction device 100 (Figure 1A). Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of the portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk authoring module 388, and / or a spreadsheet module 390, whereas memory 102 of the portable multifunction device 100 (Figure 1A) optionally does not store these modules.

[0123] Each of the elements identified above in Figure 3 is optionally stored in one or more of the memory devices described above. Each of the modules identified above corresponds to an instruction set that performs the function described above. The modules or computer programs (e.g., instruction sets or instructions) identified above do not need to be implemented as separate software programs (e.g., computer programs (including instructions)), procedures, or modules, and therefore in various embodiments, various subsets of these modules are optionally combined or otherwise reconfigured. In some embodiments, memory 370 optionally stores subsets of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0124] Next, we optionally turn our attention to an embodiment of a user interface implemented in, for example, a portable multi-functional device 100.

[0125] Figure 4A shows an exemplary user interface for an application menu on a portable multifunction device 100 according to several embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or subsets or supersets thereof. ● Signal strength indicators (single or multiple) for wireless communication (single or multiple) such as cellular signals and Wi-Fi signals 402, ●Time 404, ●Bluetooth indicator 405, ●Battery status indicator 406, ●Tray 408 containing icons for frequently used applications, as shown below. ○Optionally including an indicator 414 for the number of missed calls or voicemail messages, an icon 416 of the telephone module 138 labeled "Telephone", ○Optionally including an indicator 410 for the number of unread emails, an icon 418 of the email client module 140 labeled "Mail", ○ Icon 420 of browser module 147, labeled "Browser", and ○ Icon 422 for the video and music player module 152, also known as the iPod (trademark of Apple Inc.) module 152, which is labeled "iPod", and ● Icons of other applications, such as the following: ○ Icon 424 of IM module 141, labeled "Message", ○ Icon 426 of calendar module 148, labeled "Calendar", ○ Icon 428 of image management module 144, labeled "Photo" ○ Icon 430 of camera module 143, labeled "Camera" ○ Icon 432 of online video module 155, labeled "online video" ○ Icon 434 of stock price widget 149-2, labeled "Stock Price" ○ Icon 436 of map module 154, labeled "Map" ○ Icon 438 of weather widget 149-1, labeled "Weather" ○ Icon 440 of the alarm clock widget 149-4, labeled "Clock" ○ Icon 442 of training support module 142, labeled "Training Support" ○ Icon 444 of memo module 153, labeled as "Memo", and ○ An icon 446 labeled "Settings," which provides access to the settings of device 100 and its various applications 136, for a settings application or module.

[0126] Please note that the icon labels shown in Figure 4A are for illustrative purposes only. For example, the icon 422 for the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label for an individual application icon includes the name of the application to which that individual application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds.

[0127] Figure 4B shows an exemplary user interface on a device (e.g., device 300 in Figure 3) having a touch-sensitive surface 451 (e.g., tablet or touchpad 355 in Figure 3) separate from the display 450 (e.g., touchscreen display 112). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of contact on the touch-sensitive surface 451, and / or one or more tactile output generators 357 for generating tactile output to the user of device 300.

[0128] Some of the following embodiments are given by reference to input on a touchscreen display 112 (a combination of a touch-sensing surface and a display), but in some embodiments, the device detects input on a touch-sensing surface separate from the display, as shown in Figure 4B. In some embodiments, the touch-sensing surface (e.g., 451 in Figure 4B) has a primary axis (e.g., 452 in Figure 4B) corresponding to a primary axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to these embodiments, the device detects contact with the touch-sensing surface 451 (e.g., 460 and 462 in Figure 4B) at locations corresponding to each location on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in Figure 4B). In this way, user input (e.g., touches 460 and 462, and their movement) detected by the device on a touch-sensitive surface (e.g., 451 in Figure 4B) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B) when the touch-sensitive surface is separate from the display. It should be understood that a similar method may be optionally used for other user interfaces described herein.

[0129] In addition, while the following examples are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture), it should be understood that in some embodiments, one or more of the finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of touch), followed by a mouse click with cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may optionally be replaced by a mouse click (e.g., instead of touch detection and subsequent cessation of touch detection) while the cursor is located over the tap gesture location. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0130] Figure 5A shows an exemplary personal electronic device 500. Device 500 includes a body 502. In some embodiments, device 500 may include some or all of the functions described with respect to devices 100 and 300 (e.g., Figures 1A to 4B). In some embodiments, device 500 has a touch-sensitive display screen 504, hereafter referred to as touchscreen 504. Alternatively, or in addition to touchscreen 504, device 500 may have a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors that detect the intensity of the applied contact (e.g., touch). One or more intensity sensors on touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of device 500 may respond to touches based on their intensity, meaning that touches of different intensity may invoke different user interface behaviors on device 500.

[0131] For example, see, for instance, International Patent Application PCT / US2013 / 040061, “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” filed 8 May 2013, published as International Publication WO / 2013 / 169849, and International Patent Application PCT / US2013 / 069483, “Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships,” filed 11 November 2013, published as International Publication WO / 2014 / 105276.

[0132] In some embodiments, the device 500 has one or more input mechanisms 506 and 508. The input mechanisms 506 and 508 may be physical, if included. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, the device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, can allow the device 500 to be attached to, for example, hats, eyeglasses, earrings, necklaces, shirts, jackets, bracelets, watch bands, chains, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow the user to wear the device 500.

[0133] Figure 5B shows an exemplary personal electronic device 500. In some embodiments, the device 500 may include some or all of the components described with respect to Figures 1A, 1B, and 3. The device 500 has a bus 512 that operably connects an I / O section 514 to one or more computer processors 516 and memory 518. The I / O section 514 may be connected to a display 504, which may have a touch-sensing component 522 and optionally an intensity sensor 524 (e.g., a contact intensity sensor). In addition, the I / O section 514 may be connected to a communication unit 530 that receives application and operating system data using Wi-Fi, Bluetooth, near-field communication (NFC), cellular, and / or other wireless communication technologies. The device 500 may include input mechanisms 506 and / or 508. The input mechanism 506 may optionally be, for example, a rotatable input device or a pressable and rotatable input device. In some embodiments, the input mechanism 508 is optionally a button.

[0134] In some embodiments, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors such as a GPS sensor 532, an accelerometer 534, a direction sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or a combination thereof, all of which can be operably connected to the I / O section 514.

[0135] The memory 518 of the personal electronic device 500 may include one or more non-temporary computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, can cause the computer processors to perform techniques described later, including, for example, processing 700 (Figure 7). The computer-readable storage media may be any medium that can tangibly contain or store computer-executable instructions used by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a temporary computer-readable storage medium. In some examples, the storage medium is a non-temporary computer-readable storage medium. The non-temporary computer-readable storage medium may include, but is not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, CDs, DVDs, or optical disks based on Blu-ray® technology, as well as persistent solid-state memories such as flash and solid-state drives. The personal electronic device 500 is not limited to the components and configurations shown in Figure 5B, and may include other or additional components in multiple configurations.

[0136] As used herein, the term “affordance” refers to user interaction graphical user interface objects that are optionally displayed on the display screens of devices 100, 300, and / or 500 (Figures 1A, 3, and 5A-5B). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute an affordance.

[0137] As used herein, the term “focus selector” refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations, including a cursor or other location marker, the cursor acts as a “focus selector,” and therefore, when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3 or touch-sensitive surface 451 in Figure 4B) while the cursor is positioned over a particular user interface element, the particular user interface element is adjusted according to the detected input. In some implementations, including a touchscreen display that allows direct interaction with user interface elements on the touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A or touchscreen 112 in Figure 4A), a detected contact on the touchscreen acts as a “focus selector,” and therefore, when an input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of ​​the user interface to another without corresponding cursor movement or touch movement on the touchscreen display (for example, by using the tab key or arrow keys to move focus from one button to another), and in these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is generally a user interface element (or touch on the touchscreen display) controlled by the user to communicate the user's intended interaction with the user interface (for example, by pointing to the device an element of the user interface through which the user intends to interact).For example, the location of a focus selector (e.g., cursor, touch, or selection box) over an individual button while pressure input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that individual button (rather than other user interface elements displayed on the device's display).

[0138] As used herein and in the claims, the term “characteristic intensity” of a contact refers to the characteristics of that contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a set of intensity samples collected over a predetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) associated with a predetermined event (e.g., after detection of contact, before detection of lift-off of contact, before or after detection of the start of movement of contact, before detection of the end of contact, before or after detection of an increase in contact intensity, and / or before or after detection of a decrease in contact intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the top 10 percentile value of the contact intensity, the maximum half value of the contact intensity, the maximum 90 percent value of the contact intensity, and so on. In some embodiments, the duration of contact is used when determining characteristic intensity (for example, when characteristic intensity is the average intensity of contact over time). In some embodiments, characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action has been performed by the user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this embodiment, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity above the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity above the second threshold results in a third action. In some embodiments, the comparison between characteristic intensity and one or more thresholds is not used to determine whether a first action should be performed or a second action should be performed, but rather to determine whether one or more actions should be performed at all (for example, whether individual actions should be performed or whether individual actions should be withheld).

[0139] Next, we will focus on embodiments of user interfaces ("UI") and related processes implemented on electronic devices such as portable multifunction device 100, device 300, or device 500.

[0140] Figures 6A to 6F show exemplary user interfaces for managing communication with external devices in several embodiments. These user interfaces are used to illustrate processes described later, including the process shown in Figure 7.

[0141] Figure 6A shows a computer system 600 including a first display 602 and a second display 604. In some embodiments, the computer system 600 is part of the vehicle, and the first display 602 and the second display 604 are integrated into and / or installed in the vehicle. In some embodiments, the first display 602 is a display installed in the vehicle's front console (e.g., center console) and / or positioned in the console between the driver's seat and the passenger seat. In some embodiments, the first display 602 is part of an infotainment display. In some embodiments, the second display 604 is a display installed in the vehicle's front console and / or positioned in front of the driver's seat of the vehicle. In some embodiments, the second display 604 is an instrument cluster display. In some embodiments, the first display 602 and / or the second display 604 are touchscreen displays. In some embodiments, the computer system 600 includes one or more features of devices 100, 300, and / or 500.

[0142] In Figure 6A, the computer system 600 is off or in low-power mode. For example, the first display 602 and the second display 604 are off or in low-power mode, for example, when the vehicle is turned off, locked, or in low-power mode. In Figure 6A, the computer system detects user input (corresponding to, for example, a user accessing the vehicle, a user entering the vehicle, the vehicle starting up, and / or a user attempting to access the computer system 600) (for example, while the computer system 600 is not connected to the user's phone), and accordingly, the computer system 600 turns on the first display 602 and / or the second display 604 (or transitions from low-power mode (for example, to standard power mode)) as shown in Figure 6B, and the computer system 600 attempts to connect to a phone (one or more phones) paired with the computer system 600 (for example, a phone authorized to connect to the computer system 600).

[0143] In Figure 6B, the computer system 600 displays, via the first display 602, an indication 610A that the computer system 600 is attempting to connect to the user's phone, an image of a smartphone 610B (to indicate that the computer system 600 is attempting to connect to the phone), and a basic console user interface 610 including the vehicle's interior environment controls 612. The interior environment controls 612 are selectable options that provide the user with visual feedback regarding the vehicle's interior environment and allow the user to control the interior environment. The air temperature control 612A, when activated, changes (increases / decreases) the target interior temperature. The seat temperature control 612B, when activated, changes (turns on / off and / or increases / decreases) the heating and / or cooling settings for the driver's and passenger's seats. The air circulation control 612C, when activated, turns the circulation of interior air on or off. The windshield defrost button 612D and the rear window defrost button 612E, when activated, turn the defrosting of the individual windows on or off. Therefore, while the computer system 600 is attempting to connect to the user's phone, the computer system 600 can still receive user input via the basic console user interface 610 to control the interior environment of the vehicle in which the computer system 600 is installed. The basic console user interface 610 (e.g., layout, background, and / or content) is not based on information received from the user's phone. In some embodiments, the basic console user interface 610 is a default console user interface that is not user-specific or device-specific (e.g., phone-specific). In some embodiments, the basic console user interface 610 does not contain additional information / controls.

[0144] In Figure 6B, the computer system 600 displays a basic instrument cluster user interface 620 via a second display 604, which includes the vehicle manufacturer indication 620A, the vehicle range and fuel / charge level 620B, the vehicle's current speed 620C, and the vehicle's current shift mode (e.g., drive, neutral, parking, and / or reverse). The basic instrument cluster user interface 620 (e.g., layout, instrument cluster style, background, and / or content) is not based on information received from the user's phone. In some embodiments, the basic instrument cluster user interface 620 is a default instrument cluster user interface that is not user-specific or device-specific (e.g., phone-specific). In some embodiments, the basic instrument cluster user interface 620 does not include additional instruments / information / controls.

[0145] In some embodiments, as shown in Figure 6B, the computer system 600 connects to the user's phone within a first threshold duration (for example, within 2 or 8 seconds of initiating an attempt to connect to a paired phone) and displays the user interface shown in Figure 6C accordingly.

[0146] In Figure 6C, based on the connection of a telephone, the computer system 600 displays an extended console user interface 614 via the first display 602. The extended console user interface 614 includes more information and / or controls compared to the basic console user interface 610. In some embodiments, the extended console user interface 614 is based on information received from the connected telephone. In some embodiments, the extended console user interface 614 is a personalized user interface displayed based on the computer system 600 identifying that a telephone has been connected to the computer system 600 (compared to a second different telephone). In some embodiments, the extended console user interface 614 is a personalized user interface displayed based on the computer system 600 receiving a user interface layout, background image, and / or other information while connected to the telephone, which is used when preparing and / or displaying the extended console user interface 614.As shown in Figure 6C, the extended console user interface 614 includes media controls 614A (e.g., track name, artist name, album art, transfer / reward / pause controls, and / or progress indicator), news application option 614B (e.g., when activated, causes the news application to be displayed (on the first display 602 and / or the second display 604)), media library option 614C (e.g., when activated, causes the media controls 614A and / or the media library to be displayed (on the first display 602 and / or the second display 604)), messaging application option 614D (e.g., when activated, causes the messaging (e.g., instant messaging and / or email) application to be displayed (on the first display 602 and / or the second display 604)), and messaging application option 614D (e.g., when activated, causes the messaging (e.g., instant messaging and / or email) application to be displayed (on the first display 602 and / or the second display 604). The options include: indoor environment option 614E (for example, when activated, causes the display of indoor environment control options (on the first display 602 and / or the second display 604)), configuration option 614F (for example, when activated, causes the display of configuration options for the vehicle and / or computer system 600 (e.g., Wi-Fi or cellular, user account, and / or home address) (on the first display 602 and / or the second display 604)), and home option 614G (for example, when activated, causes the display of a home user interface (on the first display 602 and / or the second display 604) which includes options for launching multiple applications (e.g., messaging, email, news, stocks, and / or maps)).

[0147] In Figure 6C, after connecting to a telephone, the computer system 600 displays the extended instrument cluster user interface 624 via the second display 604. The extended instrument cluster user interface 624 includes more information and / or controls compared to the basic instrument cluster user interface 620. In some embodiments, the extended instrument cluster user interface 624 is based on information received from the connected telephone. In some embodiments, the extended instrument cluster user interface 624 is a personalized user interface displayed when the computer system 600 identifies that a telephone has been connected to the computer system 600 (compared to a second different telephone). In some embodiments, the extended instrument cluster user interface 624 is a personalized user interface displayed when the computer system 600 receives user interface layout, background image, and / or other information while the telephone is connected. This information is used when preparing and / or displaying the extended instrument cluster user interface 624. As shown in Figure 6C, the Extended Instrument Group User Interface 624 includes media information 624A (e.g., album art, track name, artist name, and / or playback progress), a speedometer 624B (e.g., optionally including a pre-filled path to indicate the vehicle's current speed and an indicator along the path to indicate the current speed limit), a tachometer 624C (e.g., indicating the vehicle's engine speed), and a fuel / charge level 624D (e.g., indicating the vehicle's fuel or charge level). In some embodiments, the Extended Instrument Group User Interface 624 also includes the vehicle's current shift mode (e.g., drive, neutral, parking, and / or reverse), the vehicle's tire pressure, a navigation map of the vehicle's route, and / or turn-by-turn instructions for navigating to a destination. In some embodiments, when the computer system 600 connects to a second (different) phone, the computer system 600 displays an Extended Instrument Group User Interface and an Extended Console User Interface specific to the user of the second (different) phone based on information received from the second (different) phone.Therefore, each user can have their own personalized user interface when using the vehicle.

[0148] In some embodiments, as shown in Figure 6B, the computer system 600 is paired with multiple phones, and if the computer system 600 fails to connect to one of the multiple phones within a first threshold duration (for example, within 2 or 8 seconds of initiating an attempt to connect to a paired phone), it displays a selectable option 616 to switch to displaying the vehicle's native user interface (or multiple native user interfaces), as shown in Figure 6D.

[0149] In Figure 6D, after a phone has not been connected within a first threshold duration, the computer system 600 displays a selectable option 616 via the first display 602 and / or the second display 604 to switch to displaying the vehicle's native user interface (or a set of native user interfaces). When the computer system 600 detects the activation of the selectable option 616 (e.g., tap input on), the computer system 600 updates the first display 602 and the second display 604 to display the native user interface (e.g., a user interface native to the vehicle (such as a user interface provided by the vehicle manufacturer and / or a user interface specific to this manufacturer / model of the vehicle)) as shown in Figure 6E (but without notification 632). In some embodiments, the native user interface is not a user-specific user interface. In some embodiments, the native user interface includes a native console user interface 630 and / or a native instrument cluster user interface 640. The native console user interface 630 and the native instrument cluster user interface 640 (e.g., their respective layouts, backgrounds, content, instrument clusters, and / or instrument cluster types) are not based on information received from the user's phone. In some embodiments, the native console user interface 630 and / or the native instrument cluster user interface 640 are not user-specific or device-specific (e.g., phone-specific).

[0150] In Figure 6D, while displaying a selectable option 616 for switching to displaying the vehicle's native user interface (or multiple native user interfaces), the computer system 600 detects that a third threshold duration has been reached without the selectable option 616 being activated, and accordingly, the computer system 600 automatically updates the first display 602 and the second display 604 to display the native user interfaces 630 / 640 and notification 632 (e.g., on the first display 602 and / or the second display 604) (e.g., overlaid on the native console user interface 630). Activation of option 632A stops the display of the notification and optionally provides the user with access to the native user interfaces 630 / 640 (e.g., its touch input control). Therefore, in some embodiments, if the user does not manually activate one of the selectable options 616 within a third threshold duration, the computer system 600 automatically displays the native user interface along with a notification (e.g., a warning) that switching to the native user interface may rearrange information (e.g., the location / appearance of indoor environment controls may change).

[0151] In Figure 6E, as shown in the native console user interface 630, the air temperature control 630A has a different display location, appearance, and control technique (e.g., tap vs. slide) compared to the air temperature control 612A; the native console user interface 630 does not include a seat temperature control; the air circulation control 630D has a different display location and appearance compared to the air circulation control 612C; and the windshield defrost button 630C and rear window defrost button 630B have a different display location and arrangement compared to the windshield defrost button 612D and rear window defrost button 612E. In some embodiments, the appearance and / or display location of 640A-640D within the native instrument cluster user interface 640 also changes on the display 604 compared to the basic instrument cluster user interface 620 (e.g., the native instrument cluster user interface 640 looks different from the basic instrument cluster user interface 620).

[0152] In some embodiments, as shown in Figure 6B, the computer system 600 is paired with a single phone (compared to multiple phones), and the computer system 600 fails to connect to that single phone within a first threshold duration (e.g., within 2 or 8 seconds of initiating an attempt to connect to the single paired phone), and accordingly, the computer system 600 automatically displays the native console user interface 630 and / or the native instrument cluster user interface 640 (without displaying or receiving a selection of any selectable options 616), as shown in Figure 6E. In some embodiments, when the native console user interface 630 and / or the native instrument cluster user interface 640 are automatically displayed, the computer system 600 does not display notification 632. In some embodiments, when the native console user interface 630 and / or the native instrument cluster user interface 640 are automatically displayed, the computer system 600 displays notification 632 (e.g., overlaid on the native console user interface 630). Therefore, when the computer system 600 is paired with a single telephone and the computer system 600 does not connect to the telephone within a first threshold duration, the computer system 600 proceeds to display the native user interface rather than prompting the user to authorize displaying the native user interface.

[0153] As shown in Figure 6E, while displaying the native user interface 630 / 640 (with or without notification 632), the computer system 600 detects that a connection has been made to the user's phone (e.g., the phone was previously turned off and is now turned on). In response to detecting that a connection has been made to the phone while displaying the native user interface 630 / 640, the computer system 600 displays notification 634, as shown in Figure 6F, indicating that the computer system 600 has connected to the phone and requesting authorization to switch to displaying the extended user interface (e.g., 614 / 624). As described above, the extended user interface (e.g., 614 / 624) optionally includes more information, a better / more familiar user experience, and more configuration options compared to the native user interface 630 / 640 (and compared to the basic user interface 610 / 620). Furthermore, the extended user interface (e.g., 614 / 624) is personalized (e.g., user-specific and / or device-specific (e.g., phone-specific)). In Figure 6F, the computer system 600 detects the activation of option 634A and, accordingly, displays the extended console user interface 614 and / or the extended instrument cluster user interface 624, as shown in Figure 6C.

[0154] Figure 7 is a flowchart illustrating a method for managing communication with an external device according to several embodiments. Method 700 is performed in a computer system (e.g., 100, 300, 500, and / or 600) (e.g., infotainment system and / or instrument panel system) of a vehicle (e.g., automobile, boat, or airplane) that communicates with a first display generation component (e.g., 602) (e.g., a vehicle display generation component, a set of displays in the vehicle's front console, and / or a display positioned in front of the driver's seat of the vehicle). Some operations of Method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0155] As will be described later, Method 700 provides an intuitive way to manage communication with external devices. This method reduces the cognitive burden on the user that relies on the computer system to communicate with external devices, thereby creating a more efficient human-machine interface. In the case of battery-powered computing devices, power is saved and the intervals between battery charges are increased by enabling the user to manage communication with external devices more quickly and efficiently.

[0156] The computer system (e.g., 600) (e.g., when the computer system and / or vehicle is started up (or accordingly)) (e.g., while attempting to connect to an external device) displays an indication (e.g., 610A and / or 610B) (e.g., the indication says "Connecting to your phone" or "Connecting to a smart device") via a first display generating component (e.g., 602) (e.g., as part of a first user interface (e.g., 610), as part of a first user interface generated by the computer system, and / or as part of a user interface not native to the vehicle) that the computer system (e.g., 600) is attempting to connect to one or more external devices (e.g., the user's smartphone and / or devices in the vehicle) (e.g., wirelessly, via RF, and / or via a short-range radio) (e.g., the indication says "Connecting to your phone" or "Connecting to a smart device"), and information (e.g., 612, 6 12A~612E) The first set (e.g., reduced set, basic set, and / or limited set) (e.g., vehicle, current interior environment (e.g., temperature) information and / or controls, interior environment information and / or controls (e.g., target interior temperature, seat heating / cooling settings, air recirculation information, windshield defrosting settings, rear window defroster settings, driver / passenger environment information)) is displayed without displaying (702) the second set (e.g., extended set, advanced set, enlarged set, and / or rich set) (e.g., vehicle, media player controls, current interior environment information and / or controls, interior environment information and / or controls, options for displaying various applications, maps and / or user interfaces for navigation, messaging (e.g., instant messaging and / or email) information and / or interfaces, and / or calendar information),

[0157] Before reaching a first threshold duration (e.g., 2 seconds, 8 seconds, 30 seconds, or 2 minutes) (e.g., from when the computer system is started, from when an attempt to connect to an external device is initiated, from when the vehicle is started, and / or from when the computer system detects a user in the vehicle), the computer system (e.g., 600) and an external device (e.g., a device that is not part of the computer system and / or the vehicle, a device located inside the vehicle, a device among one or more external devices, a personal / mobile device, and / or a smartphone) are successfully established, according to a determination (e.g., a determination made while displaying a first set of information and an indication that the computer system is attempting to connect to one or more external devices) (and / or in response to connecting to an external device), the computer The data system 600 displays (704) a second set of information (e.g., an extended set, an advanced set, an expanded set, and / or a rich set) (e.g., 614, 614A-614G) (based on information received from an external device) (with or without displaying an indication (e.g., 610A-610B) that the computer system is attempting to connect to one or more external devices, and with or without displaying the first set of information, via a first display generation component (e.g., 602) (e.g., as part of a second (extended) user interface (e.g., 614) different from the first user interface, as part of a second user interface generated by a computer system, and / or as part of a user interface not native to the vehicle), (e.g., by replacing all or part of the first user interface), a second set of information (e.g., an extended set, an advanced set, an expanded set, and / or a rich set) (e.g., 614, 614A-614G) (e.g., without displaying an indication (e.g., 610A-610B) that the computer system is attempting to connect to one or more external devices),

[0158] Before reaching a second threshold duration (e.g., the same as the first threshold duration, 2 seconds, 8 seconds, 30 seconds, or 2 minutes) (e.g., from when the computer system is started, from when an attempt to connect to an external device is initiated, from when the vehicle is started, and / or from when the computer system detects a user in the vehicle), the computer system (e.g., 600) determines that a second set of criteria is met, which includes connection failure criteria that are met when a connection is not successfully established between the computer system (e.g., 600) and an external device (e.g., a device that is not part of the computer system and / or the vehicle, a device located inside the vehicle, a device among one or more external devices, a personal / mobile device, and / or a smartphone), in accordance with a determination (e.g., a determination made while the first set of information and an indication that the computer system is attempting to connect to one or more external devices) the computer system (e.g., 600) will generate a first display generation component (e.g., 6 (02) displays (e.g., as part of the first user interface and / or by replacing the first information set) (e.g., 630 and / or 640) via (02) without displaying the native user interface (e.g., 630 and / or 640) and without displaying the second information set (e.g., 614, 614A-614G) (and optionally, by ceasing to display the first information set (e.g., 612, 612A-612E)) a selectable option (e.g., 616) for switching to the native user interface (e.g., 630 and / or 640) of the vehicle (e.g., as part of the first user interface and / or by replacing the first information set) (e.g., as part of the first user interface and / or by replacing the first information set).In some embodiments, the computer system detects the activation of an option to switch to the vehicle's native user interface and, accordingly, displays the vehicle's native user interface via a first display generation component (e.g., as part of a third user interface not generated by the computer system, as part of a user interface generated by a vehicle system different from the computer system, as part of a user interface provided by the vehicle manufacturer, as a default user interface, and / or as part of a user interface not based on information from an external device). By automatically updating the computer system's display to show an extended user interface with a second set of information when a connection to an external device is established, the number of inputs required to view the second set of information is reduced, thereby improving the man-machine interface. Automatically displaying the option to access the native user interface provides the user with the ability to quickly access the native user interface while maintaining the existing content being displayed, and thus allowing the user to easily continue accessing the first set of information.

[0159] In some embodiments, the computer (e.g., 600) determines that a third set of criteria is met, which includes an unselected criterion that is met when it does not detect a selection of a selectable option (e.g., 616) before reaching a third threshold duration (e.g., longer than the first and second threshold durations and / or reached while the selectable options are displayed) (e.g., from the time the computer system is started, from the time an attempt to connect to an external device is initiated, from the time the vehicle is started, and / or from the time the computer system detects a user in the vehicle), and the computer (e.g., from the time the computer system is started, from the time an attempt to connect to an external device is initiated, from the time the vehicle is started, and / or from the time the computer system detects a user in the vehicle). This determination is made according to the computer (e.g., a determination made while the selectable options for switching to the vehicle's native user interface are displayed). The system (e.g., 600) displays the vehicle's native user interface (e.g., 630) via a first display generation component (e.g., 602) (e.g., as part of a user interface generated by a vehicle system different from the computer system, as part of a user interface provided by the vehicle manufacturer, a default user interface, and / or a user interface not based on information from an external device) (e.g., automatically by replacing a first set of information and / or selectable options for switching to the vehicle's native user interface without receiving user input). Thus, in some embodiments, if the user is prompted with a selectable option to switch to the vehicle's native user interface but does not select an option for a threshold duration, the computer system automatically switches to displaying the vehicle's native user interface. Automatically displaying the native user interface when the computer system does not detect an option selection allows the computer system to provide the native user interface to the user without requiring user input, thereby reducing the number of inputs required to access the native user interface.

[0160] In some embodiments, a second set of information (e.g., 614, 614A-614G) is based on information received from an external device (e.g., graphics, layout, data, data arrangement, playlists, media selection or preferences, application icons) (e.g., information received after a connection has been successfully established between the computer system and the external device, and / or information received during previous communication between the computer system and the external device). In some embodiments, the second set of information is received from an external device. In some embodiments, the first set of information is not based on information received from an external device (e.g., graphics, layout, data, data arrangement, playlists, media selection or preferences, application icons). The second set of information based on information from an external device enables the computer system to display a personalized user interface based on information from the user's external device (e.g., a telephone) without the user having to repeatedly make those personalization choices.

[0161] In some embodiments, the layout (e.g., arrangement and / or placement on the display, such as in relation to other displayed content) of a second information set (e.g., 614, 614A-614G) is based on information received from an external device. In some embodiments, the computer system receives information from an external device regarding how to lay out the second information set on the computer system's display. In some embodiments, the display layout of the first information set is not based on information received from an external device. In some embodiments, the computer system connects to an external device, receives layout information for the second information set (from the external device), and uses the layout information to display the second information set. In some embodiments, the computer system connects to an external device and receives the second information set to be displayed (from the external device). In some embodiments, the second information set includes a background image received from an external device, which is displayed as a background via the first display generation component. The layout of the second information set based on information from an external device allows the computer system to display a personalized user interface based on information from the user's external device (e.g., a phone) without the user having to repeatedly make those personalized choices (such as layouts).

[0162] In some embodiments, a computer system (e.g., 600) is configured to communicate with a second display generating component (e.g., 604). The computer system (e.g., 600) communicates (e.g., when the computer system and / or vehicle are started (or accordingly)) (e.g., while attempting to connect to an external device) via the second display generating component (e.g., 604) (e.g., as part of a first instrument cluster user interface, as part of a first instrument cluster user interface generated by the computer system, and / or as part of a non-vehicle native user interface), (e.g., a display positioned behind or adjacent to the steering wheel of the vehicle, a display positioned in front of the driver of the vehicle (e.g., directly in front), (e.g., the vehicle, the vehicle speed (first speedometer), the vehicle range, battery charge or fuel gauge, and / or The system displays a first set of instrument cluster information (e.g., 620, 620A-620D) (e.g., a reduced set, a basic set, and / or a limited set) (related to the vehicle's driving mode (parking, drive, and / or reverse)) but does not display a second set of instrument cluster information (e.g., 624, 624A-624D) (e.g., an extended set, an advanced set, an expanded set, and / or a rich set) (related to the vehicle, the vehicle's driving mode (parking, drive, and / or reverse), a second speedometer indicating the vehicle's current speed, the current road speed limit, media playback information (artist, track, duration, progress), the vehicle's range, the distance to the destination, the battery charge or fuel gauge, and / or the vehicle's battery charge). In some embodiments, the computer system does not display an indication via a second display generation component that the computer system is attempting to connect to one or more external devices.In accordance with a determination that a first set of criteria is met (for example, a determination made while displaying the first set of instrument cluster information) (and / or in response to connecting to an external device), the computer system (e.g., 600) displays a second set of instrument cluster information (e.g., 624, 624A-624D) (e.g., based on information received from an external device) (e.g., by replacing all or part of the first instrument cluster user interface) via a second display generation component (e.g., 604) (e.g., as part of a second instrument cluster user interface generated by the computer system, as part of a second instrument cluster user interface different from the first instrument cluster user interface, and / or as part of a user interface not native to the vehicle) (e.g., by replacing all or part of the first instrument cluster user interface). When a connection to an external device is established, the computer system's second display is automatically updated to show an extended instrument cluster user interface with a second instrument cluster information set, thereby reducing the number of inputs required to view the second instrument cluster information set and improving the man-machine interface.

[0163] In some embodiments, the second instrument cluster information set (e.g., 624, 624A-624G) is based on information received from an external device (e.g., graphics, layout, data, speedometer type / size / color, data placement, instrument cluster gauge type / size / location) (e.g., information received after a connection between the computer system and the external device has been successfully established, and / or information received during previous communication between the computer system and the external device). In some embodiments, the second instrument cluster information set is received from an external device. In some embodiments, the first instrument cluster information set is not based on information received from an external device (e.g., graphics, layout, data, speedometer type / size / color, data placement, instrument cluster gauge type / size / location). The second instrument cluster information set based on information from an external device enables the computer system to display a personalized user interface based on information from the user's external device (e.g., a telephone) without the user having to repeatedly make those personalization choices.

[0164] In some embodiments, the layout (e.g., arrangement and / or placement on the display, such as in relation to other displayed content) of a second instrument group information set (e.g., 624, 624A-624G) is based on information received from an external device. In some embodiments, the computer system receives information from an external device regarding how to lay out the second instrument group information set on a second display of the computer system (e.g., an instrument group display). In some embodiments, the display layout of the first instrument group information set is not based on information received from an external device. In some embodiments, the computer system connects to an external device, receives layout information for the second instrument group information set (from the external device), and uses the layout information to display the second instrument group information set. In some embodiments, the computer system connects to an external device and receives the second instrument group information set to be displayed (from the external device). In some embodiments, the second instrument group information set includes an instrument group background image received from an external device, which is displayed as a background via a second display generation component. The layout of the second instrument cluster information set based on information from an external device allows the computer system to display a personalized user interface based on information from the user's external device (e.g., a telephone) without the user having to repeatedly make those personalization choices (such as the layout).

[0165] In some embodiments, while a computer system (e.g., 600) is displaying the vehicle's native user interface (e.g., 630) via a first display generation component (e.g., 602), the computer system (e.g., 600) detects that a connection has been successfully established between the computer system (e.g., 600) and an external device. In response to detecting that a connection has been successfully established between the computer system (e.g., 600) and an external device, the computer system (e.g., 600) automatically stops displaying a second set of information (e.g., 614, 614A-614G) (e.g., an extended set, an advanced set, an expanded set, and / or a rich set) (e.g., via a second user interface) via the first display generation component (e.g., 602) (e.g., based on information received from an external device). In some embodiments, if a connection to an external device is made while the native user interface is being displayed, the native user interface is not automatically replaced by a second (e.g., extended) user interface. By maintaining the display of the vehicle's native user interface when the computer system successfully connects to an external device, the computer system can maintain a consistent user interface without changing the location of the displayed elements.

[0166] In some embodiments, while displaying the vehicle's native user interface (e.g., 630) via a first display generation component (e.g., 602), the computer system (e.g., 600) detects that a connection has been successfully established between the computer system (e.g., 600) and an external device. In response to detecting that a connection has been successfully established between the computer system (e.g., 600) and an external device, the computer system (e.g., 600) displays a second selectable option (e.g., 634A) for switching to displaying a second set of information (e.g., 614, 614A-614G) (e.g., a second (extended) user interface) via the first display generation component (e.g., 602). In some embodiments, if a connection to an external device is made while the native user interface is being displayed, the computer system displays an option to replace the native user interface with the second (extended) user interface. By displaying a selectable option to switch from displaying the native user interface to displaying the second set of information, the computer system is enabled to receive user input for making changes. In addition, displaying selectable options provides the user with feedback that the connection has been successfully established, thereby providing improved visual feedback.

[0167] In some embodiments, the second set of criteria includes a multiple-authorized-device criterion that is met when the computer system (e.g., 600) is configured to connect to one of several authorized external devices (e.g., the computer system is paired with several external devices and attempts to connect to one of those several external devices at startup). The second set of criteria includes a connection failure criterion and a single-authorized-device criterion that is met when the computer system is configured to connect to a single authorized external device (e.g., the computer system is paired with a single external device (e.g., a smartphone) and attempts to connect to that single external device (not to other external devices) at startup). In accordance with a determination that a fourth set of criteria is met (e.g., a determination made while displaying the first set of information and an indication that the computer system is attempting to connect to one or more external devices), the computer system (e.g., 600) may, via a first display generation component (e.g., 602), (e.g., without receiving user input) The vehicle's native user interface (e.g., 630) is displayed (e.g., without displaying the selectable option (e.g., 616) for switching to the vehicle's native user interface) automatically by replacing the first set of information and / or the selectable option for switching to the vehicle's native user interface (e.g., 630) (e.g., as part of a third user interface not generated by the computer system, as part of a user interface generated by a vehicle system different from the computer system, as a user interface provided by the vehicle manufacturer, a default user interface, and / or a user interface not based on information from an external device).In some embodiments, the computer system is configured to connect to one of several external devices (e.g., several devices are authorized), and if a second threshold duration is reached without connection, the computer system does not automatically display the native user interface, but instead displays an option to display the native user interface when activated. In some embodiments, the computer system is configured to connect to a single external device (not one of several external devices) (e.g., only one external device is authorized), and if a second threshold duration is reached without connection, the computer system automatically displays the native user interface (e.g., without displaying a selectable option to display the native user interface). By automatically displaying the native user interface, the computer system can provide feedback to the user that the connection failure criterion has been met, enabling the display of the native user interface without requiring further user input, thereby enhancing the man-machine interface.

[0168] In some embodiments, a computer system (e.g., 600) is configured to communicate with one or more input devices (touch-sensitive surfaces and / or microphones). A second set of information (e.g., 614, 614A-614G) is based on information received from an external device. While displaying a second set of information (e.g., an extended set, an advanced set, an enlarged set, and / or a rich set) via a first display generating component (and optionally while connected to an external device), the computer system (e.g., 600) detects user input via one or more input devices requesting connection to a second external device different from the first external device. In response to detecting user input requesting connection to a second external device, the computer system (e.g., 600) connects to the second external device (e.g., wirelessly, via RF, and / or via a short-range radio) (and optionally disconnects from the external device). After connecting to (and optionally disconnecting from) a second external device (for example, accordingly), the computer system (e.g., 600) displays a third set of information (e.g., an extended set, an advanced set, an enlarged set, and / or rich set) (e.g., an extended set, an advanced set, an enlarged set, and / or rich set) (e.g., as part of a fourth (extended) user interface) based on (and optionally, not based on) information received from the second external device (e.g., vehicle, media player controls, current interior environment information and / or controls, options for displaying various applications, maps and / or user interfaces for navigation, messaging information and / or interfaces (e.g., relating to instant messaging and / or email), and / or calendar information) via a first display generation component (e.g., 602).In some embodiments, while displaying an enhanced user interface (having a second set of information) based on information from an external device, the computer system switches to communicating with the second external device and, accordingly, displays a new enhanced user interface (having a third set of information) based on information from the second external device. The new (enhanced) user interface has different information and / or layout compared to the enhanced user interface based on information from the external device. By updating the user interface(s)

[0169] In some embodiments, while displaying a second set of information (e.g., 614, 614A-614G) via a first display generation component (e.g., 602) (e.g., as part of a second (extended) user interface), the computer system (e.g., 600) detects a disconnection from an external device (e.g., an unexpected disconnection such as the external device being removed from communication range) (e.g., the computer system and the external device are no longer in communication range). In response to detecting a disconnection from the external device, the computer system (e.g., 600) displays a native user interface (e.g., 630) via the first display generation component (e.g., 602) (and optionally a notification (e.g., 632) indicating that "information can be rearranged by switching to the native user interface," as well as an option to dismiss the notification and / or proceed) (e.g., without receiving any additional user input). Automatically displaying the native user interface when an external device is disconnected from the computer system provides the user with visual feedback that the external device has been disconnected and allows the user to continue using some of the computer system's functions (e.g., a reduced set).

[0170] In some embodiments, while a second set of information (e.g., 614, 614A-614G) is displayed (e.g., as part of a second (extended) user interface) via a first display generation component (e.g., 602), the computer system (e.g., 600) determines that a disconnection from the external device is likely to occur (e.g., within a threshold duration) (e.g., based on the battery level of the external device and / or based on low or reducing communication signal strength between the computer system and the external device). In response to determining that a disconnection from the external device is likely to occur (e.g., within a threshold duration), the computer system (e.g., 600) displays a warning (e.g., that a disconnection is occurring) via the first display generation component (e.g., 602) (and / or the second display generation component) based on its detection of a high probability of a disconnection from the external device. Displaying a warning when it is determined that an external device is likely to disconnect from the computer system provides the user with feedback that the external device is likely to disconnect, thereby providing the user with improved visual feedback.

[0171] The above is written with reference to specific embodiments for illustrative purposes. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the exact forms disclosed. Many modifications and variations are possible in light of the above teachings. Embodiments have been selected and described to best illustrate the principles of the Art and their practical applications. This will enable other persons skilled in the art to best utilize the Art and its various embodiments with various modifications suitable for their intended specific applications.

[0172] While the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art. Such changes and modifications should be understood to fall within the scope of the present disclosure and examples as defined by the claims.

[0173] As described above, one aspect of the technology involves collecting and using data available from various sources to improve communication between devices. This disclosure suggests that in some cases, such collected data may include personal information data that uniquely identifies a particular person, or personal information data that can be used to contact a particular person or locate them. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, social network IDs, home addresses, data or records relating to a user's health or fitness level (e.g., vital signs measurements, medication information, exercise information), birth dates, or any other identifying or personal information.

[0174] This disclosure acknowledges that such use of personal data in this technology may be for the benefit of the user. For example, personal data can be used to facilitate the automation of the display of an augmented user interface. Furthermore, other uses of personal data that may benefit the user are also conceivable in this disclosure.

[0175] This disclosure implies that entities involved in the collection, analysis, disclosure, transmission, storage, or other use of such personal data should adhere to a robust privacy policy and / or privacy practices. Specifically, such entities should implement and consistently use a privacy policy and practices that are generally recognized as meeting or exceeding industry or government requirements for the strict confidentiality of personal data. Such policies should be readily accessible to users and should be updated as data collection and / or use changes. Personal data from users should be collected for the lawful and legitimate use of the entity and should not be shared or sold for any other purpose. Furthermore, such collection / sharing should be carried out only after informing and obtaining the user's consent. In addition, such entities should consider taking all necessary steps to protect and secure access to such personal data and to ensure that others with access to the personal data faithfully adhere to those privacy policies and procedures. Furthermore, such entities may undergo third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be adapted to the specific types of personal data collected and / or accessed, and should comply with applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, the collection or access to certain health data may be subject to federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA). Health data in other countries, on the other hand, may be subject to other regulations and policies and should be addressed accordingly. Therefore, different privacy practices should be maintained in each country with respect to different types of personal data.

[0176] Notwithstanding the foregoing, the Disclosure also envisions embodiments that allow a user to selectively prevent the use of or access to personal data. That is, the Disclosure envisions that hardware and / or software elements may be provided to prevent or prevent access to such personal data. For example, in managing communication between devices, the Technology may be configured to allow a user to choose to “opt in” or “opt out” of participating in the collection of personal data during or at any time thereafter when registering for a service. In addition to providing “opt-in” and “opt-out” options, the Disclosure envisions providing notices regarding access to or use of personal data. For example, a user may be notified when downloading an app that will access their personal data, and then again immediately before the app accesses their personal data.

[0177] Furthermore, the intent of this disclosure is that personal data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data when it is no longer needed. In addition, where applicable in certain health-related applications, data anonymization can be used to protect user privacy. Anonymization can be facilitated, where appropriate, by removing certain identifiers (e.g., birth dates), controlling the amount or specificity of data stored (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data across all users), and / or by other means.

[0178] Therefore, while this disclosure broadly covers the use of personal data to implement one or more different disclosed embodiments, it is also conceivable that these different embodiments could be implemented without requiring access to such personal data. In other words, the different embodiments of the technology would not be rendered inoperable by the absence of all or part of such personal data. For example, a user interface could be selected and presented to the user by inferring preferences based on non-personal data or minimal personal information.

[0179] [Section 1] It is a method, In the vehicle's computer system, the computer system communicates with a first display generation component. The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, and a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A method comprising: determining that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached; and displaying selectable options for switching to the vehicle's native user interface via the first display generation component without displaying the native user interface and without displaying the second set of information. [Section 2] In accordance with the determination that a third set of criteria is met, the third set of criteria includes an unselected criterion that is met when the computer system does not detect the selection of the selectable option before the third threshold duration is reached, and displays the vehicle's native user interface via the first display generation component. The method described in item 1, further comprising: [Section 3] The second set of information is the method according to item 1 or 2, based on information received from the external device. [Section 4] The layout of the second information set is the method according to any one of items 1 to 3, based on the information received from the external device. [Section 5] The computer system is configured to communicate with a second display generation component, and the method is To display the first instrument group information set without displaying the second instrument group information set via the second display generation component, The method according to any one of claims 1 to 4, further comprising displaying the second set of instrument group information via the second display generation component in accordance with the determination that the first set of criteria is satisfied. [Section 6] The second instrument group information set is the method according to item 5, based on information received from the external device. [Section 7] The layout of the second instrument group information set is the method according to item 5 or 6, based on the information received from the external device. [Section 8] To detect that a connection between the computer system and the external device has been successfully established while the vehicle's native user interface is being displayed via the first display generation component, Upon detecting that a connection has been successfully established between the computer system and the external device, the system will stop automatically displaying the second information set via the first display generation component. The method described in any one of items 1 to 7, further including the method described in any one of items 1 to 7. [Section 9] To detect that a connection between the computer system and the external device has been successfully established while the vehicle's native user interface is being displayed via the first display generation component, Upon detecting that a connection has been successfully established between the computer system and the external device, a second selectable option is displayed via the first display generation component to switch to displaying the second set of information via the first display generation component. The method described in any one of items 1 to 8, further including the method described in any one of items 1 to 8. [Section 10] The second set of criteria includes multiple authorization device criteria that are satisfied when the computer system is configured to connect to one of a plurality of authorization external devices, and the method is The method according to any one of claims 1 to 9, further comprising displaying the native user interface of the vehicle via the first display generation component in accordance with a determination that a fourth set of criteria is met, which includes the connection failure criterion and a single authorization device criterion that is met when the computer system is configured to connect to a single authorization external device. [Section 11] The computer system is configured to communicate with one or more input devices, and the second set of information is based on information received from the external device, and the method is While displaying the second set of information via the first display generation component, the system detects user input via one or more input devices requesting connection to a second external device different from the external device, In response to detecting the user input requesting connection to the second external device, the system connects to the second external device and The method according to any one of claims 1 to 10, further comprising: connecting to the second external device, and then displaying a third set of information based on information received from the second external device via the first display generation component. [Section 12] While the second information set is being displayed via the first display generation component, a disconnection from the external device is detected, In response to detecting the disconnection from the external device, the native user interface is displayed via the first display generation component, The method described in any one of items 1 to 11, further including the method described in any one of items 1 to 11. [Section 13] It is determined that there is a high probability of disconnection from the external device occurring while the second information set is being displayed via the first display generation component, In response to determining that there is a high probability of disconnection from the external device, a warning is displayed via the first display generation component based on the detection of a high probability of disconnection from the external device. The method described in any one of items 1 to 12, further including the method described in any one of items 1 to 12. [Section 14] A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with a first display generation component, wherein the one or more programs include instructions to perform the method described in any one of items 1 to 13. [Section 15] A computer system configured to communicate with a first display generation component, wherein the computer system One or more processors, A computer system comprising: memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method described in any one of the items 1 to 13. [Section 16] A computer system configured to communicate with a first display generation component, A computer system comprising means for performing the method described in any one of items 1 to 13. [Section 17] A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that communicate with a first display generation component, wherein the one or more programs include instructions to perform the method described in any one of items 1 to 13. [Section 18] A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a vehicle's computer system, wherein the computer system communicates with a first display generation component, and the one or more programs The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, along with a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A non-temporary computer-readable storage medium, which includes instructions to display selectable options for switching to the vehicle's native user interface via the first display generation component, without displaying the native user interface and without displaying the second set of information, in accordance with the determination that the second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and which displays, via the first display generation component, selectable options for switching to the vehicle's native user interface. [Section 19] A vehicle computer system configured to communicate with a first display generation component, One or more processors, The system includes a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, along with a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A computer system including instructions to display, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information, in accordance with the determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and the second set of criteria includes instructions for displaying, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information. [Section 20] A vehicle computer system configured to communicate with a first display generation component, Means for displaying an indication that the computer system is attempting to connect to one or more external devices via the first display generation component, and a first set of information, without displaying a second set of information, In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and means for displaying the second set of information via the first display generation component, A computer system comprising: a determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and means for displaying selectable options for switching to the vehicle's native user interface via the first display generation component without displaying the native user interface and without displaying the second set of information. [Section 21] A computer program product comprising one or more programs configured to be executed by one or more processors of a vehicle's computer system, wherein the computer system communicates with a first display generation component, and the one or more programs The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, along with a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A computer program product comprising instructions to display, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second information set, in accordance with the determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and which displays, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information.

Claims

1. It is a method, In the vehicle's computer system, the computer system communicates with a first display generation component. The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, and a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A method comprising: determining that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached; and displaying selectable options for switching to the vehicle's native user interface via the first display generation component without displaying the native user interface and without displaying the second set of information.

2. In accordance with the determination that a third set of criteria is met, the third set of criteria includes an unselected criterion that is met when the computer system does not detect the selection of the selectable option before the third threshold duration is reached, and displays the vehicle's native user interface via the first display generation component. The method according to claim 1, further comprising:

3. The method according to claim 1 or 2, wherein the second set of information is based on information received from the external device.

4. The layout of the second information set is based on the information received from the external device, according to any one of claims 1 to 3.

5. The computer system is configured to communicate with a second display generation component, and the method is To display the first instrument group information set without displaying the second instrument group information set via the second display generation component, The method according to any one of claims 1 to 4, further comprising displaying the second instrument group information set via the second display generation component in accordance with the determination that the first set of criteria is satisfied.

6. The method according to claim 5, wherein the second instrument group information set is based on information received from the external device.

7. The layout of the second instrument group information set is based on information received from the external device, according to the method of claim 5 or 6.

8. To detect that a connection between the computer system and the external device has been successfully established while the vehicle's native user interface is being displayed via the first display generation component, Upon detecting that a connection has been successfully established between the computer system and the external device, the automatic display of the second information set via the first display generation component is discontinued. The method according to any one of claims 1 to 7, further comprising:

9. To detect that a connection between the computer system and the external device has been successfully established while the vehicle's native user interface is being displayed via the first display generation component, Upon detecting that a connection has been successfully established between the computer system and the external device, a second selectable option is displayed via the first display generation component to switch to displaying the second set of information via the first display generation component. The method according to any one of claims 1 to 8, further comprising:

10. The second set of criteria includes multiple authorization device criteria that are satisfied when the computer system is configured to connect to one of a plurality of authorization external devices, and the method is The method according to any one of claims 1 to 9, further comprising displaying the native user interface of the vehicle via the first display generation component in accordance with a determination that a fourth set of criteria is satisfied, which includes the connection failure criterion and a single authorization device criterion that is satisfied when the computer system is configured to connect to a single authorization external device.

11. The computer system is configured to communicate with one or more input devices, and the second set of information is based on information received from the external device, and the method is While the second information set is being displayed via the first display generation component, a user input requesting connection to a second external device different from the external device is detected via one or more input devices. In response to detecting the user input requesting connection to the second external device, the system connects to the second external device, The method according to any one of claims 1 to 10, further comprising: connecting to the second external device, and then displaying a third set of information based on information received from the second external device via the first display generation component.

12. While the second information set is being displayed via the first display generation component, a disconnection from the external device is detected, In response to detecting the disconnection from the external device, the native user interface is displayed via the first display generation component, The method according to any one of claims 1 to 11, further comprising:

13. It is determined that there is a high probability of disconnection from the external device occurring while the second information set is being displayed via the first display generation component, In response to determining that there is a high probability of disconnection from the external device, a warning is displayed via the first display generation component based on the detection of a high probability of disconnection from the external device. The method according to any one of claims 1 to 12, further comprising:

14. A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with a first display generation component, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 13.

15. A computer system configured to communicate with a first display generation component, wherein the computer system One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 13.

16. A computer system configured to communicate with a first display generation component, A computer system comprising means for performing the method described in any one of claims 1 to 13.

17. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with a first display generation component, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 13.

18. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a vehicle's computer system, wherein the computer system communicates with a first display generation component, and the one or more programs The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, and a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A non-temporary computer-readable storage medium, which includes instructions to display selectable options for switching to the vehicle's native user interface via the first display generation component, without displaying the native user interface and without displaying the second set of information, in accordance with the determination that the second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and which displays, via the first display generation component, selectable options for switching to the vehicle's native user interface.

19. A vehicle computer system configured to communicate with a first display generation component, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, and a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A computer system comprising instructions to display, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information, in accordance with the determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and which displays, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information.

20. A vehicle computer system configured to communicate with a first display generation component, Means for displaying an indication that the computer system is attempting to connect to one or more external devices via the first display generation component, and a first set of information, without displaying a second set of information, In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and means for displaying the second set of information via the first display generation component, A computer system comprising: a determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and means for displaying selectable options for switching to the vehicle's native user interface via the first display generation component without displaying the native user interface and without displaying the second set of information.

21. A computer program product comprising one or more programs configured to be executed by one or more processors of a vehicle's computer system, wherein the computer system communicates with a first display generation component, and the one or more programs The first display generation component displays an indication that the computer system is attempting to connect to one or more external devices, and a first set of information, without displaying a second set of information. In accordance with the determination that a first set of criteria is met, the first set of criteria includes connection establishment criteria that are met when a connection between the computer system and an external device is successfully established before a first threshold duration is reached, and the second set of information is displayed via the first display generation component. A computer program product comprising instructions to display, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second information set, in accordance with the determination that a second set of criteria has been met, wherein the second set of criteria includes connection failure criteria that are met when a connection is not successfully established between the computer system and the external device before a second threshold duration is reached, and which displays, via the first display generation component, selectable options for switching to the vehicle's native user interface without displaying the native user interface and without displaying the second set of information.

Images