Integration of hardware and software to track user interface

Abstract

The invention relates to integration of hardware and software to track a user interface. The present disclosure generally relates to capturing video using a movable mount.

Description

[0001] Cross-references to related applications

[0002] This application is a divisional application of the invention patent application with international application number PCT / US2024 / 010547, international application date of January 5, 2024, entry into the Chinese national phase date of July 15, 2025, Chinese national application number 2024800077984, and invention title "Integration of Hardware and Software for Tracking User Interface".

[0003] This application claims priority to U.S. Application No. 18 / 523,076, filed November 29, 2023, entitled “INTEGRATION OF HARDWARE AND SOFTWARE TRACKING USER INTERFACES”; U.S. Provisional Application No. 63 / 445,871, filed February 15, 2023, entitled “INTEGRATION OF HARDWARE AND SOFTWARE TRACKING USER INTERFACES”; and U.S. Provisional Application No. 63 / 439,560, filed January 17, 2023, entitled “INTEGRATION OF HARDWARE AND SOFTWARE TRACKING USER INTERFACES”, the entire disclosure of which is incorporated herein by reference. Technical Field

[0004] This disclosure relates generally to computer user interfaces, and more specifically to techniques for integrating hardware and software for tracking user interfaces. Background Technology

[0005] Electronic devices can be used to capture images and / or videos. The captured images and / or videos can be used to track one or more subjects. Summary of the Invention

[0006] However, some of the technologies used to integrate the hardware and software for tracking user interfaces into electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may include multiple buttons or keystrokes. These existing technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.

[0007] Therefore, this technology provides electronic devices with a faster and more efficient method and interface for integrating hardware and software for tracking user interfaces. Such methods and interfaces optionally complement or replace other methods for integrating hardware and software for tracking user interfaces. These methods and interfaces reduce the cognitive burden on users and result in more efficient human-machine interfaces. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging cycles.

[0008] According to some embodiments, a method is described. The method includes: at a computer system communicating with two or more camera sensors and one or more input devices: detecting, via the one or more input devices, a request to capture video using the field of view of the one or more camera sensors, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; in response to detecting the request to capture video: initiating a process of capturing video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a removable mount; and initiating a process of capturing video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the removable mount.

[0009] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with two or more camera sensors and one or more input devices. The one or more programs include instructions for performing the following operations: detecting a request to capture video using the field of view of the one or more camera sensors via the one or more input devices, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; in response to detecting the request to capture video: initiating a process to capture video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the removable mount.

[0010] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with two or more camera sensors and one or more input devices. The one or more programs include instructions for performing the following operations: detecting a request to capture video using the field of view of the one or more camera sensors via the one or more input devices, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; in response to detecting the request to capture video: initiating a process to capture video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the removable mount.

[0011] In some embodiments, a computer system is described that is configured to communicate with two or more camera sensors and one or more input devices. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing: detecting a request to capture video using the field of view of the one or more camera sensors via the one or more input devices, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; in response to detecting the request to capture video: initiating a process to capture video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the removable mount.

[0012] In some embodiments, a computer system is described that is configured to communicate with two or more camera sensors and one or more input devices. The computer system includes: components for detecting a request to capture video using the field of view of the one or more camera sensors via the one or more input devices, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; and components for performing the following operations in response to detecting the request to capture video: initiating a process to capture video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a movable mount; and initiating a process to capture video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the movable mount.

[0013] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with two or more camera sensors and one or more input devices. The one or more programs include instructions for performing the following operations: detecting a request via the one or more input devices to capture video using the field of view of the one or more camera sensors, the field of view changing based on movement of one or more subjects within the field of view of the one or more camera sensors; in response to detecting the request to capture video: initiating a process to capture video using a first camera sensor of the two or more camera sensors based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using a second camera sensor of the two or more camera sensors, different from the first camera sensor, based on determining that the computer system is not connected to the removable mount.

[0014] According to some embodiments, a method is described. The method includes: at a computer system including one or more camera sensors: detecting a request to capture video using the one or more camera sensors of the computer system; and in response to detecting the request to capture the video: based on determining that the computer system is connected to a removable mount, initiating a process to capture video using the one or more camera sensors of the computer system according to a corresponding subject tracking operation mode; and based on determining that the computer system is not connected to a removable mount, initiating a process to capture video using the one or more camera sensors without enabling the corresponding subject tracking operation mode.

[0015] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with one or more camera sensors. The one or more programs include instructions for performing the following operations: detecting a request to capture video using the one or more camera sensors of the computer system; and, in response to detecting the request to capture the video: initiating a process to capture video using the one or more camera sensors of the computer system according to a corresponding subject tracking operation mode, based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using the one or more camera sensors without enabling the corresponding subject tracking operation mode, based on determining that the computer system is not connected to a removable mount.

[0016] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with one or more camera sensors. The one or more programs include instructions for performing the following operations: detecting a request to capture video using the one or more camera sensors of the computer system; and, in response to detecting the request to capture the video: initiating a process to capture video using the one or more camera sensors of the computer system according to a corresponding subject tracking operation mode, based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using the one or more camera sensors without enabling the corresponding subject tracking operation mode, based on determining that the computer system is not connected to a removable mount.

[0017] According to some embodiments, a computer system is described that is configured to communicate with one or more camera sensors. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing: detecting a request to capture video using the one or more camera sensors of the computer system; and, in response to detecting the request to capture the video: initiating a process of capturing video using the one or more camera sensors of the computer system according to a corresponding subject tracking operation mode, based on determining that the computer system is connected to a removable mount; and initiating a process of capturing video using the one or more camera sensors without enabling the corresponding subject tracking operation mode, based on determining that the computer system is not connected to a removable mount.

[0018] According to some embodiments, a computer system is described, configured to communicate with one or more camera sensors. The computer system includes: components for detecting a request to capture video using the one or more camera sensors; and components for, in response to detecting the request to capture the video, to: initiate a process of capturing video using the one or more camera sensors according to a corresponding subject tracking operation mode, based on determining that the computer system is connected to a removable mount; and to initiate a process of capturing video using the one or more camera sensors without enabling the corresponding subject tracking operation mode, based on determining that the computer system is not connected to a removable mount.

[0019] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with one or more camera sensors. The one or more programs include instructions for performing the following operations: detecting a request to capture video using the one or more camera sensors of the computer system; and, in response to detecting the request to capture the video: initiating a process to capture video using the one or more camera sensors of the computer system according to a corresponding subject tracking operation mode, based on determining that the computer system is connected to a removable mount; and initiating a process to capture video using the one or more camera sensors without enabling the corresponding subject tracking operation mode, based on determining that the computer system is not connected to a removable mount.

[0020] According to some embodiments, a method is described. The method includes: at a computer system in communication with a removable mount: obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and in response to obtaining the indication of the event, causing the removable mount to perform a series of one or more mechanical movements of the device connected to the removable mount, wherein the series of one or more mechanical movements is associated with the event.

[0021] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a removable mount. The one or more programs include instructions for performing: obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and, in response to obtaining the indication of the event, causing the removable mount to perform a series of one or more mechanical movements of the device connected to the removable mount, wherein the series of one or more mechanical movements is associated with the event.

[0022] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a removable mount. The one or more programs include instructions for performing: obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and, in response to obtaining the indication of the event, causing the removable mount to perform a series of one or more mechanical movements of the device connected to the removable mount, wherein the series of one or more mechanical movements are associated with the event.

[0023] According to some embodiments, a computer system is described, configured to communicate with a removable mount. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing: obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and, in response to obtaining the indication of the event, causing the removable mount to perform a series of one or more mechanical movements of the device connected to the removable mount, wherein the series of one or more mechanical movements are associated with the event.

[0024] According to some embodiments, a computer system is described that is configured to communicate with a removable mount. The computer system includes: components for obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and components for causing the removable mount to perform one or more mechanical movements of the device connected to the removable mount in response to obtaining the indication of the event, wherein the one or more mechanical movements are associated with the event.

[0025] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a removable mount. The one or more programs include instructions for performing: obtaining an indication of an event associated with captured video, wherein the video is captured by a device connected to the removable mount, and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects during the capture of the video; and, in response to obtaining the indication of the event, causing the removable mount to perform a series of one or more mechanical movements of the device connected to the removable mount, wherein the series of one or more mechanical movements are associated with the event.

[0026] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0027] Therefore, faster and more efficient methods and interfaces are provided for integrating hardware and software into tracking user interfaces, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods for integrating hardware and software into tracking user interfaces. Attached Figure Description

[0028] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0029] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some implementation schemes.

[0030] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0031] Figure 2 Examples of portable multi-functional devices with touchscreens according to some implementation schemes are shown.

[0032] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0033] Figure 4A An exemplary user interface for a menu applied to a portable multi-functional device, according to some implementation schemes, is illustrated.

[0034] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is illustrated according to some embodiments.

[0035] Figure 5A Examples of personal electronic devices according to some implementation schemes are shown.

[0036] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0037] Figures 6A to 6I An exemplary user interface for capturing video is illustrated according to some implementation schemes.

[0038] Figure 7 This is a flowchart illustrating a method for capturing video according to some implementation schemes.

[0039] Figures 8A to 8P An exemplary user interface for capturing video is illustrated according to some implementation schemes.

[0040] Figure 9 This is a flowchart illustrating a method for capturing video according to some implementation schemes.

[0041] Figures 10A to 10H Exemplary techniques for performing animations using removable mounts, according to some implementation schemes, are illustrated.

[0042] Figure 11 This is a flowchart illustrating a method for performing animation using a movable mounting component, according to some implementation schemes. Detailed Implementation

[0043] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0044] Electronic devices require efficient methods and interfaces for integrating hardware and software into tracking user interfaces. In some implementations, a computer system selects a camera for capturing video based on whether the computer system is connected to a removable mount. In some implementations, a computer system enables a corresponding subject tracking operation mode based on whether the computer system is connected to a removable mount. In some implementations, the computer system causes the removable mount to perform one or more mechanical movements of the device connected to the removable mount. Such techniques reduce the cognitive burden on users of the integrated hardware and software for tracking user interfaces, thereby increasing productivity. Furthermore, such techniques reduce processor power and battery power otherwise wasted on redundant user input.

[0045] under Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5B A description of an exemplary device for performing management event notifications is provided. Figures 6A to 6I Exemplary techniques and user interfaces for capturing video are illustrated according to some implementation schemes. Figure 7 This is a flowchart illustrating a method for capturing video according to some implementation schemes. Figures 6A to 6G The user interface in this document is used to illustrate the processes described below, including... Figure 7 The process in. Figures 8A to 8P Exemplary techniques and user interfaces for capturing video are illustrated according to some implementation schemes. Figure 9 This is a flowchart illustrating a method for capturing video according to some implementation schemes. Figures 8A to 8P The user interface in this document is used to illustrate the processes described below, including... Figure 9 The process in. Figures 10A to 10H Exemplary techniques for performing animations using removable mounts, according to some implementation schemes, are illustrated. Figure 11 This is a flowchart illustrating a method for performing animation using a movable mounting component, according to some implementation schemes. Figures 10A to 10H The user interface in this document is used to illustrate the processes described below, including... Figure 11 The process in.

[0046] The processes described below enhance device operability and make the user-device interface more efficient through various technologies (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), including providing improved visual feedback to users, reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional display controls, performing operations without requiring further user input when a set of conditions are met, improving privacy, and / or additional technologies. These technologies also reduce power consumption and extend device battery life by enabling users to use the device faster and more efficiently.

[0047] Furthermore, in a method described herein where one or more steps depend on the satisfaction of one or more conditions, it should be understood that the described method can be repeated in multiple repetitions such that, during the repetitions, all conditions determining the steps in the method are satisfied in different repetitions of the method. For example, if the method requires performing a first step (if the conditions are satisfied) and a second step (if the conditions are not satisfied), those skilled in the art will know that the stated steps are repeated until both conditions are satisfied and not satisfied (in no particular order). Thus, a method described as having one or more steps depending on the satisfaction of one or more conditions can be rewritten as a method that repeats until each condition described in the method is satisfied. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing discretionary operations based on the satisfaction of the corresponding one or more conditions, and thus to determine whether possible conditions have been satisfied without explicitly repeating the steps of the method until all conditions determining the steps in the method are satisfied. Those skilled in the art will also understand that, similar to a method having discretionary steps, a system or computer-readable storage medium can repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.

[0048] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another. For example, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch, without departing from the scope of the various described embodiments. In some embodiments, a first touch and a second touch are two separate references to the same touch. In some embodiments, both a first touch and a second touch are touches, but they are not the same touch.

[0049] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in response to," or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determining..." or "in response to detecting [the stated condition or event]."

[0051] This document describes implementations of electronic devices, user interfaces for such devices, and associated processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. of Cupertino, California. Devices, iPod Equipment and Device. Optionally, other portable electronic devices may be used, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as display via a CRT display, via an LED display, or via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes displaying content (e.g., video data rendered or decoded by display controller 156) by sending data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection to visually generate content.

[0052] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0053] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0054] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or within the respective applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0055] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1AThis is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0056] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or change of the contact area detected on the touch-sensitive surface, the capacitance and / or change of the touch-sensitive surface adjacent to the contact, and / or the resistance and / or change of the touch-sensitive surface adjacent to the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement result). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of user input allows users to access additional device functionality that would otherwise be inaccessible to the user on a smaller device with limited physical space, which is used (e.g., on a touch-sensitive display) to display an indication and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0057] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or perceived by the user as the “roughness” of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of the user (e.g., “release click,” “press click,” “roughness”), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception described by a typical (or average) user.

[0058] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0059] Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0060] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0061] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, subscriber identity module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks (such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs))) and other devices. RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via short-range communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0062] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and sends the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and sends the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or sent to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both outputs (e.g., mono or binaural headphones) and inputs (e.g., a microphone).

[0063] I / O subsystem 106 couples input / output peripherals (such as touchscreen 112 and other input control devices 116) on device 100 to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / transmit electrical signals to the other input control device 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dials, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 is optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., ... Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication or via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures and / or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some implementations, air gestures are gestures detected without the user touching an input element that is part of the device (or independently of an input element that is part of the device) and based on the detected movement of a part of the user's body through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another of the user's fingers or a part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tapping gesture that includes the hand moving a predetermined amount and / or speed in a predetermined pose, or a shaking gesture that includes a predetermined speed or amount of rotation of a part of the user's body)).

[0064] A quick press of a push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of a push button (e.g., 206) optionally powers the device 100 on or off. The functionality of one or more of these buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0065] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0066] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0067] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that from Apple Inc. (Cupertino, California). and iPod The technology used.

[0068] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0069] The touch-sensitive display in some embodiments of the touchscreen 112 is described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, “Mode-Based Graphical User Interfaces For Touch Sensitive Input”. (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, “Multi-Functional Hand-Held Device”. The full text of all these applications is incorporated herein by reference.

[0070] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with touchscreen 112. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positioning or commands for performing the user-desired actions.

[0071] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0072] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0073] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also referred to as a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, facing away from a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the positioning of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensors within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0074] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor 175 is shown coupled to a depth camera controller 169 in I / O subsystem 106. The depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with an imaging module 143 (also referred to as a camera module), the depth camera sensor 175 is optionally used to determine depth maps of different portions of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while a user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, the depth camera sensor 175 is located at the rear of the device, or at both the rear and front of device 100. In some embodiments, the positioning of the depth camera sensor 175 can be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that the depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0075] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1AA contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0076] The device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally be configured as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, the proximity sensor is turned off and the touchscreen 112 is disabled when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

[0077] The device 100 may optionally also include one or more tactile output generators 167. Figure 1AA haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0078] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 is shown coupled to a peripheral device interface 118. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally be configured as described in the following U.S. Patent Publications: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods And Apparatuses For Operating A Portable DeviceBased On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0079] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, indicating what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information relating to the device's position and / or orientation.

[0080] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0081] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to… (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0082] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disengagement). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0083] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any of a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0084] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes: detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes: detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0086] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from applications, etc., to specify the graphics to be displayed, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to output to the display controller 156.

[0087] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0088] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., the contact module 137, the email client module 140, the IM module 141, the browser module 147, and any other application that requires text input).

[0089] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to telephone module 138 for use in location-based dialing; to camera module 143 as image / video metadata; and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0090] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0091] ●Contacts module 137 (sometimes called address book or contact list);

[0092] ● Telephone module 138;

[0093] ●Video conferencing module 139;

[0094] ●Email client module 140;

[0095] ●Instant Messaging (IM) module 141;

[0096] ● Fitness support module 142;

[0097] ● Camera module 143 for still images and / or video images;

[0098] ●Image Management Module 144;

[0099] ●Video player module;

[0100] ●Music player module;

[0101] ● Browser module 147;

[0102] ● Calendar module 148;

[0103] ● Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5 and other widgets acquired by the user, as well as user-created widget 149-6.

[0104] ● Wrapper module 150 for creating user-created widgets 149-6;

[0105] ●Search module 151;

[0106] ● Video and music player module 152, which combines a video player module and a music player module;

[0107] ●Notepad module 153;

[0108] ●Map module 154; and / or

[0109] ● Online video module 155.

[0110] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0111] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via the telephone module 138, video conferencing module 139, email client module 140, or IM module 141; and so on.

[0112] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify input telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0113] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0114] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0115] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: entering a character sequence corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing received instant messages. In some embodiments, the sent and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" means both telephone-based messages (e.g., messages delivered using SMS or MMS) and internet-based messages (e.g., messages delivered using XMPP, SIMPLE, or IMPS).

[0116] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for performing the following operations: creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0117] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0118] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for performing operations such as arranging, modifying (e.g., editing) or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0119] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for performing the following operations: browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0120] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0121] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a micro-application downloaded and used by a user (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-created micro-application (e.g., user-created widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! widgets).

[0122] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 is optionally used by the user to create widgets (e.g., turning user-specified portions of a webpage into widgets).

[0123] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for performing the following operations: searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0124] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play back recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0125] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0126] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0127] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Additional descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0128] Each of the modules and applications described above corresponds to an executable set of instructions for performing one or more functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores a subset of the above-described modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0129] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (such as push-buttons and dial pads) on device 100 is optionally reduced.

[0130] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, main desktop menu, or root menu. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0131] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of the aforementioned applications 137 to 151, 155, 380 to 390).

[0132] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0133] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0134] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0135] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 sends event information. In other implementations, peripheral device interface 118 sends event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0136] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0137] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0138] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined, at least in part, based on the hit view of the initial touch that initiates the touch-based gesture.

[0139] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0140] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0141] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0142] In some implementations, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another implementation, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0143] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0144] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0145] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).

[0146] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in events (e.g., 187-1 and / or 187-2) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lifting the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0147] In some implementations, event definition 186 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0148] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0149] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0150] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0151] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from delivering (and deferred delivering) the sub-events to the corresponding hit view. In some implementations, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag retrieves the flag and performs a predefined process.

[0152] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined process.

[0153] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the positioning of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and transmits that display information to graphics module 132 for display on a touch-sensitive display.

[0154] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0155] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0156] Figure 2A portable multifunction device 100 with a touchscreen 112 is illustrated according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0157] Device 100 optionally also includes one or more physical buttons, such as a "main desktop" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0158] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headphone jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to: power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0159] Figure 3This is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The described tactile output generator 167) and sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one described above)) Figure 1A The described contact strength sensor 165). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0160] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each module described above corresponds to a set of instructions for performing the functions described above. The modules or computer programs described above (e.g., instruction sets or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0161] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0162] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is illustrated. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:

[0163] ●Signal strength indicator 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0164] ●Time 404;

[0165] ● Bluetooth indicator 405;

[0166] ● Battery status indicator 406;

[0167] ● Tray 408 features icons for frequently used applications, such as:

[0168] ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemail messages;

[0169] ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails;

[0170] ○ The icon 420 labeled "Browser" in browser module 147; and

[0171] ○ The video and music player module 152 (also known as the iPod (Apple Inc. trademark) module 152) is marked with an icon 422 labeled "iPod"; and

[0172] ● Icons of other applications, such as:

[0173] ○The icon 424 of the IM module 141 marked as "Message";

[0174] ○The calendar module 148 has an icon 426 labeled "Calendar";

[0175] ○ The icon 428 of the image management module 144 is labeled "Photo".

[0176] ○ The icon 430 of camera module 143, which is labeled "camera";

[0177] ○ The icon 432 of the online video module 155, which is labeled "Online Video";

[0178] ○ The icon 434 labeled "Stock Market" in the Stock Market widget 149-2;

[0179] ○The icon 436 of the map module 154 that is labeled "map";

[0180] ○The weather widget 149-1 has icon 438 labeled "weather";

[0181] ○ The alarm clock widget 149-4 has an icon 440 labeled "clock";

[0182] ○ The icon 442 of the fitness support module 142 is labeled "fitness support";

[0183] ○ The icon 444 labeled "Notepad" in Notepad module 153; and

[0184] ○ Set an icon 446 labeled "Settings" for an application or module, which provides access to settings for device 100 and its various applications 136.

[0185] It should be pointed out that, Figure 4A The illustrated icon labels are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0186] Figure 4B An example is illustrated having a touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)). Figure 3 A tablet device or touchpad 355) device (e.g., Figure 3An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0187] While some examples of inputs on a touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations, the device detects inputs on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450) corresponding to the main axis on the display (e.g., Figure 4B The main shaft of 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to a specific location on the display (e.g., in the example). Figure 4B In the middle, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on that touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0188] Additionally, while the examples below are given primarily with reference to finger input (e.g., finger touch, single-finger tap gesture, finger swipe gesture), it should be understood that in some implementations, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the touch). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or mouse and finger touch are optionally used simultaneously.

[0189] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B The device 500 may include some or all of the features described herein. In some embodiments, the device 500 has a touch-sensitive display 504, referred to below as a touchscreen 504. Alternatively, or in addition to the touchscreen 504, the device 500 may also have a display and a touch-sensitive surface. Similar to the cases of devices 100 and 300, in some embodiments, the touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors of the touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of the device 500 may respond to touches based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0190] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying UserInterface Objects Corresponding to an Application,” published as WIPO Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between TouchInput to Display Output Relationships,” published as WIPO Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0191] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) may be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0192] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, device 500 may include, relative to... Figure 1A , Figure 1B and Figure 3 Some or all of the components described herein. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive component 522 and optionally have intensity sensor 524 (e.g., contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 may optionally be a button.

[0193] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0194] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, may cause those computer processors to perform, for example, the techniques described below, including processes 700, 900, and 1100. Figure 7 , Figure 9 and Figure 11A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0195] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5B A user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0196] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of enabling direct interaction with user interface elements on the touchscreen display) Figure 1A The touch-sensitive display system 112 or Figure 4AIn some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0197] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to lift off, before or after contact begins to move, before contact ends, before or after contact intensity increases and / or before or after contact intensity decreases). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, or the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether a user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the characteristic intensity and one or more thresholds is used to determine whether one or more actions should be performed (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0198] In some implementations, the computer system is in a locked or unlocked state. In the locked state, the computer system is powered on and operable, but it is prevented from performing a predefined set of operations in response to user input. This predefined set of operations optionally includes navigation between user interfaces, activation or deactivation of a predefined set of functions, and activation or deactivation of certain applications. The locked state can be used to prevent unintentional or unauthorized use of a function of the computer system, or to activate or deactivate some functions on the computer system. In some implementations, in the unlocked state, the computer system is powered on and operable, and is not prevented from performing at least some of the predefined set of operations that cannot be performed when in the locked state. When the computer system is in a locked state, it is said that the computer system is locked. When the computer system is in an unlocked state, it is said that the computer is unlocked. In some implementations, a locked computer system optionally responds to a limited set of user inputs, including inputs corresponding to an attempt to unlock the computer system or inputs corresponding to shutting down the computer system.

[0199] Now let’s turn our attention to the implementation of user interfaces (“UIs”) and associated processes on electronic devices (such as portable multifunction devices 100, 300 or 500).

[0200] Figures 6A to 6I Exemplary techniques and user interfaces for capturing video with and without a removable mount, according to some embodiments, are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 7 The process in.

[0201] Figure 6A An example is a computer system 600, which includes two or more camera sensors. Figure 6A In this context, computer system 600 is a smartphone. In some embodiments, computer system 600 is a tablet computer, laptop computer, desktop computer, smartwatch, camera device, and / or electronic control system. In some embodiments, computer system 600 is computer system 800 and / or computer system 1000 as described below. Figure 6A In the illustrated embodiment, the computer system 600 has a first camera sensor 602 positioned relative to the rear of the computer system 600 and a second camera sensor 604 positioned relative to the front of the computer system 600. Figure 6AA bracket 606 (e.g., a fixed or movable bracket) and a movable mount 608 are depicted. In some embodiments, bracket 606 is bracket 802. In some embodiments, movable mount 608 is movable mount 804 or movable mount 1002. Figure 6A As shown, the computer system 600 is initially placed on a stand 606. The computer system 600 has a display 610, which faces the user when the computer system 600 is on the stand 606.

[0202] Computer system 600 can be connected to movable mount 608. Movable mount 608 includes one or more mechanisms that enable movement of the movable mount 608, thereby moving the computer system 600 when connected to the movable mount 608. For example, movable mount 608 can rotate, pan, tilt, and / or roll, thereby causing the computer system 600 to rotate, pan, tilt, and / or roll, respectively. Figure 6B In the example shown, the movable mount 608 includes two movable joints: joint 608a and joint 608b. Joint 608a allows rotation about a first axis (e.g., an azimuth axis), thereby enabling the movable mount 608 to pan the computer system 600. Joint 608b allows rotation about one or more other axes (e.g., a pitch axis and / or a roll axis), enabling the movable mount 608 to tilt and / or roll the computer system 600. For example, rotation about the roll axis enables the movable mount 608 to rotate the computer system 600 between longitudinal and lateral orientations.

[0203] exist Figure 6B At this point, computer system 600 detects input 650 (e.g., a tap and / or other input) to select camera application icon 612. Figure 6C As illustrated, in response to the detection of input 650 selecting the camera application icon 612, the computer system 600 displays the camera application user interface 614. The camera application user interface 614 indicates various available camera modes (e.g., video, photo, portrait, time-lapse, and / or slow motion). Figure 6C In this configuration, the camera mode is set to video (e.g., as indicated by a video option that is centered and bold compared to other options), causing the computer system 600 to capture and record video. The camera application user interface 614 includes recording controls 616. Figure 6C At this point, computer system 600 detects input 652 to recording control 616 (e.g., tap and / or other input).

[0204] like Figure 6DAs illustrated, in response to detecting input 652 to recording control 616, computer system 600 initiates a process of capturing video using a second camera sensor 604 (e.g., a front-facing camera sensor). The second camera sensor 604 has a field of view 618 (e.g., by...). Figure 6D (Indicated by the dashed line at the top). During the process of capturing video using the second camera sensor 604, the camera application records video within the field of view 618. Figure 6D In the computer system 600, the video of the field of view 618 and the recording indicator 620 indicating that the video is being captured are displayed on the monitor 610.

[0205] exist Figure 6E At this point, the user removes the computer system 600 from the bracket 606 and places it on the removable mount 608. Since the computer system 600 is connected to the removable mount 608 (e.g., based on a determination that the computer system 600 is connected to the removable mount 608), the computer system 600 initiates a process of capturing video using the first camera sensor 602 (e.g., a rear camera sensor). The first camera sensor 602 has a field of view 622. In this example, the user places the computer system 600 on the removable mount 608 such that the display 610 faces away from the user and the first camera sensor 602 faces the user. In some embodiments, if the user orients the display 610 toward the user and the first camera sensor 602 away from the user, the removable mount 608 reorients the computer system 600 such that the display 610 faces away from the user and the first camera sensor 602 faces the user.

[0206] In this example, the first camera sensor 602 has a narrower field of view than the second camera sensor 604. This difference is due to... Figure 6C and Figure 6D The field of view 618 of the second camera sensor 604 depicted is exemplified, which is larger than... Figure 6E The field of view 622 of the first camera sensor 602 depicted is wider.

[0207] In this example, the first camera sensor 602 has higher image quality than the second camera sensor 604. In some embodiments, the first camera sensor 602 has reduced image distortion relative to the second camera sensor 604 (e.g., due to a smaller field of view relative to the second camera sensor 604) and / or better low-light sensitivity relative to the second camera sensor 604 (e.g., due to a larger pixel count relative to the second camera sensor 604). When the computer system 600 is using the second camera sensor 604, the difference in image quality is due to… Figure 6C and Figure 6DThe shading shown is an illustration (e.g., the shading is for illustrative purposes only and is not actually displayed on the computer system 600). However, when the computer system 600 is using the first camera sensor 602, the second camera sensor 604 has a higher image quality, therefore... Figure 6E The shadow lines are not shown in the image.

[0208] exist Figure 6F In the illustrated embodiments, when (or in some embodiments, because) the computer system 600 is connected to the movable mount 608, a corresponding subject tracking operation mode is used. In the corresponding subject tracking operation mode, the computer system 600 adjusts the field of view 622 of the first camera sensor 602, transmits tracking data to the movable mount 608, and / or moves the movable mount 608 to track the movement of one or more subjects (e.g., a user and / or an object of interest).

[0209] exist Figure 6F In the example shown, the user moves to one side of the movable mount 608. Therefore, the computer system 600 moves the movable mount 608, thereby changing the field of view 622 of the first camera sensor 602, keeping the user within the field of view 622. In some embodiments, when a corresponding subject tracking operation mode is enabled, the computer system 600 tracks the one or more subjects mechanically, such as by mechanically zooming the first camera sensor 602 and / or by mechanically panning the first camera sensor 602 by rotating the movable mount 608, keeping the one or more subjects within the field of view 622.

[0210] In some implementations, based on (or in response to) determining that the computer system 600 is connected to the removable mount 608, the corresponding subject tracking operation mode is automatically enabled. In some implementations, based on (or in response to) determining that the computer system 600 is disconnected from the removable mount 608, the corresponding subject tracking operation mode is automatically disabled.

[0211] exist Figure 6G In this process, the computer system 600 is removed from the movable mount 608 and placed on the bracket 606. Consequently, the corresponding subject tracking operation mode is automatically disabled, and the computer system 600 returns to using the second camera sensor 604. Figure 6G The example illustrates computer system 600, which is more advanced than previously seen. Figure 6F The image displayed is of lower quality, showing the wider field of view 618 of the second camera sensor 604.

[0212] exist Figure 6HIn the illustrated implementation, the user moves to one side of the bracket 606. However, since the corresponding subject tracking operation mode is not enabled, the field of view 618 of the second camera sensor 604 remains unchanged, and the user is no longer centered in the field of view 618.

[0213] In some implementations, when the corresponding subject tracking operation mode is not enabled, the computer system 600 digitally tracks the one or more subjects (e.g., by digitally zooming, panning, tilting, and / or rotating the second camera sensor 604) without making any mechanical adjustments. For example, Figure 6I An example is illustrated where computer system 600 digitally tracks a user, centering the user within the field of view 624 of a second camera sensor 604. In this scenario, field of view 618 represents the entire range of the image that the second camera sensor 604 can capture. However, computer system 600 digitally zooms to field of view 624, thereby limiting the portion of the field of view displayed on display 610, thus centering the user within the displayed video.

[0214] Figure 7 This is a flowchart illustrating a method for initiating a process of capturing video using a computer system, according to some embodiments. Method 700 is performed at a computer system (e.g., 100, 300, 500, 600, 800, 1000, smartphone, tablet, laptop, desktop computer, smartwatch, camera device, and / or electronic control system) communicating with two or more camera sensors and one or more input devices (e.g., touch-sensitive surfaces, touchscreen displays, buttons, keyboards, mice, joysticks, camera sensors, and / or microphones). Some operations in method 700 may be combined, the order of some operations may be changed, and some operations may be omitted.

[0215] As described below, Method 700 provides an intuitive way to initiate the process of capturing video. This method reduces the cognitive burden on the user regarding initiating the video capture process, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to initiate video capture faster and more efficiently saves power and increases the time interval between battery charging.

[0216] The computer system detects (702) a request (e.g., a set of one or more inputs) (e.g., 650 and / or 652) via the one or more input devices to capture video using the field of view (e.g., 618) of one or more camera sensors (e.g., the field of view of a first camera sensor among the two or more camera sensors and / or the field of view of a second camera sensor among the two or more camera sensors), the field of view being based on the movement of one or more subjects (e.g., a person moving from their position within the field of view of the one or more camera sensors) within the field of view of the one or more camera sensors. Figure 6E The position in Figure 6F The field of view is altered (e.g., adjusted mechanically and / or electronically) by the movement of the user's position. In some implementations, the field of view is adjusted (e.g., translated and / or zoomed in or out) to follow or track a user or object moving from one location to another.

[0217] In response to a request to detect captured video (704): Based on (e.g., in response to) determining that the computer system is connected to a movable mount (e.g., 608) (e.g., a mount configured to physically support or bear the computer system via gravity, magnetic mechanisms, and / or mechanical mechanisms) (e.g., physically connected to the movable mount, physically mounted on the movable mount, and / or communicating with the movable mount via a wired or wireless connection) (e.g., computer system 600 is connected to...) Figure 6E The computer system initiates (706) a process of capturing video using a first camera sensor (e.g., 602) of the two or more camera sensors (a movable mount 608). In some embodiments, the mount has a motor and / or controls capable of rotating, panning (e.g., rotating about a vertical axis), tilting (e.g., rotating about a horizontal axis), and / or rolling (e.g., rotating about a second horizontal axis perpendicular to both the horizontal and vertical axes) one or more portions of the mount. In some embodiments, the computer system has a rechargeable battery, and the movable mount charges the battery (e.g., via wireless charging) when the computer system is connected to the movable mount. In some embodiments, initiating the video capture process includes displaying selectable graphical elements (e.g., buttons, icons, and / or power indicators) that, when selected, cause video (e.g., via one or more of the one or more camera sensors) to be captured. In some embodiments, initiating the video capture process includes detecting a series of inputs, including inputs corresponding to a request to launch an application for video capture (e.g., a camera application). In some implementations, initiating a video capture process includes performing a video capture procedure and / or capturing the video.

[0218] In response to a request to detect captured video (704): Based on (e.g., in response to) determining that the computer system is not connected to the removable mount (e.g., not physically connected to the removable mount, not physically mounted on the removable mount, and / or not communicating with the removable mount) (e.g., computer system 600 is located in...), Figure 6B On the bracket 606 in the middle, the computer system initiates (708) a process of capturing video using a second camera sensor (e.g., 604) that is different from the first camera sensor among the two or more camera sensors. Using the first camera sensor when the computer system is connected to the removable mount and using the second camera sensor when the computer system is not connected to the removable mount enables the computer system to use a better camera to capture video in a given situation without requiring additional input from the user, thereby performing improved operations without further user input when a set of conditions has already been met, and reducing the amount of input required to perform the operation.

[0219] In some implementations, the first camera sensor has a narrower field of view than the second camera sensor (e.g., 622 compared to 618) (e.g., the second camera is a wide-angle camera). Initiating a process to capture video using a camera sensor with a narrower field of view when the computer system is connected to a removable mount enables the computer system to automatically use a camera with better optical properties to capture video, thereby performing improved operations without further user input once a set of conditions are met, and reducing the amount of input required to perform the operation.

[0220] In some implementations, the first camera sensor has higher image quality (e.g., higher resolution) than the second camera sensor (e.g., with...). Figure 6D Compared to the image with shading shown Figure 6E (See the image shown). When a computer system is connected to a removable mount, it initiates a process to capture video using a camera sensor with higher image quality. This enables the computer system to automatically use a camera with better optical properties to capture video, thereby performing improved operations when a set of conditions are already met without further user input and reducing the amount of input required to perform the operation.

[0221] In some implementations, the computer system detects that it has been attached to a removable installation when the corresponding subject tracking mode is disabled. In some implementations, in response to detecting that the computer system is connected to the removable installation, the computer system enables the corresponding subject tracking operation mode (e.g., enables tracking for...). Figures 8G to 8P(The tracking of human movement is illustrated) (e.g., tracking is automatically enabled when the computer system is connected to the movable mount). In some embodiments, enabling the corresponding subject tracking operation mode includes: enabling the corresponding subject tracking algorithm, adjusting the field of view of the camera sensor, transmitting tracking data (e.g., from the computer system) to the movable mount, and / or (e.g., based on the tracking data) moving the movable mount. In some embodiments, in the corresponding subject tracking operation mode, the computer system adjusts the field of view of the first camera sensor, transmits tracking data to the movable mount, and / or moves the movable mount to track the movement of one or more subjects (e.g., a user and / or an item of interest). In some embodiments, when the corresponding subject tracking operation mode is enabled, the computer system tracks the one or more subjects mechanically, such as by mechanically zooming the first camera sensor and / or mechanically panning the first camera sensor by rotating the movable mount, so that the one or more subjects remain in the field of view. In some implementations, when a corresponding subject tracking operation mode is enabled, the computer system tracks one or more subjects by combining mechanical and digital adjustments to the field of view (e.g., mechanically panning and digitally zooming the first camera sensor by rotating a movable mount). Automatically enabling the corresponding subject tracking operation mode when the computer system is connected to the movable mount reduces the amount of input required to enable it and allows the user to enable the mode without displaying additional controls. This allows operation to be performed when a set of conditions is met without further user input, reducing the amount of input required to perform the operation and providing additional control options without cluttering the user interface with additional display controls.

[0222] In some implementations, based on (or in some implementations, in response to) determining that the computer system has been removed from the removable installation, the computer system disables the corresponding subject tracking operating mode (e.g., tracking is automatically disabled when the computer system is disconnected from the removable installation) (e.g., see reference). Figure 6G(As described). In some implementations, when the corresponding subject tracking operation mode is enabled, the computer system receives (e.g., detects) an instruction to remove the computer system from the removable installation; and in response to receiving the instruction to remove the computer system from the removable installation, the computer system disables the corresponding subject tracking operation mode. Automatically disabling the corresponding subject tracking operation mode based on determining that the computer system has been removed from the removable installation reduces the amount of input required to disable the corresponding subject tracking operation mode, and allows the user to disable the corresponding subject tracking operation mode without displaying additional controls. This allows operations to be performed when a set of conditions has already been met without requiring further user input, reducing the amount of input required to perform the operation, and providing additional control options without cluttering the user interface with additional display controls.

[0223] In some implementations, the computer system detects changes in the spatial arrangement of one or more subjects detected by the one or more camera sensors (e.g., a person's position from their position in...). Figure 6E The position in Figure 6F (e.g., the appearance of one or more subjects, the disappearance of one or more subjects, and / or the movement of one or more subjects in the physical environment). In some embodiments, in response to detecting a change in the spatial arrangement of the one or more subjects, the field of view of the one or more camera sensors is changed based on the change in the spatial arrangement of the one or more subjects (e.g., field of view 622 from...). Figures 6E to 6F (Changes in the spatial arrangement of one or more of these subjects) to track one or more subjects. In response to detecting a change in the spatial arrangement of one or more subjects, the field of view of the camera sensor is changed to track the one or more subjects. This reduces the amount of input required to track these subjects based on changes in the spatial arrangement of one or more subjects, thereby performing operations when a set of conditions are already met without further user input, and reducing the amount of input required to perform operations.

[0224] In some embodiments, altering the field of view of one or more camera sensors to track one or more of these subjects based on changes in the spatial arrangement of the one or more subjects includes: mechanically altering the field of view of the one or more camera sensors based on determining that a first camera sensor is being used to capture video (and / or in some embodiments, based on determining that a computer system is connected to a movable mount). This can be done by mechanically moving the computer system 600 and thereby removing the first camera sensor 602 and its corresponding field of view 622 from the computer system 600. Figure 6E Move to the position shown Figure 6F(e.g., mechanically zooming the first camera sensor and / or mechanically panning the first camera sensor by rotating a movable mount to maintain the one or more subjects within the field of view of the one or more camera sensors). In some embodiments, changing the field of view of the one or more camera sensors to track one or more of these subjects based on changes in the spatial arrangement of the one or more objects includes changing the field of view of the one or more camera sensors both mechanically and digitally. In some embodiments, mechanically tracking the one or more subjects includes adjusting one or more physical components of the first camera sensor, computer system, and / or movable mount to change the field of view of the first camera sensor, such as zooming, panning, tilting, and / or rotating the field of view of the first camera sensor. In some embodiments, the one or more subjects include people and / or objects of interest. Determining that the first camera sensor is being used to capture video to mechanically track one or more subjects enables the computer system to track subjects using improved settings when using the first camera sensor, thereby performing operations when a set of conditions are already met without requiring further user input and reducing the amount of input required to perform the operation.

[0225] In some implementations, altering the field of view of one or more camera sensors to track one or more of these subjects based on changes in the spatial arrangement of the one or more subjects includes: digitally altering the field of view of the one or more camera sensors (e.g., based on determining that a second camera sensor is being used to capture video (and / or in some implementations, based on determining that a computer system is not connected to a removable mount) based on determining that a second camera sensor is being used to capture video (and / or, based on determining that a computer system is not connected to a removable mount). Figure 6I The change from field of view 618 to field of view 624 (e.g., digitally zooming the second camera sensor and / or digitally panning the second camera sensor without mechanical adjustments to maintain the one or more subjects within the field of view of the one or more camera sensors). In some embodiments, digitally tracking the one or more subjects includes digitally adjusting the field of view of the first camera sensor, such as zooming, panning, tilting, and / or rotating the field of view of the second camera sensor. In some embodiments, the one or more subjects include people and / or objects of interest. Determining that the second camera sensor is being used to capture video for digitally tracking one or more subjects enables a computer system to track subjects using improved settings when using the second camera sensor, thereby performing operations when a set of conditions are already met without further user input and reducing the amount of input required to perform the operations.

[0226] It should be noted that the above text regarding method 700 (for example, Figure 7The details of the process described herein also apply in a similar manner to the methods described below. For example, methods 900 and 1100 optionally include one or more features of the various methods described above with reference to method 700. For example, a method for capturing video using a first or second camera sensor may be combined with a method for capturing video based at least in part on whether the computer system is connected to a movable mount, according to a corresponding subject tracking operation mode. For the sake of brevity, these details will not be repeated below.

[0227] Figures 8A to 8P Exemplary techniques and user interfaces for capturing video with and without a removable mount, according to some embodiments, are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 9 The process in.

[0228] Figure 8A Example includes a computer system 800, a bracket 802, and a movable mounting component 804. In Figure 8A In this context, computer system 800 is a smartphone. In some implementations, computer system 800 is a tablet computer, laptop computer, desktop computer, smartwatch, camera device, and / or electronic control system. For example... Figure 8A As shown, computer system 800 is initially placed on stand 802. Computer system 800 has a display 806 that faces the user when computer system 800 is on stand 802. In some embodiments, computer system 800 is computer system 600 and / or 1000; stand 802 is stand 606; and movable mount 804 is movable mount 608 or movable mount 1002.

[0229] exist Figure 8B At this point, computer system 800 detects input 850 (e.g., a tap and / or other input) from selecting the camera application icon 808. Figure 8C As illustrated, in response to the detection of input 850 selecting the camera application icon 808, the computer system 800 displays the camera application user interface 810. The camera application user interface 810 indicates various available camera modes (e.g., video, photo, portrait, time-lapse, and / or slow motion). Figure 8C In this configuration, the camera mode is set to video (e.g., as indicated by a video option that is centered and bold compared to other options), causing the computer system 800 to capture and record video. The camera application user interface 810 includes recording controls 812. Figure 8C At this point, the computer system 800 detects input 852 to the recording control 812 (e.g., a tap and / or other input).

[0230] like Figure 8D As illustrated, in response to the detection of input 852 to the recording control 812, the computer system 800 initiates a process of capturing video using one or more camera sensors. Field of view 814 (e.g., by...) Figure 8D The dotted line at the top indicates the field of view of the one or more camera sensors and / or the portion of the field of view of the one or more camera sensors displayed on display 806. During the process of capturing video using the one or more camera sensors, the camera application records video within the field of view 814. Figure 8D In the computer system 800, a video field 814 and / or a portion thereof, as well as a recording indicator 816 indicating that video is being captured, are displayed on a monitor 806.

[0231] In some implementations, the process of capturing video when the computer system 800 is mounted on the bracket 802 uses different logic than when the computer system 800 is mounted on the removable mount 804. For example, in response to detecting input 852 to the recording control 812, and based on determining that the computer system 800 is connected to the removable mount 804, the computer system 800 initiates a process of capturing video according to a corresponding subject tracking operation mode. In this case, initiating the process of capturing video according to the corresponding subject tracking operation mode includes using logic for tracking the subject, such as by physically moving and / or zooming the computer system 800 and / or the one or more camera sensors. In response to detecting input 852 to the recording control 812, and based on determining that the computer system 800 is not connected to the removable mount 804, the computer system 800 initiates a process of capturing video without enabling the corresponding subject tracking operation mode. In this context, the process of initiating video capture without enabling the corresponding subject tracking operation mode includes different logic for tracking the subject, such as digitally moving and / or zooming the one or more camera sensors (e.g., without physically moving and / or zooming the computer system 800 and / or the one or more camera sensors).

[0232] exist Figure 8E At this location, computer system 800 is positioned on bracket 802, and the corresponding subject tracking operation mode is not enabled. In the example shown, first person 840a is gazing at an object of interest (e.g., a whiteboard). Because the corresponding subject tracking operation mode is not enabled, field of view 814 remains unchanged, and only a portion of the whiteboard is visible within field of view 814.

[0233] exist Figure 8FAt this location, computer system 800 is positioned on bracket 802, and the corresponding subject tracking operation mode is not enabled. In the example shown, a second person 840b approaches a first person 840a. Although the first person 840a is looking at the second person 840b, the field of view 814 remains unchanged because the corresponding subject tracking operation mode is not enabled, and the second person 840b cannot be seen in the field of view 814.

[0234] exist Figure 8G In this process, the first person 840a removes the computer system 800 from the bracket 802 and places the computer system 800 on the removable mount 804. In some embodiments, after the computer system 800 has been previously operated without being connected to the removable mount 804, and without enabling the corresponding subject tracking operation mode, the computer system 800 automatically enables the corresponding subject tracking operation mode based on (or in response to) determining that the computer system 800 is connected to the removable mount 804.

[0235] In some embodiments, the computer system 800 causes an output indicating that a corresponding subject tracking operation mode is enabled. In some embodiments, the computer system 800 activates an LED indicator (e.g., located on the movable mount 804 and / or on the computer system 800) based on determining that a corresponding subject tracking operation mode is enabled. For example, the computer system 800 causes the LED indicator to flash based on determining that a corresponding subject tracking operation mode is enabled and is being actively used to track a subject (such as the first person 840a), and the computer system 800 causes the LED indicator to remain constantly lit based on determining that a corresponding subject tracking operation mode is enabled but is not being actively used to track a subject.

[0236] exist Figure 8H At this location, computer system 800 is connected to movable mount 804, and a corresponding subject tracking operation mode is enabled. In the corresponding subject tracking operation mode, computer system 800 adjusts the field of view 814 of the one or more camera sensors and / or the display portion of the field of view 814, transmits tracking data to movable mount 804, and / or moves movable mount 804 to track one or more subjects (e.g., a user and / or an object of interest). In some embodiments, the corresponding subject tracking operation mode tracks a specific subject (e.g., the opposite of the centroid method, which attempts to keep two or more subjects in the field of view when there are two or more subjects of interest).

[0237] In some embodiments, the specific subject tracked in the corresponding subject tracking operation mode is the first person detected within the field of view 814 when the corresponding subject tracking operation mode is enabled. In some embodiments, the subject tracked in the corresponding subject tracking operation mode is selected for tracking based on facial detection (e.g., by identifying facial features). In some embodiments, the subject tracked in the corresponding subject tracking operation mode is selected for tracking based on body detection (e.g., by identifying the torso and / or limbs). In some embodiments, the subject tracked in the corresponding subject tracking operation mode is selected for tracking based on one or more facial features associated with a specific person (e.g., facial features associated with the device's primary user, a person saved as a contact, and / or a person previously identified by computer system 800).

[0238] In some cases, when a corresponding subject tracking operation mode is enabled, the computer system 800 detects more than one subject within the field of view 814. In some embodiments, a specific subject is tracked in the corresponding subject tracking operation mode because that specific subject is the person closest to the center of the field of view 814 when the corresponding subject tracking operation mode is enabled. In some embodiments, a specific subject is tracked in the corresponding subject tracking operation mode because that specific subject is the person closest to the center of the field of view 814 who is looking at the one or more camera sensors when the corresponding subject tracking operation mode is enabled.

[0239] exist Figure 8H In the example shown, the corresponding subject tracking operation mode tracks the first person 840a because the first person 840a is the first person detected within the field of view 814 when the computer system 800 is connected to the movable mount 804 and the corresponding subject tracking operation mode is enabled. In some embodiments, if the subject looks away from the camera and looks at a person or location for a predetermined period of time (e.g., 1 second, 2 seconds, or 5 seconds), the computer system 800 identifies that person or location as an object of interest. Figure 8H In this process, the first person 840a shifts their gaze away from the one or more camera sensors and focuses on an object of interest (e.g., a whiteboard). Since a corresponding subject tracking operation mode is enabled, the computer system 800 adjusts its field of view 814 based on the first person 840a's gaze on the object of interest (e.g., the whiteboard), such as by rotating, panning, tilting, and / or rolling the movable mount 804. Figure 8H For example, the movable mount 804 rotates and / or pans the computer system 800, making the entire whiteboard visible in the field of view 814.

[0240] exist Figure 8IAt this location, computer system 800 is connected to movable mount 804, and a corresponding subject tracking operation mode is enabled. In the example shown, a second person 840b approaches a first person 840a, and the first person 840a is looking at the second person 840b. Because the corresponding subject tracking operation mode is enabled, computer system 800 adjusts its field of view 814 based on the first person 840a looking at the second person 840b, such as by rotating, panning, tilting, and / or rolling computer system 800 around movable mount 804. Figure 8H For example, the movable mount 804 rotates and / or pans the computer system 800, making the second person 840b visible in the field of view 814.

[0241] In some embodiments, after the computer system identifies the item of interest and / or an additional subject to be tracked (such as a second person 840b), in addition to tracking the first person 840a, the computer system 800 also tracks (or also causes the computer system to track) the item of interest and / or the additional subject (such as the second person 840b). In some embodiments, the second person 840b is identified as another person to be tracked because the first person 840a is looking at the second person 840b. In some embodiments, the second person 840b is identified as another person to be tracked because the second person 840b is facing and / or looking at the one or more camera sensors. When more than one subject is being tracked, the computer system 800 adjusts the display portion of the field of view 814 as much as possible to include all subjects. For example, in Figure 8I In the middle, the computer system 800 adjusts the field of view 814 to include the first person 840a, the second person 840b, and the whiteboard.

[0242] In some implementations, the computer system 800 enables a corresponding subject tracking operation mode to track a specific subject (e.g., instead of zooming out to make multiple subjects visible in the field of view 814). Figure 8J As shown, when the first person 840a moves away from the whiteboard and the second person 840b moves, the computer system 800 adjusts the field of view 814 to track the first person 840a (e.g., instead of narrowing it so that the first person 840a, the whiteboard, and the second person 840b are all visible within the field of view 814). In the example shown, the computer system 800 rotates the movable mount 804 and / or pans the computer system 800 so that the first person 840a remains centered in the field of view 814.

[0243] In some cases, it is physically impossible for the computer system 800 to adjust the field of view 814 to include the corresponding subject and one or more identified objects of interest. For example, in Figure 8KIn the scenario shown, even if the field of view 814 is reduced, it is impossible for the field of view 814 to include the first person 840a, the second person 840b, and the whiteboard based on the positions of the one or more camera sensors. In this case, the computer system 800 continues to track the first person 840a according to the corresponding subject tracking operation mode and stops adjusting the display portion of the field of view 814 to include the second person 840b and / or the whiteboard.

[0244] exist Figure 8K At this point, computer system 800 tracks first person 840a according to a corresponding subject tracking operation mode, and first person 840a moves outside of field of view 814. For example, first person 840a moves and computer system 800 is unable to adjust field of view 814 to keep first person 840a within field of view 814 (e.g., because the speed of movement of first person 840a differs from the speed at which computer system 800 adjusts field of view 814). In some embodiments, the corresponding subject tracking operation mode causes computer system 800 to continue adjusting field of view 814 based on the subject's last known trajectory. For example, if computer system 800 is adjusting field of view 814 to follow first person 840a's movement in a particular direction, after first person 840a moves outside of field of view 814, computer system 800 will continue to adjust field of view 814 in the same direction in an attempt to reposition first person 840a within field of view 814.

[0245] In some implementations, the corresponding subject tracking mode includes changing the display portion of the field of view 814 based on one or more movement characteristics of the tracked subject (such as the amount of movement and / or speed of movement of the first person 840a). In some implementations, the computer system 800 changes the display portion of the field of view 814 if the movement meets a predetermined threshold (e.g., a predetermined distance and / or a predetermined speed). For example, the computer system 800 changes the display portion of the field of view 814 based on the movement of the first person 840a from its position in the field of view. Figure 8I The position in Figure 8J The display portion of the field of view 814 is changed by moving the position within it. In some embodiments, if the movement does not meet a predetermined threshold, the display portion of the field of view remains unchanged. For example, based on the first person 840a speaking and / or gesturing but otherwise remaining in the field of view... Figure 8G As shown in the approximate position, the computer system 800 does not change the display portion of the field of view 814.

[0246] Figure 8LThis example illustrates that when the first person 840a moves outside the field of view 814, the computer system 800 continues to adjust the field of view 814 along the trajectory in which the first person 840a is moving. Since the first person 840a continues to move in the same direction and the computer system 800 adjusts the field of view 814 in the same direction, the computer system 800 repositions the first person 840a within the field of view 814, as shown below. Figure 8L As shown.

[0247] In some embodiments, the movable mount 804 has one or more movement limits, and when a movement limit is reached, the computer system 800 cannot adjust the field of view 814 beyond that limit. For example, the movable mount 804 cannot rotate (e.g., about a vertical axis) a total of more than 360 degrees. Starting from a 0-degree starting position, the movable mount 804 can rotate 180 degrees clockwise and 180 degrees counterclockwise. Once the movable mount has reached 180 degrees in either direction, the movement limit is reached, and the movable mount 804 cannot move further in that direction. As another example, the movable mount 804 cannot move beyond a predetermined pitch angle. Figure 8M As illustrated, the first person 840a moves beyond the movement limit of the movable mount 804, the first person 840a is outside the field of view 814, and the movable mount 804 cannot physically move the one or more camera sensors to bring the first person 840a into the field of view 814 of the one or more camera sensors.

[0248] In some cases, computer system 800 cannot reposition the subject within field of view 814. One example is when the subject moves beyond movement limits, such as relative to... Figure 8M As described. Another example is when the subject moves outside the field of view 814 (e.g., at a faster speed than the computer system 800 is capable of moving (or configured to move) to track the subject) (as relative to...). Figure 8K (As described) and the computer system 800 is unable to detect the trajectory along which the subject moves. In some implementations, if the subject is not located within a predetermined time period, the tracking of the subject will "time out" (e.g., the computer system 800 will stop tracking or attempt to track the subject).

[0249] In some implementations, when tracking of the subject times out (e.g., the subject has been outside the field of view for a predetermined period of time), the computer system 800 maintains the field of view 814 at the position it was in when the subject was lost. Figure 8M At this point, the first person 840a moves outside the field of view 814. After a predetermined time period has elapsed since the first person 840a has been outside the field of view 814, the tracking of the first person 840a times out and the field of view 814 remains unchanged (e.g., maintaining its current position and size).

[0250] In some implementations, when tracking of a subject times out (e.g., the subject has been outside the field of view for a predetermined period of time), the computer system 800 changes the corresponding subject tracking operation mode from tracking a first subject to tracking a second subject. Figure 8M At that point, the first person 840a moves and is outside the field of view 814 for a predetermined period of time. Therefore, in Figure 8N At that point, according to the corresponding subject tracking operation mode, computer system 800 tracks the second subject (e.g., the second person 840b).

[0251] In some implementations, the second subject is identified as the person closest to the frame center when the first subject has been outside the field of view 814 for a predetermined time period. In some implementations, the second subject is previously identified within the field of view 814. For example, in... Figure 8N In this process, computer system 800 tracks the second person 840b because the second person 840b was previously identified in field of view 814, such as... Figure 8I As illustrated. In some implementations, the computer system 800 adjusts the field of view 814 to the last known position of the second subject. For example, as... Figure 8N As shown, the computer system 800 adjusts the field of view 814 to the position where the second person 840b was previously marked.

[0252] In some cases, after the computer system 800 has been tracking the second person 840b according to the corresponding subject tracking operation mode, the second person 840b moves outside the field of view 814. For example, when the second person 840b moves beyond the movement limit, the second person 840b moves outside the field of view 814, as relative to... Figure 8M As discussed, as another example, when the second person 840b moves at a speed faster than the computer system 800 is capable of moving (or configured to move) to track the subject, the second person 840b moves outside the field of view 814, and the computer system 800 cannot detect the trajectory along which the second person 840b moves, as relative to... Figure 8K As described. In some implementations, when the tracking of the second person 840b times out (e.g., the second person has been out of the field of view for a predetermined period of time), the computer system 800 stops tracking the second person 840b and resumes tracking the first person 840a by scanning the last known location of the first person 840a and / or the possible trajectory of the first person 840a.

[0253] In some implementations, the computer system 800 includes two or more camera sensors. If the computer system 800 initially uses a first camera sensor 818 and the subject moves out of the field of view 814, the computer system 800 changes to begin using a second camera sensor 820 to track the subject. For example, in Figure 8O In this scenario, the first person 840a moves outside the field of view 814, and the movable mount 814 has reached its movement limit. Therefore, the computer system 800 is no longer able to use the first camera sensor 818 to track the first person 840a. Figure 8P As illustrated, the computer system 800 changes from using a first camera sensor 818 to using a second camera sensor 820. The first person 840a is within the field of view 822 of the second camera sensor 820, and therefore, the computer system 800 is able to use the second camera sensor 820 to track the user.

[0254] Figure 9 This is a flowchart illustrating a method for initiating a process using a computer system to capture video using one or more camera sensors, according to some embodiments. Method 900 is performed at a computer system (e.g., 100, 300, 500, 600, 800, 1000, smartphone, tablet, laptop, desktop computer, smartwatch, camera device, and / or electronic control system) communicating with one or more camera sensors (e.g., 818 and / or 820). Some operations in method 900 may be combined, the order of some operations may be changed, and some operations may be omitted.

[0255] As described below, Method 900 provides an intuitive way to initiate a process of capturing video using one or more camera sensors. This method reduces the cognitive burden on the user in initiating the process of capturing video using one or more camera sensors, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to initiate the process of capturing video using one or more camera sensors faster and more efficiently saves power and increases the time interval between battery charging.

[0256] In some embodiments, the computer system detects (902) a request to capture video using one or more camera sensors of the computer system. In some embodiments, in response to detecting the request to capture video (e.g., 850 and / or 852) (904): based on determining that the computer system is connected to a movable mount (e.g., 804) (e.g., physically connected to the movable mount, physically mounted on the movable mount, and / or communicating with the movable mount via a wired or wireless connection) (e.g., based on (or in some embodiments, in response to) determining that the computer system is connected to the movable mount), the computer system initiates (906) a process of capturing video using one or more camera sensors of the computer system according to a corresponding subject tracking operation mode (e.g., the visible portion of the field of view changes based on detected changes within the field of view (such as movement of one or more subjects in the field of view) (e.g., adjusted mechanically and / or electronically) (e.g., the field of view is adjusted to follow a subject moving from one location to another)). In some embodiments, the corresponding subject tracking operation mode causes the movable mount to move based on detected changes within the field of view. In some implementations, the computer system enables the appropriate subject tracking operation mode based on (or in some implementations, in response to) determining that the computer system is connected to the removable mount.

[0257] In some implementations, in response to a detected request to capture video (904): based on the determination that the computer system is not connected to a movable mount (e.g., a mount configured to physically support or bear the computer system via gravity, magnetic mechanisms, and / or mechanical mechanisms) (e.g., not physically connected to the movable mount, not physically mounted on the movable mount, and / or not communicating with the movable mount via a wired or wireless connection), the computer system initiates (908) a process of capturing video using the one or more camera sensors without enabling the corresponding subject tracking operation mode (e.g., not enabling the corresponding subject tracking operation mode and / or not operating according to the corresponding subject tracking operation mode) (e.g., not changing the visible portion of the field of view of the one or more camera sensors based on detected changes in the field of view (such as movement of one or more users in the field of view). When the computer system is connected to the removable mount, the process of automatically initiating video capture according to the corresponding subject tracking operation mode is reduced. This reduces the amount of input required to initiate the process of capturing video in the corresponding subject tracking mode and allows the user to capture video in the corresponding subject tracking mode without displaying additional controls. This allows the operation to be performed when a set of conditions are met without further user input, reducing the amount of input required to perform the operation and providing additional control options without cluttering the user interface with additional display controls.

[0258] In some embodiments, the mount has a motor and / or controls capable of rotating, panning (e.g., rotating about a vertical axis), tilting (e.g., rotating about a horizontal axis), and / or rolling (e.g., rotating about a second horizontal axis perpendicular to both the horizontal and vertical axes) one or more portions of the mount. In some embodiments, a corresponding subject tracking operation mode includes moving the movable mount (e.g., rotating it about one or more axes) to change the field of view of one or more camera sensors of the computer system to track one or more subjects (e.g., to keep the one or more users within the visible portion of the field of view). In some embodiments, when the corresponding subject tracking mode is enabled, the computer system (e.g., digitally via image processing) tracks one or more subjects in the field of view of the one or more camera sensors and / or (e.g., via wireless communication) transmits instructions for moving the mount to move the computer system and change the field of view of one or more camera sensors of the computer system to keep the one or more users within the visible portion of the field of view. In some embodiments, the movable mount is configured to move according to instructions from the computer system. In some embodiments, operating the video capture user interface without enabling the corresponding subject tracking operation mode includes operating the video capture user interface according to a second subject tracking mode different from the corresponding subject tracking mode (e.g., when the second subject tracking mode is enabled and the corresponding subject tracking mode is not enabled). In some embodiments, operating the video capture user interface without enabling the corresponding subject tracking operation mode includes digitally altering (e.g., panning and / or zooming) a display portion of the field of the one or more camera sensors without moving the mount.

[0259] In some embodiments, after capturing video using the one or more camera sensors, if the computer system is not connected to the removable mount and the corresponding subject tracking operation mode is not enabled, the computer system receives an instruction to connect to the removable mount. In some embodiments, in response to receiving the instruction to connect the computer system to the removable mount, the computer system enables the corresponding subject tracking operation mode (e.g., as referenced). Figure 8G(As described) (For example, when the computer system is connected to the removable mount after it has been previously operated without using the corresponding tracking mode, the tracking is automatically enabled). Automatically enabling the corresponding subject tracking operation mode when the computer system is connected to the removable mount after it has previously captured video without being connected to the removable mount and without the corresponding subject tracking mode being enabled reduces the amount of input required to switch from capturing video without the corresponding subject tracking mode being enabled to capturing video with the corresponding subject tracking mode enabled. This allows operations to be performed when a set of conditions are already met without further user input, and reduces the amount of input required to perform the operation.

[0260] In some implementations, capturing video using one or more camera sensors according to the corresponding subject tracking operation mode includes using a first algorithm (e.g., as referenced). Figures 8G to 8P The described algorithm (e.g., including logic for tracking a subject, such as by physical movement and / or zoom) is used to capture video. In some implementations, using the one or more camera sensors to capture video without enabling the corresponding subject tracking operation mode includes using a second algorithm different from the first algorithm (e.g., as described in the reference). Figures 8C to 8F The first algorithm is used to capture video (e.g., excluding algorithms for tracking subjects, including algorithms for tracking multiple subjects, and / or including algorithms for tracking one or more subjects by digitally moving and / or zooming) (and in some embodiments, without using the first algorithm). In some embodiments, the first and second algorithms include different logic for tracking one or more subjects at different speeds. In some embodiments, the first and second algorithms include different logic for centering one or more subjects within the field of view of the one or more camera sensors. In some embodiments, the first algorithm tracks a primary subject, while the second algorithm tracks one or more subjects identified within the field of view of the one or more camera sensors. In some embodiments, the first algorithm primarily tracks a first subject and tracks a second subject as an item of interest, while the second algorithm tracks both the first and second subjects in the same manner. Using the first algorithm when capturing video according to a corresponding subject tracking operation mode and using the second algorithm when capturing video without enabling the corresponding subject tracking operation mode enables the computer system to use improved settings for capturing video and tracking subjects (when available), thereby performing operations when a set of conditions is already met without further user input and reducing the amount of input required to perform the operations.

[0261] In some embodiments, a first subject (e.g., 840a) (e.g., a first person) and a second subject (e.g., a second person) (e.g., 840b) are within the field of view of the one or more camera sensors. In some embodiments, when capturing video using the one or more camera sensors according to a corresponding subject tracking operation mode, in response to detecting movement of the first subject (e.g., the first person 840a moving from its position...), Figure 8I The position in Figure 8J The computer system tracks the movement of the first subject (e.g., the first subject moves away from the second subject, such that the computer system cannot (or in some embodiments, does not) change the field of view of the one or more camera sensors to include both the first and second subjects). In some embodiments, the first and second subjects are within the field of view of the one or more camera sensors, and a corresponding subject tracking operation mode causes the field of view of the one or more camera sensors to change based on (e.g., in response to and / or according to) the movement of the first subject, but not based on the movement of the second subject. In some embodiments, when the corresponding subject tracking operation mode is disabled (e.g., the computer system does not operate according to the corresponding subject tracking operation mode), the field of view of the one or more camera sensors does not change based on the movement of the first subject. Tracking the movement of the first subject when the first person moves enables the computer system to track the movement of the first subject without additional input from the user, and allows the computer system to track a specific subject when there are two subjects in the field of view, thereby performing an operation when a set of conditions has been met without further user input, and reducing the amount of input required to perform the operation.

[0262] In some implementations, when capturing video using one or more camera sensors according to a corresponding subject tracking operation mode, the gaze of a first subject (e.g., a first person) is determined to be directed (e.g., moved to) an item of interest (e.g., a second person and / or an object, such as a whiteboard) (e.g., the first person 840a is gazing at the whiteboard). Figure 8H (As shown) and / or the first person 840a is looking at the second person 840b ( Figure 8I As shown), the computer system alters the field of view (e.g., 814) of one or more camera sensors to include the item of interest (e.g., Figure 8I(The whiteboard shown and / or the second person 840b) (and / or in some embodiments, such that the item of interest is located in a predetermined portion (e.g., the central portion) of the field of view). In some embodiments, when a corresponding subject tracking operation mode is enabled, the computer system changes the field of view of the one or more camera sensors based on the first person's gaze at the item of interest, such that the field of view includes the item of interest. In some embodiments, because the computer system changes the field of view to include the item of interest, the subject is further away from the center of the field of view compared to a situation where the subject's gaze is not directed at the item of interest (e.g., at the edge of the field of view of the one or more camera sensors) and / or partially outside the view of the one or more camera sensors. Automatically changing the field of view of the camera sensors to include the item of interest based on determining the first subject's gaze at the item of interest reduces the amount of input required to adjust the field of view, thereby performing the operation without further user input when a set of conditions has already been met, and reducing the amount of input required to perform the operation.

[0263] In some implementations, when capturing video using one or more camera sensors according to a corresponding subject tracking operation mode, in response to detecting movement of a first subject (e.g., a first person) (e.g., the first person 840a moving from its position in...), Figure 8I The position in Figure 8J (Regarding the movement of a subject within a camera sensor), the computer system alters the field of view of the one or more camera sensors to track the first subject without changing the zoom level of the field of view of the one or more camera sensors (e.g., without zooming in or out to increase or decrease the angular range of the field of view). In some embodiments, when a corresponding subject tracking operation mode is enabled, the computer system alters the field of view of the one or more camera sensors based on the movement of the first subject, without altering the zoom level of the field of view of the one or more camera sensors based on the movement of the second subject (e.g., shifting, panning, and / or tilting the field of view of the one or more camera sensors to track the first subject, rather than zooming in or out). Altering the field of view of the camera sensors to track the first subject in response to detecting that the first subject has moved without zooming out allows the computer system to maintain the image quality and / or size of the first subject within the field of view while continuing to track the first subject, thereby performing improved operations without requiring further user input when a set of conditions has been met, and reducing the amount of input required to perform the operation.

[0264] In some implementations, when capturing video using the one or more camera sensors according to a corresponding subject tracking operation mode, based on the determination that a first subject (e.g., a first person) has moved outside the field of view of the one or more camera sensors (e.g., ...), Figure 8KThe computer system, based on the trajectory of the first subject (e.g., path direction and / or velocity) (e.g., the first subject 840a moves outside the field of view 814), determines the location of the first subject from its position within the field of view 814. Figure 8J The position in Figure 8K The computer system alters the field of view of one or more camera sensors based on the direction of movement of the first subject (e.g., the last known movement path of the first subject and / or the estimated future movement path of the first subject). In some embodiments, when a corresponding subject tracking operation mode is enabled, when the first subject moves outside the field of view of the one or more camera sensors, the computer system alters the field of view of the one or more camera sensors based on the last known trajectory of the first subject (e.g., the computer system alters the field of view of the camera sensors to follow the movement path taken by the first subject). By determining that the first subject has moved outside the field of view and altering the field of view of the camera sensors based on the trajectory of the first subject, the computer system can attempt to reposition the subject back into the field of view, thereby performing improved operations without further user input when a set of conditions has already been met, and reducing the amount of input required to perform the operation.

[0265] In some implementations, when capturing video using the one or more camera sensors according to a corresponding subject tracking operation mode, the first subject is determined to be outside the field of view of the one or more camera sensors for a predetermined period of time (e.g., the first subject 840a is...). Figure 8M Beyond the field of view 814 in the system (for a predetermined time), the computer system maintains (e.g., stops changing) the field of view of the one or more camera sensors (e.g., Figure 8M (Field of view 814 in the image). In some embodiments, when a corresponding tracking operation mode is enabled, the computer system maintains (e.g., stops changing) the field of view of the one or more camera sensors based on determining that the first subject is outside the field of view of the one or more camera sensors for a predetermined period of time (e.g., if the first subject is not found within the predetermined period of time, the corresponding subject tracking operation mode times out). In some embodiments, when the corresponding subject tracking operation mode is enabled, the computer system changes the field of view to an initial position and maintains that initial position if the first subject is lost. In some embodiments, when the corresponding subject tracking operation mode is enabled, the computer system maintains the field of view at the time the first subject is lost. Maintaining the field of view of the camera sensors after the first subject has been outside the field of view for a predetermined period of time allows the computer system to conserve resources, rather than continuing to attempt to locate the first subject when the computer system is unlikely to be able to locate the first subject, thereby performing improved operations without further user input when a set of conditions has been met, and reducing the amount of input required to perform the operation.

[0266] In some embodiments, the computer system includes two or more camera sensors (e.g., 818 and 820) (e.g., camera sensors on the front and rear of the computer system). In some embodiments, the computer system uses a first camera sensor (e.g., a camera sensor on the front of the computer system) to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., a first person) based on the video captured by the first camera sensor (e.g., 818 or 820). In some implementations, based on the determination that the first subject is outside the field of view of the first camera sensor among the two or more camera sensors (e.g., outside the field of view of the camera sensor on the front of the computer system) and the first subject is within the field of view of the second camera sensor (e.g., 820 or 818) among the two or more camera sensors (e.g., within the field of view of the camera sensor on the rear of the computer system), the computer system uses the second camera sensor (e.g., using the camera sensor on the rear of the computer system) to capture video according to a corresponding subject tracking operation mode (e.g., to continue tracking the first person based on the video captured by the second camera sensor). Using the second camera to capture video after the first subject has been outside the field of view of the first camera for a predetermined period of time allows the computer system to switch camera sensors, rather than continuing to attempt to locate the first subject using the first camera when it is unlikely that the computer system will be able to do so. This allows for improved operation when a set of conditions has already been met without further user input and reduces the amount of input required to perform the operation.

[0267] In some implementations, the computer system uses one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., the first person) detected in the field of view of the one or more camera sensors when the corresponding subject tracking operation mode is enabled. Figure 8G (Detected in the field of view 814 when enabled). Tracking the movement of the first subject based on the fact that the first subject is the first person detected in the field of view enables the computer system to automatically identify the subject to be tracked, thereby performing operations when a set of conditions are met without further user input and reducing the amount of input required to perform the operation.

[0268] In some implementations, the computer system uses one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., a first person), wherein the first subject is tracked (at least in part) based on the detection of the first subject's face (e.g., the detection of one or more facial features such as eyes, nose, mouth, and / or facial shape) when the corresponding subject tracking operation mode is enabled (or, in some implementations, selected for tracking) (e.g., tracking the movement of the first subject is performed based on determining that the first subject's face has been detected). In some implementations, if the first subject's face is not detected, the first subject is not tracked. Tracking the movement of the first subject based on face detection enables the computer system to automatically identify the relevant subject to be tracked, thereby performing operations when a set of conditions has been met without further user input and reducing the amount of input required to perform the operations.

[0269] In some implementations, the computer system uses one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., a first person), wherein the first subject is tracked (at least in part) based on the detection of the first subject's body (e.g., the detection of the torso and / or one or more limbs) when the corresponding subject tracking operation mode is enabled (or, in some implementations, selected for tracking) (e.g., tracking the movement of the first subject is performed based on determining that the first subject's body has been detected). In some implementations, if the first subject's body is not detected, the first subject is not tracked. Tracking the movement of the first subject based on the detection of the body enables the computer system to automatically identify the relevant subject to be tracked, thereby performing operations when a set of conditions has been met without further user input and reducing the amount of input required to perform the operations.

[0270] In some implementations, the computer system uses one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., a first person). In some implementations, the first subject is tracked (at least in part) based on the detection of one or more facial features (e.g., eyes, nose, mouth, and / or facial shape) of a known subject (e.g., the main user, contact, and / or person previously identified by the computer system) when the corresponding subject tracking operation mode is enabled (e.g., selected for tracking). (For example, tracking the movement of the first subject is performed based on determining that one or more facial features of a known subject have been detected). In some implementations, the first subject is not tracked if one or more facial features of a known subject are not detected. Tracking the movement of the first subject based on the detection of one or more facial features of a known subject enables the computer system to automatically identify the relevant subject to be tracked, thereby performing operations when a set of conditions has been met without further user input and reducing the amount of input required to perform the operation.

[0271] In some implementations, the computer system uses one or more camera sensors to capture video, including tracking the movement of a first subject (e.g., a first person), according to a corresponding subject tracking operation mode. In some implementations, the first subject is (at least in part) tracked based on the fact that the first subject is the person closest to the center of the field of view of the one or more camera sensors when the corresponding subject tracking operation mode is enabled (or, in some implementations, selected for tracking). Tracking the movement of the first subject based on the fact that the first subject is the person closest to the center of the field of view allows the computer system to automatically identify the relevant subject to be tracked, thereby performing operations when a set of conditions are already met without further user input and reducing the amount of input required to perform the operations.

[0272] In some implementations, the computer system uses the one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the movement of a first subject (e.g., a first person). In some implementations, the first subject is (at least in part) tracked based on the fact that the first subject is the person closest to the center of the field of view of the one or more camera sensors and is looking at the one or more camera sensors (e.g., facing the one or more camera sensors and / or making eye contact with the one or more camera sensors) when the corresponding subject tracking operation mode is enabled (or, in some implementations, selected for tracking). Tracking the movement of the first subject based on the fact that the first subject is the person closest to the center of the field of view and is looking at the one or more camera sensors allows the computer system to automatically identify the relevant subject to be tracked, thereby performing operations when a set of conditions are already met without further user input and reducing the amount of input required to perform the operations.

[0273] In some implementations, a first subject (e.g., a first person) and a second subject (e.g., a second person) are within the field of view of the one or more camera sensors. In some implementations, the computer system uses the one or more camera sensors to capture video according to a corresponding subject tracking operation mode, including tracking the first subject (e.g., tracking the movement of the first subject). In some implementations, after tracking the first subject, based on determining that the first subject is outside the field of view of the one or more camera sensors for a predetermined time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 3 seconds, 5 seconds, 10 seconds, 15 seconds, 60 seconds, or 120 seconds), the computer system changes the field of view of the one or more camera sensors based on the position of the second subject (e.g., changes based on the position of the second person 840b). Figure 8N (e.g., adjusting the field of view 814 in the image) to center on the second subject and / or switching from tracking the first subject to tracking the second subject based on the fact that the first subject has been outside the field of view of the one or more camera sensors for a predetermined period of time. Changing the field of view of the camera sensors based on the position of the second subject allows the computer system to automatically switch to tracking the second subject after the first subject has been outside the field of view for a predetermined period of time. This enables improved operations to be performed when a set of conditions have been met without further user input and reduces the amount of input required to perform the operation.

[0274] In some implementations, the field of view of the one or more camera sensors is (at least in part) determined based on the location associated with the second subject, which is the person closest to the center of the field of view of the one or more camera sensors when the first subject is determined to be outside the field of view of the one or more camera sensors for a predetermined time period (e.g., if the first subject 840a leaves). Figure 8I In the field of view 814 shown, the second subject is identified as the second person 840b because the second person 840b is the subject closest to the center of the field of view 814 when the first person 840a is identified as being outside the field of view 814 for a predetermined time period. Therefore, the computer system 800 will track the second person 840b, as shown. Figure 8N (As shown). Tracking the movement of the second subject, which is the person closest to the center of the field of view, allows the computer system to automatically identify the second subject to be tracked when the first subject has been outside the field of view for a predetermined period of time. This enables the system to perform operations when a set of conditions are met without further user input and reduces the amount of input required to perform the operations.

[0275] In some implementations, the field of view of the one or more camera sensors is determined (e.g., at least in part) based on the fact that the second subject is a person previously identified within the field of view of the one or more camera sensors (e.g., Figure 8N The second person in the group, 840b, previously... Figure 8I The identification of the second subject within the field of view 814 (e.g., the first person looking at the second person) changes based on the position associated with the second subject. In some embodiments, the second subject was previously identified because the second subject was looking at the one or more camera sensors (e.g., facing the one or more camera sensors and / or making eye contact with the one or more camera sensors). Tracking the movement of the second subject based on the fact that the second subject was previously identified within the field of view enables the computer system to automatically identify the second subject to be tracked when the first subject has been outside the field of view for a predetermined period of time, thereby performing operations when a set of conditions has been met without further user input and reducing the amount of input required to perform the operations.

[0276] In some implementations, the location associated with the second subject is the last known location of the second subject (e.g., Figure 8I The position of the second person 840b is previously known, therefore this position is related to... Figure 8N(The location associated with the second person 840b in the image). Changing the field of view of the camera sensor to the last known position of the second subject allows the computer system to automatically attempt to position the second subject within the field of view after the first subject has been outside the field of view for a predetermined period of time. This enables the operation to be performed when a set of conditions has been met without further user input and reduces the amount of input required to perform the operation.

[0277] In some implementations, after changing the field of view of the one or more camera sensors based on the position of the second subject: depending on whether the second subject is outside the field of view of the one or more camera sensors for a predetermined period of time (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 3 seconds, 5 seconds, 10 seconds, 60 seconds, or 120 seconds), the computer system changes the field of view of the one or more camera sensors based on the position of the first subject (e.g., adjusting the field of view to be centered on the first subject and / or switching from tracking the second subject to tracking the first subject based on the predetermined period of time the second subject has been outside the field of view of the one or more camera sensors). Changing the field of view of the camera sensors based on the position of the first subject allows the computer system to automatically switch to tracking the first subject after the second subject has been outside the field of view for a predetermined period of time, thereby performing improved operations without further user input when a set of conditions has been met, and reducing the amount of input required to perform the operations.

[0278] In some implementations, the computer system uses one or more camera sensors to capture video, including tracking a first subject, based on a corresponding subject tracking operation mode. In some implementations, based on determining that a movable mount has reached a movement limit (e.g., Figure 8MWhen the movable mount 804 is rotated to a position (e.g., a minimum or maximum position limit; and / or the movable mount cannot be physically moved further), the computer system stops tracking the first subject (e.g., stops changing the field of view of the one or more camera sensors to track the first subject, optionally by maintaining the field of view of the one or more camera sensors, by changing the field of view of the one or more camera sensors to track a different subject, or by changing the field of view of the one or more camera sensors to a different position selected based on other criteria (such as a default or neutral position)). In some embodiments, when a corresponding subject tracking operation mode is enabled, if the movable mount meets a movement limit threshold, the computer system maintains (e.g., stops changing) the field of view of the one or more camera sensors. In some embodiments, when the movable mount reaches a movement limit and can no longer mechanically change the field of view of the one or more camera sensors, the computer system continues to track the first subject by digitally changing the field of view of the one or more camera sensors. When the movable mount reaches its movement limit, the tracking of the first subject stops, enabling the computer system to automatically stop tracking when it is no longer possible. This allows the system to perform operations when a set of conditions have been met without requiring further user input and reduces the amount of input required to perform the operation.

[0279] In some embodiments, when capturing video using one or more camera sensors of a computer system according to a corresponding subject tracking operation mode, the computer system changes the displayed field of view of the one or more camera sensors based on determining that one or more movement characteristics of the corresponding subject (e.g., distance and / or velocity) meet a first criterion (e.g., a predetermined distance threshold and / or velocity threshold). In some embodiments, when capturing video using one or more camera sensors of a computer system according to a corresponding subject tracking operation mode, the computer system maintains (e.g., does not change) the displayed field of view of the one or more camera sensors based on determining that one or more movement characteristics of the corresponding subject do not meet the first criterion. In some embodiments, the change in the displayed field of view is proportional to the movement of the corresponding subject (e.g., if the corresponding subject moves rapidly, the displayed field of view is changed rapidly). In some embodiments, the displayed field of view is maintained (e.g., not changed) or changed slowly based on minor movements of the corresponding subject (e.g., gesturing and / or moving limbs). In some embodiments, the displayed field of view of the one or more camera sensors is rapidly changed based on large and / or significant movements of the subject (e.g., moving from one location to another and / or moving throughout space). In some embodiments, the displayed field of view is maintained (e.g., not changed) or slightly changed based on minor movements of the subject (e.g., gesturing and / or moving limbs). In some embodiments, the displayed field of view of the one or more camera sensors is significantly changed based on large and / or significant movements of the subject (e.g., moving from one location to another and / or moving throughout space). Changing the displayed field of view based on one or more movement characteristics of the subject enables the computer system to efficiently track the subject without additional input from the user, thereby performing improved operations without further user input when a set of conditions has been met, and reducing the amount of input required to perform the operation.

[0280] In some embodiments, when video is captured using one or more camera sensors of a computer system according to a corresponding subject tracking operation mode, and after changing the displayed field of view of the one or more camera sensors based on determining that one or more motion characteristics (e.g., distance and / or velocity) of the corresponding subject meet a first criterion (e.g., a predetermined distance threshold and / or velocity threshold): the computer system changes the displayed field of view of the one or more camera sensors based on determining that one or more motion characteristics (e.g., distance and / or velocity) of the corresponding subject meet a second predetermined criterion (e.g., a second predetermined distance threshold and / or velocity threshold). In some embodiments, when video is captured using one or more camera sensors of a computer system according to a corresponding subject tracking operation mode, and after changing the displayed field of view of the one or more camera sensors based on determining that one or more motion characteristics (e.g., distance and / or velocity) of the corresponding subject meet a first criterion (e.g., a predetermined distance threshold and / or velocity threshold): the computer system maintains (e.g., does not change) the displayed field of view of the one or more camera sensors based on determining that one or more motion characteristics of the corresponding subject do not meet the second predetermined criterion. The computer system can change the displayed field of view by determining that the movement of the subject is above a predetermined threshold and maintain the displayed field of view by determining that the movement of the subject is below a predetermined threshold. This allows the computer system to track the subject more efficiently without additional input from the user. As a result, it can perform improved operations when a set of conditions are met without further user input and reduces the amount of input required to perform the operation.

[0281] In some implementations, upon determining that a corresponding subject tracking operation mode is enabled, the computer system causes the output of an indication that the corresponding subject tracking operation mode is enabled (e.g., activating an LED indicator located on the computer system and / or on a movable mount and / or transmitting data indicating that the corresponding subject tracking operation mode is enabled to the movable mount). Determining that the corresponding subject tracking operation mode is enabled provides the user with an indication that the corresponding subject tracking operation mode is enabled, thereby providing the user with improved visual feedback.

[0282] In some implementations, causing the output of an indication that the corresponding subject tracking operation mode is enabled includes: based on determining that the corresponding subject tracking operation mode is active (e.g., the corresponding subject tracking operation mode is enabled and is being actively used to track the corresponding subject), causing an indication having a first characteristic (e.g., a flashing and / or illuminating LED indicator) to be output. Outputting an indication with the first characteristic based on determining that the corresponding subject tracking operation mode is active provides the user with an indication that the corresponding subject tracking operation mode is active, thereby providing the user with improved visual feedback.

[0283] In some implementations, causing the output of an indication that the corresponding subject tracking operation mode is enabled includes: based on determining that the corresponding subject tracking operation mode is not in an active state (e.g., the corresponding subject tracking operation mode is enabled but not actively used to track the corresponding subject), outputting an indication having a second characteristic different from the first characteristic (e.g., a constantly lit LED indicator). Outputting an indication with the second characteristic based on determining that the corresponding subject tracking operation mode is not in an active state provides the user with an indication that the corresponding subject tracking operation mode is not in an active state, thereby providing the user with improved visual feedback.

[0284] Figures 10A to 10H Exemplary techniques for performing animations using a removable mount, according to some embodiments, are illustrated. The animations and user interfaces in these figures are used to illustrate the processes described below, including... Figure 11 The process in.

[0285] Figure 10A The example illustrates device 1000 and movable mounting 1002. In Figure 10A In this embodiment, device 1000 is a smartphone with a display 1004. In some embodiments, device 1000 is a tablet computer, laptop computer, desktop computer, smartwatch, camera device, and / or electronic control system. In some embodiments, computer system 1000 is computer system 600 and / or 800; and movable mount 1002 is movable mount 608 or movable mount 804. Figure 10A As shown, device 1000 was initially placed on the table.

[0286] exist Figure 10BAt this point, the user initiates a process associated with capturing video, such as placing device 1000 on removable mount 1002. In some implementations, the user places device 1000 on removable mount 1002 after launching the camera application, such that device 1000 is operating the camera application while placed on removable mount 1002. When the user initially connects device 1000 to removable mount 1002, device 1000 is not in a particular orientation.

[0287] exist Figure 10C At this point, device 1000 is connected to movable mount 1002, and device 1000 and / or movable mount 1002 receive an indication that a process associated with capturing video has been initiated, such as an indication that a camera application is operating. In response to receiving the indication, movable mount 1002 executes an animation. Figure 10C In the example shown, the movable mount 1002 moves the device 1000 such that the device 1000 is oriented toward the user. For example, depending on the original orientation of the device 1000 placed on the movable mount 1002, (e.g., by the device 1000 or other computer systems communicating with the device 1000 and / or the movable mount 1002) the movable mount 1002 is raised, lowered, and / or rotated such that the device 1000 is oriented toward the user. In some embodiments, the device 1000 is oriented toward the user in a lateral orientation, such as... Figure 10C As shown. In some embodiments, the device 1000 is oriented longitudinally toward the user.

[0288] The movable mount 1002 can execute different animations for different events. Figure 10C In some implementations, events include indications of the start of the video capture process, such as indications that device 1000 has been connected to removable mount 1002 and / or that the camera application is in operation, and animations include moving device 1000 toward the user. In some implementations, events include indications of the end of the video capture process, such as indications that device 1000 has been disconnected from removable mount 1002, device 1000 has entered a low-power mode, and / or the camera application is no longer in operation. In this case, the resulting animation includes moving device 1000 such that device 1000 is oriented downwards.

[0289] exist Figure 10D At this point, device 1000 enters a low-power state. Device 1000 was previously oriented towards the user, therefore device 1000 remains oriented towards the user when entering the low-power state. For example, as... Figure 10D As illustrated, the display 1004 of device 1000 dims to save power.

[0290] like Figure 10EAs illustrated, in response to an indication that device 1000 has entered a low-power state, movable mount 1002 moves device 1000 so that device 1000 faces downwards. Figure 10E As shown, the movable mount 1002 moves the device 1000 so that the device 1000 faces the table.

[0291] exist Figure 10F At this point, the user performs an action (such as speaking), which causes device 1000 to switch from a low-power mode to a normal operating mode. For example, the user says, "Hey! Turn on the camera." Therefore, device 1000 changes from a low-power mode to a normal operating mode, and device 1000 displays the camera application user interface 1006. In some embodiments, device 1000 changes from a low-power mode to a normal operating mode and displays the camera application user interface 1006 in response to one or more inputs from the user.

[0292] like Figure 10G As illustrated, in response to an indication that device 1000 is transitioning from a low-power mode to a normal mode, movable mount 1002 moves device 1000 such that device 1000 is oriented towards the user. In some embodiments, device 1000 is oriented towards the user based on the user's last known location. For example, Figure 10G This illustrates how device 1000 is oriented to a location where the user was previously positioned, such as in... Figure 10C middle.

[0293] In some implementations, the movable mount 1002 orients the device 1000 toward the user based on the location of audio (such as the location of a user's voice when instructing the device 1000 to switch from a low-power mode to a normal mode). Figure 10H An example is shown where a movable mount 1002 orients the device 1000 in a different orientation than before, based on the user's voice projected from different locations.

[0294] Figure 11 This is a flowchart illustrating a method for using a computer system to move a device connected to a movable mount according to some embodiments. Method 1100 is performed at a computer system (e.g., 100, 300, 500, 600, 800, 1000, smartphone, tablet, laptop, desktop computer, smartwatch, camera device, and / or electronic control system) that communicates with the movable mount (e.g., 1002) (e.g., a mount configured to physically support or bear the computer system and / or device via gravity, magnetic mechanisms, and / or mechanical mechanisms). Some operations in method 1100 may be combined, some operations may be changed in order, and some operations may be omitted.

[0295] As described below, method 1100 provides an intuitive way to move a device connected to a removable mount. This method reduces the cognitive burden on users when moving a device connected to a removable mount, thus creating a more efficient human-machine interface. For battery-powered computing devices, enabling users to move devices connected to a removable mount faster and more efficiently saves power and increases the time interval between battery charging sessions.

[0296] In some implementations, a computer system (e.g., via sensors of an electronic device and / or via signals from the electronic device) obtains (1102) indications of events associated with the captured video (e.g., device 1000 is connected to...). Figure 10B The instructions for the removable mount 1002 indicate that video is captured by a device (e.g., 1000) (e.g., a smartphone, one or more camera sensors, a tablet, and / or a smartwatch) connected to the removable mount (e.g., physically connected to the removable mount, physically mounted on the removable mount, and / or communicating with the removable mount via a wired or wireless connection), and wherein the device connected to the removable mount is moved via the removable mount to track one or more objects (e.g., one or more users and / or one or more objects of interest) while the video is captured. Figures 8A to 8P (The movement of the first person 840a in the image). In some embodiments, device 1000 is also a computer system. In some embodiments, the computer system receives indications for events not associated with capturing video (e.g., events associated with playing video without capturing video). In some embodiments, the computer system is a device connected to (or included in) a removable mount. In some embodiments, the device is moved such that the visible field of view of the video changes to follow a user moving from one location to another. In some embodiments, the device has a rechargeable battery, and the removable mount charges the battery (e.g., via wireless charging) when the device is connected to the removable mount.

[0297] In some implementations, in response to receiving an indication of the event, the computer system causes (1104) the removable mount to perform a movement of the device connected to the removable mount (e.g., such as...). Figures 10B to 10CA series of one or more mechanical movements of a movable device (as shown) are associated with an event. In some embodiments, the device is a computer system. In some embodiments, performing the series of one or more mechanical movements of the movable device includes changing the orientation of the device (e.g., raising, lowering, or orienting towards the user). In some embodiments, in response to receiving indication of the event, the computer system causes the device to perform animation by, for example, emitting sound and / or vibration (e.g., vibrating to a musical rhythm). Responding to receiving indication of the event by causing a movable mount to perform a series of mechanical movements provides the user with indication that the event has occurred, associated with the series of mechanical movements, thereby providing the user with improved visual feedback.

[0298] In some implementations, causing the movable mount to perform the series of one or more mechanical movements includes: based on determining that the event corresponds to a first event, causing the movable mount to perform a first series of one or more mechanical movements (e.g., as...). Figure 10C (The elevation shown). In some embodiments, causing the movable mount to perform this series of one or more mechanical movements includes: based on determining that the event corresponds to a second event, causing the movable mount to perform a second series of one or more mechanical movements (e.g., as shown). Figure 10D (As shown in the reduction), wherein the second series of one or more mechanical movements differs from the first series of one or more mechanical movements (e.g., the first series of one or more mechanical movements and the second series of one or more mechanical movements include movements in different directions, at different speeds, with different amplitudes, and / or with different patterns). In some embodiments, the series of one or more mechanical movements is based on the event (e.g., it is different for different events). Making the movable mount perform the first series of mechanical movements in response to receiving an indication of a first event and the second series of mechanical movements in response to receiving an indication of a second event provides the user with an indication of which event has occurred, thereby providing the user with improved visual feedback.

[0299] In some embodiments, the device includes a camera sensor. In some embodiments, when the series of one or more mechanical movements is completed (e.g., at the end of the series of one or more mechanical movements), the camera sensor is oriented downwards (e.g., the camera sensor on the front of device 1000 is...). Figure 10EOriented downwards (e.g., downwards relative to gravity, downwards relative to the orientation or configuration of a movable mount, away from the user, pointing towards a table, and / or towards the floor). Orienting the camera sensor downwards when one or more mechanical movements are completed provides the user with an indication that the series of mechanical movements has ended, thus providing improved visual feedback. In cases where the series of one or more mechanical movements corresponds to an event associated with video recording, orienting the camera sensor downwards when the series of one or more mechanical movements is completed provides enhanced privacy to the user by clearly indicating that video recording has stopped.

[0300] In some implementations, the indication of the event includes an indication of the commencement of the process of using the device to capture video (e.g., as referenced). Figure 10B (As described) (e.g., this series of one or more mechanical movements is performed at the start of the process). In some embodiments, the indication of this event includes an indication that the connected device is connected to the movable mount and / or that the connected device transitions from a low-power mode to a normal mode. Responding to an indication that the process of using the device to capture video has begun, causing the movable mount to perform a series of mechanical movements provides the user with an indication that the video capture process has started, thereby providing the user with improved visual feedback.

[0301] In some embodiments, the connected device includes a display generation component. In some embodiments, the series of one or more mechanical movements includes: moving the connected device such that the device's camera sensor (and / or the device's display) is oriented upwards (e.g., the camera sensor on the front of device 1000 is...). Figure 10D (e.g., the display is oriented towards the user, such as in portrait orientation or landscape orientation relative to the user). In response to receiving an indication that the video capture process has begun, the movable mount moves to oriented the display upwards, providing the user with an indication that the video capture process has started, thus providing improved visual feedback to the user.

[0302] In some implementations, the indication of the event includes an indication of the end of the process of using the device to capture video (e.g., the series of one or more mechanical movements is performed at the end of the process). In some implementations, the indication of the event includes an indication of disconnection from the removable mount and / or an indication that the connected device has switched from a normal mode to a low-power mode. Responding to an indication of the end of the process of using the device to capture video by performing a series of mechanical movements provides the user with an indication that the video capture process has ended, thereby providing improved visual feedback. Responding to an indication of the end of the process of using the device to capture video by performing a series of mechanical movements also provides the user with enhanced privacy by clearly indicating that the video capture process has ended.

[0303] In some embodiments, the device includes a display generating component. In some embodiments, the series of one or more mechanical movements includes: moving the device to orient the device's camera sensor (and / or the device's display) downwards (e.g., the camera sensor on the front of device 1000 is...). Figure 10E (e.g., downward orientation relative to gravity, downward orientation relative to the orientation or configuration of the movable mount, away from the user, pointing towards a table, and / or pointing towards the floor). Relocating the movable mount to face downward in response to an indication that the video capture process has ended provides the user with improved visual feedback. Relocating the movable mount to face downward in response to an indication that the video capture process has ended also provides improved privacy benefits to the user by clearly indicating that the video capture process has ended.

[0304] In some implementations, the indication of this event includes the device transitioning from a low-power mode (e.g., sleep mode, a mode where the device's display is off or dimmed, and / or a mode where a predefined set of operations is stopped, paused, or updated at a lower rate than during normal operation) to a normal mode (e.g., as referenced). Figures 10E to 10G The indication (as described) refers to a state of normal operation, a non-low-power mode, a mode where the device's display is on and / or not dimmed, and / or a mode that performs a predetermined set of operations according to a standard operating mode. In some embodiments, this event includes a user touching or using the device. Responding to an indication that the device is transitioning from a low-power mode to a normal mode, a series of mechanical movements of the movable mount provide the user with an indication of the device's mode change, thereby providing the user with improved visual feedback.

[0305] In some embodiments, the device includes a display generation component. In some embodiments, the series of one or more mechanical movements includes: moving the device to orient the device's camera sensor (and / or the device's display) toward the user orientation (e.g., a camera sensor on the front of device 1000 toward the user orientation) (e.g., in a longitudinal orientation or a lateral orientation relative to the user). Orienting the movable mount toward the user orientation in response to receiving an indication that the device is transitioning from a low-power mode to a normal mode provides the user with an indication of the device changing mode, thereby providing the user with improved visual feedback.

[0306] In some implementations, the device's camera sensor (and / or the device's display) is based on audio location (e.g., based on...). Figure 10F and Figure 10H The device is oriented towards the user based on the location of the sound "Hey! Open the camera." (e.g., based on the location of the main speech). In some implementations, the event includes the detection of the audio (e.g., a series of one or more mechanical movements are performed in response to the detection of the audio). In some implementations, the movable mount orients the device toward a first location based on determining that the source of the audio is in a first location; and the movable mount orients the device toward a second location based on determining that the source of the audio is in a second location different from the first location. Orienting the movable mount toward the user based on the location of the audio allows the computer system to locate the user, thereby performing an operation when a set of conditions has been met without requiring further user input, and reducing the amount of input required to perform the operation.

[0307] In some implementations, the device's camera sensor (and / or the device's display) is oriented towards the user based on the user's last known location (e.g., Figure 10G The orientation of the camera sensor in device 1000 is based on Figure 10C (Regarding the user's location). In some implementations, based on determining that the user's last known location is a first location, the movable mount orients the device toward the first location; and based on determining that the last known location is a second location different from the first location, the movable mount orients the device toward the second location. Orienting the movable mount toward the user based on the user's last known location enables the computer system to locate the user, thereby performing an operation when a set of conditions has been met without requiring further user input, and reducing the amount of input required to perform the operation.

[0308] In some embodiments, the indication of the event includes an indication that the device is capturing video (e.g., recording video and / or sharing video during a video call). In some embodiments, the device includes a display generating component. In some embodiments, the series of one or more mechanical movements includes: moving the device to orient the display generating component toward the user (e.g., according to...). Figures 8G to 8P The illustrated corresponding subject tracking operation mode for the mobile device 1000 (e.g., in portrait orientation or landscape orientation relative to the user). In response to receiving an indication that the device is capturing video, orienting the movable mount towards the user provides the user with this indication and allows the user to see what is being recorded and / or information about the video call (e.g., video of one or more other participants in the video call), thereby providing the user with improved visual feedback.

[0309] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. These embodiments were chosen and described in order to best explain the principles of these techniques and their practical application. Others skilled in the art will thus be able to best utilize these techniques and the various embodiments with various modifications suitable for the particular intended use.

[0310] While this disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of this disclosure and examples as defined by the claims.

[0311] As described above, one aspect of this technology involves collecting and using data from various sources. This disclosure envisions that, in some instances, this collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. Such personal information may include demographic data, location-based data, telephone numbers, email addresses, social network IDs, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0312] This disclosure recognizes that the use of such personal information data in the techniques of this invention can be used to benefit users. For example, health and fitness data can be used to provide insights into a user's overall health status or can be used as positive feedback to individuals using the technology to pursue health goals.

[0313] This disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with robust privacy policies and / or privacy measures. Specifically, such entities should implement and adhere to privacy policies and measures recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable entity purposes and should not be shared or sold outside of these legitimate purposes. Furthermore, such collection / sharing should be conducted only after receiving informed consent from the user. Additionally, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that other entities authorized to access such personal information data comply with their privacy policies and procedures. Moreover, such entities may subject themselves to third-party assessments to demonstrate compliance with widely accepted privacy policies and privacy practices. Furthermore, policies and practices should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including considerations of specific jurisdictions. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0314] Regardless of the foregoing, this disclosure also anticipates implementation schemes for users to selectively block the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, the technology of this invention can be configured to allow users to opt-in or opt-out to participate in the collection of personal information data during or at any time after registering for the service. In addition to providing "opt-in" and "opt-out" options, this disclosure also anticipates providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0315] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by restricting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0316] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not become inoperable due to the absence of all or part of such personal information data. For example, non-personal information data or an absolute minimum amount of personal information (such as content requested by a device associated with a user, other available non-personal information, or publicly available information) can be used.

Claims

1. A method, the method comprising: At a computer system that includes one or more camera sensors: Detect a request to capture video using one or more camera sensors of the computer system; as well as In response to the detection of the request to capture the video: Based on the determination that the computer system is connected to the removable mount, a process is initiated to capture video using the one or more camera sensors of the computer system according to the corresponding subject tracking operation mode; as well as Based on the determination that the computer system is not connected to the removable mount, a process is initiated to capture video using the one or more camera sensors without enabling the corresponding subject tracking operation mode.

2. The method according to claim 1, further comprising: After capturing video using the one or more camera sensors, if the computer system is not connected to the movable mount and the corresponding subject tracking operation mode is not enabled, an instruction is received that the computer system is connected to the movable mount; as well as In response to receiving the instruction that the computer system is connected to the removable mount, the corresponding subject tracking operation mode is enabled.

3. The method according to any one of claims 1 to 2, wherein: Capturing video using one or more camera sensors according to the corresponding subject tracking operation mode includes capturing video using a first algorithm; and Capturing video using the one or more camera sensors without enabling the corresponding subject tracking operation mode includes capturing video using a second algorithm different from the first algorithm.

4. The method of claim 3, wherein the first body and the second body are located within the field of view of the one or more camera sensors, the method further comprising: When capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode: In response to detecting movement of the first subject, the movement of the first subject is tracked.

5. The method according to any one of claims 3 to 4, further comprising: When capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode: Based on determining the gaze direction of the first subject towards the item of interest, the field of view of the one or more camera sensors is changed to include the item of interest.

6. The method according to any one of claims 3 to 5, further comprising: When capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode: In response to detecting movement of a first subject, the field of view of the one or more camera sensors is changed to track the first subject without changing the zoom level of the field of view of the one or more camera sensors.

7. The method according to any one of claims 1 to 6, further comprising: When capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode: Based on the determination that the first subject has moved outside the field of view of the one or more camera sensors, the field of view of the one or more camera sensors is changed based on the trajectory of the first subject.

8. The method according to claim 7, further comprising: When capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode: The field of view of the one or more camera sensors is maintained based on a predetermined time period after determining that the first subject has been outside the field of view of the one or more camera sensors.

9. The method according to any one of claims 1 to 8, wherein the computer system comprises two or more camera sensors, and the method further comprises: According to the corresponding subject tracking operation mode, the first camera sensor of the two or more camera sensors is used to capture video, including tracking the movement of the first subject based on the video captured by the first camera sensor; as well as Based on the determination that the first subject is outside the field of view of the first camera sensor among the two or more camera sensors and the first subject is within the field of view of the second camera sensor among the two or more camera sensors, video is captured using the second camera sensor among the two or more camera sensors according to the corresponding subject tracking operation mode.

10. The method according to any one of claims 1 to 9, further comprising: Capturing video using the one or more camera sensors according to the corresponding subject tracking operation mode, including tracking the movement of the first subject based on determining that the first subject is a first person detected in the field of view of the one or more camera sensors when the corresponding subject tracking operation mode is enabled.

11. The method according to any one of claims 1 to 10, further comprising: According to the corresponding subject tracking operation mode, video is captured using the one or more camera sensors, including tracking the movement of a first subject, wherein the first subject is tracked based on the detection of the face of the first subject when the corresponding subject tracking operation mode is enabled.

12. The method according to any one of claims 1 to 11, further comprising: The video is captured using one or more camera sensors according to the corresponding subject tracking operation mode, including tracking the movement of a first subject, wherein the first subject is tracked based on the detection of the first subject's body when the corresponding subject tracking operation mode is enabled.

13. The method according to any one of claims 1 to 12, further comprising: According to the corresponding subject tracking operation mode, video is captured using the one or more camera sensors, including tracking the movement of a first subject, wherein the first subject is tracked based on the detection of one or more facial features of a known subject when the corresponding subject tracking operation mode is enabled.

14. The method according to any one of claims 1 to 13, further comprising: Video is captured using the one or more camera sensors according to the corresponding subject tracking operation mode, including tracking the movement of a first subject, wherein the first subject is tracked based on the fact that the first subject is the person closest to the center of the field of view of the one or more camera sensors when the corresponding subject tracking operation mode is enabled.

15. The method according to any one of claims 1 to 14, the method further comprising: Video is captured using the one or more camera sensors according to the corresponding subject tracking operation mode, including tracking the movement of a first subject, wherein the first subject is tracked based on the fact that the first subject is the person closest to the center of the field of view of the one or more camera sensors and is looking at the one or more camera sensors when the corresponding subject tracking operation mode is enabled.

16. The method according to any one of claims 1 to 15, wherein the first body and the second body are located within the field of view of the one or more camera sensors, the method further comprising: According to the corresponding subject tracking operation mode, video is captured using the one or more camera sensors, including tracking the first subject; as well as After tracking the first subject, the field of view of the one or more camera sensors is changed based on the position of the second subject, according to the predetermined time period during which the first subject is determined to be outside the field of view of the one or more camera sensors.

17. The method of claim 16, wherein the field of view of the one or more camera sensors is changed based on the position associated with the second subject, according to determining that the second subject is the person closest to the center of the field of view of the one or more camera sensors when the first subject is determined to be outside the field of view of the one or more camera sensors for the predetermined time period.

18. The method of any one of claims 16 to 17, wherein the field of view of the one or more camera sensors is changed based on the determination that the second subject is a person previously identified within the field of view of the one or more camera sensors, based on the location associated with the second subject.

19. The method according to any one of claims 16 to 18, wherein the location associated with the second body is the last known location of the second body.

20. The method according to claim 19, further comprising: After changing the field of view of the one or more camera sensors based on the position of the second subject: Based on the position of the first subject, the field of view of the one or more camera sensors is changed according to the predetermined time period during which the second subject is determined to be outside the field of view of the one or more camera sensors.

21. The method according to any one of claims 16 to 20, further comprising: According to the corresponding subject tracking operation mode, video is captured using the one or more camera sensors, including tracking a first subject; as well as Once it is determined that the movable mounting component has reached its movement limit, tracking of the first subject is stopped.

22. The method according to any one of claims 1 to 21, further comprising: When capturing video using one or more camera sensors of the computer system according to the corresponding subject tracking operation mode: Based on determining that one or more movement characteristics of the corresponding subject meet a first criterion, the field of view displayed by the one or more camera sensors is changed; as well as Based on the determination that one or more movement characteristics of the corresponding subject do not meet the first criterion, the field of view displayed by the one or more camera sensors is maintained.

23. The method according to claim 22, further comprising: When capturing video using the one or more camera sensors of the computer system according to the corresponding subject tracking operation mode, and after changing the field of view displayed by the one or more camera sensors based on determining that one or more movement characteristics of the corresponding subject meet the first criterion: Based on determining that one or more movement characteristics of the corresponding subject meet a second predetermined criterion, the field of view displayed by the one or more camera sensors is changed; as well as Based on the determination that the one or more movement characteristics of the corresponding subject do not meet the second predetermined criterion, the field of view displayed by the one or more camera sensors is maintained.

24. The method according to any one of claims 1 to 23, further comprising: Based on the determination that the corresponding subject tracking operation mode is enabled, an indication that the corresponding subject tracking operation mode is enabled is output.

25. The method of claim 24, wherein the indication that enables the output of the corresponding subject tracking operation mode comprises: Based on the determination that the corresponding subject tracking operation mode is active, the output is the indication with the first characteristic.

26. The method of claim 24, wherein the indication that enables the output of the corresponding subject tracking operation mode comprises: Based on the determination that the corresponding subject tracking operation mode is not in an active state, the output has a second characteristic that is different from the first characteristic.

27. A computer-readable storage medium, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more camera sensors, the one or more programs including instructions for performing the method according to any one of claims 1 to 26.

28. A computer system configured to communicate with one or more camera sensors, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 26.

29. A computer system configured to communicate with one or more camera sensors, the computer system comprising: Components for performing the method according to any one of claims 1 to 26.

30. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more camera sensors, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 26.

Images