Method for capturing content

Abstract

This disclosure relates generally to capturing content. Described herein are some techniques for physically moving a camera to capture a static representation of an environment. In such techniques, a static representation can be performed for a video call such that the video of the video call can be combined with the static representation to provide a greater field of view for a recipient. Described herein are other techniques for moving a camera toward different subjects. In such techniques, a camera can be oriented toward a primary subject during content capture, and in response to a gaze of the primary subject satisfying a set of criteria relative to another subject, the camera can be oriented toward the other object.

Description

Cross-references to related applications

[0001] This international patent application claims priority to U.S. Nonprovisional Patent Application Serial No. 19 / 045,815, filed February 5, 2025, entitled “METHODS FOR CAPTURING CONTENT,” and U.S. Provisional Patent Application Serial No. 63 / 575,480, filed April 5, 2024, entitled “METHODS FOR CAPTURING CONTENT,” the entire contents of which are incorporated herein by reference for all purposes. Background Technology

[0002] Electronic devices are increasingly being used for content capture. For example, they are commonly used for video calls and video recording. As electronic devices become more diverse and capable, managing the capture of this content has become increasingly difficult. Therefore, there is a need to improve the technologies used for content capture. Summary of the Invention

[0003] Current technologies used for capturing content are often ineffective and / or inefficient. For example, some technologies limit the field of view of the content and / or require the user to physically move the device to capture different fields of view. This disclosure provides an example of automatically moving a smartphone to illustrate a more effective and / or efficient technology for capturing content. It should be recognized that other types of electronic devices can be used in conjunction with the technologies described herein. For example, removable mounts and / or head-mounted display devices can use the technologies described herein. Furthermore, these technologies may optionally complement or replace other technologies used for capturing content.

[0004] This document describes several techniques for physically moving a camera (and / or computing system) on a movable mount to capture a static representation of the environment. In such techniques, a static representation can be performed once for a video call, allowing the video of the video call and the static representation to be combined to provide a larger field of view for the receiver. In some embodiments, the static representation is transmitted separately from the video call by a transmitting device, allowing the receiving device to combine the static representation with the video call while receiving the video call. In other embodiments, the transmitting device combines the static representation and the video call to transmit the combined frame as a video call to the receiving device. This document also describes other techniques for pointing a camera toward different subjects. In such techniques, the camera can be pointed toward a primary subject during content capture, and the camera can be pointed toward another object in response to a set of criteria satisfied by the primary subject's gaze relative to another subject.

[0005] In some embodiments, a method is described that is performed at a first computer system communicating with a moving component and one or more cameras. In some embodiments, the method includes: connecting to a second computer system different from the first computer system via a communication session; physically moving one or more cameras via the moving component in conjunction with the connection to the second computer system via the communication session; and capturing a first representation of a portion of the environment via the one or more cameras while the cameras are physically moved; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

[0006] In some embodiments, a non-transitory computer-readable storage medium is described, which stores one or more programs configured to be executed by one or more processors of a first computer system in communication with a moving component and one or more cameras. In some embodiments, the one or more programs include instructions for performing the following operations: connecting to a second computer system different from the first computer system via a communication session; in conjunction with connecting to the second computer system via the communication session: physically moving one or more cameras via the moving component; and capturing a first representation of a portion of the environment via the one or more cameras while physically moving the cameras; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment and wherein the second representation is different from the first representation.

[0007] In some embodiments, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a first computer system in communication with a moving component and one or more cameras. In some embodiments, the one or more programs include instructions for performing the following operations: connecting to a second computer system different from the first computer system via a communication session; in conjunction with the connection to the second computer system via the communication session: physically moving one or more cameras via the moving component; and capturing a first representation of a portion of the environment via the one or more cameras while physically moving the cameras; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment and wherein the second representation is different from the first representation.

[0008] In some embodiments, a first computer system configured to communicate with a moving component and one or more cameras is described. In some embodiments, the first computer system includes: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for performing the following operations: connecting to a second computer system different from the first computer system via a communication session; physically moving one or more cameras via the moving component in conjunction with the connection to the second computer system via the communication session; and capturing a first representation of a portion of the environment via the one or more cameras while the cameras are physically moved; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment and wherein the second representation is different from the first representation.

[0009] In some embodiments, a first computer system configured to communicate with a moving component and one or more cameras is described. In some embodiments, the first computer system includes components for performing each of the following steps: connecting to a second computer system different from the first computer system via a communication session; combining the connection to the second computer system via the communication session with: physically moving one or more cameras via the moving component; and capturing a first representation of a portion of the environment via the one or more cameras while physically moving the cameras; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

[0010] In some embodiments, a computer program product is described. In some embodiments, the computer program product includes one or more programs configured to execute by one or more processors of a first computer system communicating with a moving component and one or more cameras. In some embodiments, the one or more programs include instructions for performing the following operations: connecting via a communication session to a second computer system different from the first computer system; combining the connection via the communication session to the second computer system with: physically moving one or more cameras via the moving component; and while physically moving the cameras, capturing a first representation of a portion of the environment via the one or more cameras; and after capturing the first representation of the portion of the environment, transmitting a second representation of the portion of the environment to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment and wherein the second representation is different from the first representation.

[0011] In some embodiments, a method is described that is performed at a computer system communicating with a mobile component and one or more input devices. In some embodiments, the method includes: while capturing media content via one or more input devices: oriented one or more input devices toward a user via the mobile component; while oriented one or more input devices toward the user, detecting the user's gaze at a location via the one or more input devices; and in response to detecting the user's gaze at the location: continuing to oriented one or more input devices toward the user via the mobile component based on determining that the user's gaze does not meet a set of one or more criteria; and oriented one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0012] In some embodiments, a non-transitory computer-readable storage medium is described, which stores one or more programs configured to be executed by one or more processors of a computer system communicating with a moving component and one or more input devices. In some embodiments, the one or more programs include instructions for performing the following operations: when capturing media content via the one or more input devices: oriented the one or more input devices toward a user via the moving component; when oriented the one or more input devices toward the user, detecting the user's gaze at a location via the one or more input devices; and in response to detecting the user's gaze at the location: continuing to oriented the one or more input devices toward the user via the moving component based on determining that the user's gaze does not meet a set of one or more criteria; and oriented the one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0013] In some embodiments, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system communicating with a moving component and one or more input devices. In some embodiments, the one or more programs include instructions for performing the following operations: when capturing media content via the one or more input devices: oriented the one or more input devices toward a user via the moving component; when oriented the one or more input devices toward the user, detecting the user's gaze at a location via the one or more input devices; and in response to detecting the user's gaze at the location: continuing to oriented the one or more input devices toward the user via the moving component based on determining that the user's gaze does not meet a set of one or more criteria; and oriented the one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0014] In some embodiments, a computer system configured to communicate with a moving component and one or more input devices is described. In some embodiments, the computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for performing the following operations: while capturing media content via the one or more input devices: facing the one or more input devices toward a user via the moving component; detecting the user's gaze at a location via the one or more input devices while facing the user; and in response to detecting the user's gaze at the location: continuing to face the one or more input devices toward the user via the moving component based on determining that the user's gaze does not meet a set of one or more criteria; and facing the one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0015] In some embodiments, a computer system configured to communicate with a mobile component and one or more input devices is described. In some embodiments, the computer system includes components for performing each of the following steps: when capturing media content via one or more input devices: oriented one or more input devices toward a user via the mobile component; when oriented one or more input devices toward the user, detecting the user's gaze at a location via the one or more input devices; and in response to detecting the user's gaze at the location: continuing to oriented one or more input devices toward the user via the mobile component based on determining that the user's gaze does not meet a set of one or more criteria; and oriented one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0016] In some embodiments, a computer program product is described. In some embodiments, the computer program product includes one or more programs configured to execute by one or more processors of a computer system communicating with a mobile component and one or more input devices. In some embodiments, the one or more programs include instructions for performing the following operations: while capturing media content via one or more input devices: oriented one or more input devices toward a user via the mobile component; while oriented one or more input devices toward the user, detecting the user's gaze at a location via the one or more input devices; and in response to detecting the user's gaze at the location: continuing to oriented one or more input devices toward the user via the mobile component based on determining that the user's gaze does not meet a set of one or more criteria; and oriented one or more input devices toward the location based on determining that the user's gaze meets a set of one or more criteria.

[0017] Executable instructions for performing these functions may optionally be included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors. Attached Figure Description

[0018] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein the same reference numerals in all the drawings indicate corresponding parts.

[0019] Figure 1 This is a block diagram illustrating a computing system based on some implementation schemes.

[0020] Figure 2 This is a block diagram illustrating a device with interconnected subsystems according to some implementation schemes.

[0021] Figures 3A to 3D An exemplary user interface for capturing content is illustrated according to some implementation schemes.

[0022] Figure 4 This is a flowchart illustrating a method for capturing content according to some implementation schemes.

[0023] Figures 5A to 5E An exemplary user interface for capturing content while on the move is illustrated according to some implementation schemes.

[0024] Figure 6 This is a flowchart illustrating a method for capturing content while in motion, according to some implementation schemes. Detailed Implementation

[0025] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0026] The methods described herein may include one or more steps depending on the satisfaction of one or more conditions. It should be understood that a method may occur in multiple iterations of the same process, where different steps of the method are satisfied in different iterations. For example, if a method requires performing a first step upon determining that one or more criteria are satisfied, and a second step upon determining that one or more criteria are not satisfied, those skilled in the art will understand that the steps of the method are repeated until the two conditions are not satisfied in a particular order. Therefore, a method described as having steps depending on the satisfaction of conditions can be rewritten as a method that repeats until each of the conditions described in the method is satisfied. However, this is not required by a system claim or computer-readable medium claim, which includes instructions for performing one or more steps depending on the satisfaction of one or more conditions. Because the instructions for a system claim or computer-readable medium claim are stored in one or more processors and / or at one or more memory locations, a system claim or computer-readable medium claim includes logic that determines whether one or more conditions have been satisfied without explicitly repeating the steps of the method until all conditions under which the steps in the method are based have been satisfied. Those skilled in the art will also understand that, similar to a method having discretionary steps, a system or computer-readable storage medium may repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.

[0027] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. In some examples, these terms are used to distinguish individual elements. For example, without departing from the scope of the various described embodiments, a first subsystem may be referred to as a second subsystem, and similarly, a second subsystem device or subsystem device may be referred to as a first subsystem device. In some examples, the first subsystem and the second subsystem are two separate references to the same subsystem. In some examples, both the first subsystem and the second subsystem are subsystems, but they are not the same subsystem or subsystems of the same type.

[0028] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in response to," "in response to detection," or "according to determination." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to," "in response to," "when [the stated condition or event] is detected," "in response to," or "according to"."

[0030] Go to Figure 1 A block diagram of computing system 100 is illustrated. Computing system 100 is a non-limiting example of a computing system that can be used to perform the functions described herein. It should be appreciated that other computer architectures can be used to perform the functions described herein.

[0031] In the illustrated example, computing system 100 includes a processor subsystem 110 that communicates (e.g., wired or wirelessly) with memory 120 (e.g., system memory) and I / O interface 130 via interconnect 150 (e.g., a system bus, one or more memory locations, or other communication channels for connecting multiple components of computing system 100). Furthermore, I / O interface 130 communicates (e.g., wired or wirelessly) with I / O device 140. In some examples, I / O interface 130 and I / O device 140 are included together such that they are a single component. It should be appreciated that one or more I / O interfaces may exist, with each I / O interface communicating with one or more I / O devices. In some examples, multiple instances of processor subsystem 110 may communicate via interconnect 150.

[0032] The computing system 100 can be any type of device from a variety of types, including but not limited to system-on-a-chip, server systems, personal computer systems (e.g., smartphones, smartwatches, wearable devices, tablets, laptops, and / or desktop computers), sensors, etc. In some examples, the computing system 100 includes or communicates with physical components for the purpose of modifying physical components in response to instructions. In some examples, the computing system 100 receives instructions for modifying physical components and, in response to those instructions, causes the physical components to be modified. In some examples, the physical components are modified via actuators, electrical signals, and / or algorithms. Examples of such physical components include accelerometer controls, brakes, gearboxes, hinges, motors, pumps, cooling systems, springs, suspension systems, steering controls, pumps, vacuum systems, and / or valves. In some examples, the sensor includes one or more hardware components that detect information about the physical environment in the vicinity (e.g., the surroundings) of the sensor. In some examples, the hardware components of the sensor include sensing components (e.g., image sensors or temperature sensors), transmitting components (e.g., laser or radio transmitters), receiving components (e.g., laser or radio receivers), or any combination thereof. Examples of sensors include angle sensors, chemical sensors, braking pressure sensors, contact sensors, non-contact sensors, electrical sensors, flow sensors, force sensors, gas sensors, humidity sensors, image sensors (e.g., camera sensors, radar sensors, and / or lidar sensors), inertial measurement units, leak sensors, level sensors, light detection and ranging systems, metal sensors, motion sensors, particle sensors, photoelectric sensors, positioning sensors (e.g., GPS), sediment sensors, pressure sensors, proximity sensors, radio detection and ranging systems, radiation sensors, velocity sensors (e.g., measuring the velocity of an object), temperature sensors, time-of-flight sensors, torque sensors, and ultrasonic sensors. In some examples, sensors include combinations of multiple sensors. In some examples, sensor data is captured by fusing data from one sensor with data from one or more other sensors. Although in Figure 1 A single computing system is shown, but computing system 100 can also be implemented as two or more computing systems operating together.

[0033] In some examples, processor subsystem 110 includes one or more processors or processing units configured to execute program instructions to perform the functionality described herein. For example, processor subsystem 110 may execute an operating system, middleware system, one or more applications, or any combination thereof.

[0034] In some examples, the operating system manages the resources of computing system 100. Examples of operating system types covered in this document include bulk operating systems (e.g., multiple virtual memory (MVS)), time-sharing operating systems (e.g., Unix), distributed operating systems (e.g., Advanced Interactive Execution (AIX)), network operating systems (e.g., Microsoft Windows Server), and real-time operating systems (e.g., QNX). In some examples, the operating system includes various programs, sets of instructions, software components, and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and for facilitating communication between various hardware and software components. In some examples, the operating system uses a priority-based scheduler that assigns priorities to different tasks that processor subsystem 110 can execute. In such examples, the priority assigned to a task is used to identify the next task to be executed. In some examples, the priority-based scheduler identifies the next task to be executed when a previous task has completed execution. In some examples, the highest priority task runs until completion unless another higher priority task becomes ready.

[0035] In some examples, the middleware system provides applications (e.g., one or more applications running on processor subsystem 110) with one or more services and / or capabilities beyond what the operating system provides (e.g., data management, application services, messaging, authentication, API management, etc.). In some examples, the middleware system is designed for heterogeneous computer clusters to provide hardware abstraction, low-level device control, implementation of common functionality, messaging between processes, package management, or any combination thereof. Examples of middleware systems include Lightweight Communication and Marshaling (LCM), PX4, Robot Operating System (ROS), and ZeroMQ. In some examples, the middleware system uses a graphical architecture to represent processes and / or operations, where processing occurs in nodes that can receive, publish, and reuse sensor data messages, control messages, status messages, planning messages, actuator messages, and other messages. In such examples, the graphical architecture may define applications (e.g., applications executing on processor subsystem 110 as described above) such that different operations of the application are included with different nodes in the graphical architecture.

[0036] In some examples, a publish-subscribe model is used to execute message delivery from a first node in the graph architecture to a second node in the graph architecture, where the first node publishes data on a channel that the second node can subscribe to. In such examples, the first node may store data in memory (e.g., memory 120 or a local memory of processor subsystem 110) and notify the second node that the data has been stored in memory. In some examples, the first node notifies the second node that the data has been stored in memory by sending a pointer (e.g., a memory pointer, such as an identifier of a memory location) to the second node, enabling the second node to access the data from the location where the first node stores the data. In some examples, the first node will send data directly to the second node, so that the second node will not need to access memory based on data received from the first node.

[0037] Memory 120 may include a computer-readable medium (e.g., a non-transitory or transient computer-readable medium) that can be used to store (e.g., configured to store, assigned to store, and / or store) program instructions that can be executed by processor subsystem 110 to cause computing system 100 to perform the various operations described herein. For example, memory 120 may store program instructions to implement methods 400 and 600 described below. Figure 4 and Figure 6 Related functionality.

[0038] Memory 120 may be implemented using various physical non-transitory memory media, such as hard disk storage devices, floppy disk storage devices, removable disk storage devices, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDRSDRAM, RAMBUS RAM, etc.), read-only memory (PROM, EEPROM, etc.). The memory in computing system 100 is not limited to main memory devices such as memory 120. Computing system 100 may also include other forms of storage devices, such as cache memory in processor subsystem 110 and auxiliary storage devices (e.g., hard disk drives, storage arrays, etc.) on I / O devices 140. In some examples, these other forms of storage devices may also store program instructions executable by processor subsystem 110 to perform the operations described herein. In some examples, processor subsystem 110 (or each processor within processor subsystem 110) includes cache or other forms of onboard memory.

[0039] I / O interface 130 can be any type of interface configured to communicate with other devices. In some examples, I / O interface 130 includes a bridge chip (e.g., a southbridge) connecting a front-side bus to one or more back-side buses. I / O interface 130 can communicate with one or more I / O devices (e.g., I / O device 140) via one or more corresponding buses or other interfaces. Examples of I / O devices include storage devices (hard disk drives, optical disk drives, removable flash drives, storage arrays, SANs, or their associated controllers), network interface devices (e.g., to a local area network or wide area network), sensor devices (e.g., cameras, radar, lidar, ultrasonic sensors, GPS, inertial measurement devices, etc.), and auditory or visual output devices (e.g., speakers, lights, screens, projectors, etc.). In some examples, computer system 100 communicates with a network via a network interface device (e.g., configured to communicate via Wi-Fi, Bluetooth, Ethernet, etc.). In some examples, computing system 100 is directly or wired connected to a network.

[0040] Figure 2 A block diagram of a device 200 with interconnected subsystems is illustrated. In the illustrated example, device 200 includes three distinct subsystems (i.e., a first subsystem 210, a second subsystem 220, and a third subsystem 230) that communicate with each other (e.g., wired or wirelessly) to create networks (e.g., personal area networks, local area networks, wireless LANs, metropolitan area networks, wide area networks, storage area networks, virtual private networks, enterprise intranets, campus area networks, system area networks, and / or controller area networks). Figure 1 (That is, computing system 100) describes as follows: Figure 2 Examples of possible computer architectures for the subsystems included. Although in Figure 2 Three subsystems are shown, but device 200 may include more or fewer subsystems.

[0041] In some examples, some subsystems are not connected to other subsystems (e.g., first subsystem 210 can be connected to second subsystem 220 and third subsystem 230, but second subsystem 220 cannot be connected to third subsystem 230). In some examples, some subsystems are connected via one or more wires, while other subsystems are wirelessly connected. In some examples, a message is set between first subsystem 210, second subsystem 220, and third subsystem 230 such that when a corresponding subsystem transmits a message, other subsystems (e.g., via wires and / or buses) receive that message. In some examples, one or more subsystems are wirelessly connected to one or more computing systems, such as server systems, external to device 200. In such examples, subsystems can be configured to communicate wirelessly with one or more computing systems external to device 200.

[0042] In some examples, device 200 includes a housing that completely or partially encloses subsystems 210 to 230. Examples of device 200 include household appliances (e.g., refrigerators or air conditioning systems), robots (e.g., robotic arms or robotic vacuum cleaners), and vehicles. In some examples, device 200 is configured to navigate in a physical environment (with or without user input).

[0043] In some examples, one or more subsystems of device 200 are used to control, manage, and / or receive data from one or more other subsystems of device 200 and / or one or more computing systems remote from device 200. For example, the first subsystem 210 and the second subsystem 220 may each be a camera capturing images, and the third subsystem 230 may use the captured images for decision-making. In some examples, at least a portion of device 200 is used as a distributed computing system. For example, a task may be broken down into different parts, with a first part performed by the first subsystem 210 and a second part performed by the second subsystem 220.

[0044] Now let’s turn our attention to the technologies used for capturing content. These technologies are described in the context of connecting smartphones to head-mounted display (HMD) devices. It should be recognized that other types of electronic devices can be used in conjunction with the technologies described herein. For example, an HMD device can use the technologies described herein to connect to another HMD device. Furthermore, the technologies may optionally complement or replace other technologies used for connecting devices.

[0045] Figures 3A to 3D Exemplary user interfaces for capturing content using a computer system, according to some embodiments, are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 4 The process in.

[0046] Figures 3A to 3DThe left side of the diagram illustrates computer system 300 as a smartphone. It should be understood that computer system 300 can be other types of computer systems, such as tablets, smartwatches, laptops, public facilities, smart speakers, personal gaming systems, desktop computers, fitness trackers, and / or head-mounted display (HMD) devices. In some embodiments, computer system 300 includes one or more input devices (e.g., cameras, depth sensors, microphones, hardware input mechanisms, rotatable input mechanisms, heart monitors, temperature sensors, and / or touch-sensitive surfaces) and / or communicates with such one or more input devices. In some embodiments, computer system 300 includes one or more output devices (e.g., display components (e.g., displays, projectors, and / or touch-sensitive displays), audio components (e.g., smart speakers, home theater systems, soundbars, binaural headphones, headphones, earbuds, speakers, TV speakers, augmented reality headset speakers, audio jacks, optical audio outputs, Bluetooth audio outputs, and / or HDMI audio outputs), speakers, haptic output devices, displays, projectors, and / or touch-sensitive displays)) and / or communicates with such one or more output devices.

[0047] Figures 3A to 3D The right side illustrates the different locations of computer system 300 within physical environment 330. Physical environment 330 includes device representation 332 (which represents the location of computer system 300 within physical environment 330) and user representation 334 (which represents the location of user 310 within physical environment 330). Figures 3A to 3D As illustrated, the physical environment 330 also includes a field of view 336, which represents the orientation of the computer system 300 within the physical environment 330. In some embodiments, the field of view 336 represents the field of view of a camera of the computer system 300.

[0048] like Figures 3A to 3D As illustrated, computer system 300 displays user interface 302. In some embodiments, user interface 302 is a user interface displayed by computer system 300 when computer system 300 receives a video call. In some embodiments, user interface 302 is displayed in response to computer system 300 detecting a request to establish a video call. It should be understood that user interface 302 is merely one example of a user interface used with the techniques described herein, and other types of user interfaces may also be used.

[0049] like Figures 3A to 3D As illustrated, user interface 302 includes a live view 304. In some embodiments, the live view 304 is a representation (e.g., an image or video) of the physical environment 330 captured by a camera. Figures 3A to 3D As illustrated, this representation includes user 310. For example... Figures 3A to 3C As illustrated, the live view 304 is displayed by the computer system 300 at the center of the user interface 302. Figure 3D As illustrated, the live view 304 is displayed by the computer system 300 in a corner of the user interface 302. In some embodiments, the live view 304 is displayed by the computer system 300 across the entire user interface 302. In some embodiments, the live view 304 is displayed by the computer system 300 at the bottom or top of the user interface 302.

[0050] exist Figure 3A At this point, computer system 300 receives a video call from John Appleseed's device, as indicated by call status indication 312. In some embodiments, John Appleseed's device includes a camera. In some embodiments, when John Appleseed's device transmits a video call to computer system 300, the device transmits media captured by the camera of John Appleseed's device.

[0051] like Figure 3A As illustrated, user representation 334 is within field of view 336 of physical environment 330. In Figure 3A At this point, computer system 300 captures and displays user 310 within a live view 304. In some embodiments, computer system 300 displays user 310 within live view 304 without displaying another object or user in the environment. For example, computer system 300 identifies user 310 and determines that no other user exists in the physical environment 330. Based on this determination, computer system 300 may identify user 310 as the primary subject in the physical environment 330 (e.g., because user 310 is the only person in the view) and modify the camera's field of view 336 based on user 310 being identified as the primary subject (e.g., by physically moving the camera, as further described below).

[0052] In some implementations, the live view 304 includes other aspects of the physical environment 330 within the field of view 336, including walls and / or objects within the physical environment 330. For example, in some implementations, a bookshelf is behind the user representation 334. In this example, the live view 304 includes the bookshelf because it is behind the user representation 334 but within the field of view 336.

[0053] For example Figure 3A As illustrated, the user interface 302 includes an accept control 306 and a reject control 308 at the bottom of the user interface 302. For example... Figure 3AAs illustrated, accept control 306 and reject control 308 are displayed simultaneously with and on top of at least a portion of the live view 304 (e.g., overlapping with the at least a portion). In some embodiments, the computer system 300 rejects the video call in response to detecting input to the reject control 308. In some embodiments, the computer system 300 stops displaying user interface 302 in response to rejecting the video call. In some embodiments, after the computer system 300 stops displaying user interface 302 in response to rejecting the video call, the computer system 300 displays another user interface, such as a home screen user interface or a lock screen.

[0054] exist Figure 3A At this point, computer system 300 detects a tap input 305a on the accept control 306. In some embodiments, tap input 305a is received from user 310. It should be recognized that tap input is merely one example of the types of input that can be used with the techniques described herein, and other types of input can be used, such as voice input detected via a microphone (e.g., “accept call”) and / or air gestures detected via the camera of computer system 300. Although the detection of tap input 305a on the accept control 306 for accepting an incoming call has been described above, in some embodiments, computer system 300 detects tap input on a user interface element for initiating an outgoing call. For example, computer system 300 detects tap input on a name displayed by computer system 300 to initiate one or more processes described below, such as Figures 3B to 3D exemplified.

[0055] exist Figure 3ASubsequently, in response to detecting a tap input 305a on the accept control 306, the computer system 300 initiates the capture of different portions of the physical environment 330. In some embodiments, a static representation of the physical environment 330 is generated after and / or simultaneously with the capture of the physical environment 330. In some embodiments, different portions of the physical environment 330 are captured to create a static representation (e.g., a 3D and / or 2D model) of the physical environment 330 for transmission to John Appleseed's device. In such embodiments, the static representation may include a live view 304 or something different from that live view. In some embodiments, the static representation of the physical environment 330 includes multiple portions of the physical environment 330 to represent the physical environment 330. For example, the static representation of the physical environment 330 includes multiple photographs aligned together to represent the physical environment 330. In other words, in some embodiments, the computer system 300 captures a static representation of the physical environment 330 by capturing a series of photographs and / or videos of the physical environment 330. For example, the computer system 300 uses many different perspectives and many different portions of the physical environment 300 to generate the static representation of the physical environment 330. In some implementations, a static representation of the physical environment 330 includes computer-generated objects representing the physical environment 330. For example, computer system 300 generates computer-generated objects based on the capture of different portions of the physical environment. In another example, computer system 300 uses a remote sensing system (e.g., LiDAR) to capture the physical environment 330. In yet another example, computer system 300 uses media captured by a camera to capture the physical environment 330 and generates a static representation of the physical environment 330 by determining the depth of various features within the physical environment 330.

[0056] In some embodiments, the static representation of the physical environment 330 includes portions of the physical environment 330 that are determined to be static and / or portions that do not include the main elements within the physical environment 330 (referred to as the static portion of the environment 330). In some embodiments, the static portion of the physical environment 330 includes objects in the environment that are determined to be unlikely to move during a video call. For example, objects in the environment determined to be unlikely to move include furniture, decorations, walls, and a television. In some embodiments, the computer system 300 removes real-time objects and / or non-static objects from the static representation of the physical environment 330 while or after capturing the static representation of the physical environment 330.

[0057] In some implementations, computer system 300 translates, tilts, moves, and / or rotates a camera (and / or a portion of computer system 300) to capture a portion and / or all of the physical environment 330. For example, computer system 300 can move field of view 336 by activating components of computer system 300, such as actuators that are part of computer system 300, where movement of the actuators alters and / or displaces field of view 336. In such examples, the actuators can move a portion of computer system 300, including the camera, to different orientations and / or directions, thereby altering field of view 336. As another example, computer system 300 can move field of view 336 by activating components remote from and / or separate from computer system 300, such as actuators physically and / or magnetically coupled to a mount of computer system 300. In such examples, computer system 300 can communicate with the mount and control the positioning of the mount by transmitting control messages to the mount. It should be recognized that, in addition to or instead of the techniques described above, other techniques known to those skilled in the art may be used to achieve the movement of the field of view 336.

[0058] In some implementations, computer system 300 moves using the techniques described above to capture the entire physical environment 330 by physically moving to capture portions of the physical environment 330 that are not within the field of view 336. For example, computer system 300 rotates 360 degrees to capture the physical environment 330. Such movement allows computer system 300 to capture a larger portion of the physical environment 330 than it might capture without moving. For example, because computer system 300 rotates to capture a larger portion of the physical environment 330, computer system 300 captures portions of the physical environment 330 that are outside the field of view 336 when in a stationary position. In another example, while computer system 300 is capturing the physical environment 330, computer system 300 determines that an obstruction exists that blocks the capture of a portion of the physical environment 330. For example, computer system 300 determines that a pillar is blocking the capture of a sofa behind the pillar from the camera's angle. In some implementations, computer system 300 moves to a position no longer obstructed by an obstruction and captures the missing portion of the physical environment 330. For example, computer system 300 moves to a location that bypasses a pillar obstructing the capture of a sofa. In another example, computer system 300 detects gaps (e.g., around a corner) in a static representation of the physical environment 330. In this example, computer system 300 moves to a certain location (e.g., around a corner) to capture the entire physical environment 330.

[0059] Although the description above and below discusses the inclusion of the entire physical environment 330 in the static representation, in some embodiments, only a portion of the physical environment 330 is included in the static representation. For example, the computer system 300 is stationary and cannot be translated, tilted, moved, and / or rotated. As another example, the computer system 300 is included in the static representation only in portions outside the area including the main subject and / or outside the field of view including the main subject. As yet another example, the computer system 300 includes only a portion of a certain area of ​​the physical environment 330, such as a user-defined portion of the computer system 300. In some embodiments, the computer system 300 moves the field of view 336 by performing software-based operations that alter the field of view 336 without physically moving the camera. For example, the camera may capture a larger field of view than illustrated in the live view 304. In such examples, Figure 3A and / or Figure 3B The real-time view 304 may be the result of cropping the field of view 336 captured by the camera. It should be appreciated that other software-based operations known to those skilled in the art can be performed to alter the field of view 336.

[0060] In some implementations, a static representation of the physical environment 330 is captured and provided to John Appleseed's device. In some implementations, John Appleseed's device controls the computer system 300 to move within the physical environment 330. For example, John Appleseed's device is an HMD device, and during a video call, the HMD device moves within its own environment to request to look around the physical environment 330. Because John Appleseed's device receives a static representation of the physical environment 330, the computer system 300 does not move when John Appleseed's device requests to look around the physical environment 330. Instead, John Appleseed's device uses the static representation of the physical environment 330 to present different perspectives of the physical environment 330. In some implementations, movement of John Appleseed's device is translated into movement of the computer system 300. For example, when John Appleseed's device looks to the left, the computer system 300 correspondingly rotates the field of view 336 to the left side of the physical environment 330. In some implementations, computer system 300 cannot rotate at the same speed as John Appleseed's device, and a static representation of the physical environment 330 is shown to John Appleseed until computer system 300 catches up with the movement of computer system 300's camera.

[0061] Although the description above and below discusses capturing the physical environment 330 and generating a static representation before connecting a video call to John Appleseed's device, in some embodiments, the computer system 300 captures the physical environment 330 and / or generates a static representation at different points in time. For example, the computer system 300 may capture the physical environment 330 at a time before receiving a video call. For example, the computer system 300 may capture the physical environment 330 and / or generate a static representation relative to different video calls (e.g., before and / or during those different video calls). In such examples, the computer system 300 uses a static representation of the physical environment 330 from a previous video call for the video call with John Appleseed's device. As another example, the computer system 300 may generate a static representation during a video call and then transmit the static representation to John Appleseed's device after generation. As yet another example, the computer system 300 may periodically capture the physical environment 330 (such as after a certain amount of time since the last capture of the physical environment 330) to maintain an up-to-date static representation. In some implementations, computer system 300 captures physical environment 330 and / or generates static representations only when receiving or making certain calls to certain devices. For example, computer system 300 may generate a static representation before making a video call to a device acting as an HMD device.

[0062] like Figure 3B exemplified, and Figure 3A In comparison, field of view 336 rotates to the right, while the user states that 334 is in the same position as... Figure 3A The same position is illustrated. In some embodiments, the field of view 336 rotates to the right as the computer system 300 moves the camera to the right. As described above, the computer system 300 moves the camera to the right to scan additional portions of the physical environment 330.

[0063] like Figure 3B As illustrated, in response to detecting a tap input 305a on the accept control 306, the computer system 300 updates the display of the call status indicator 312 to indicate that the computer system 300 is capturing the physical environment 300 (e.g., "Capturing environment..."), while continuing to display the live view 304 and the call status indicator 312 in the user interface 302. Figure 3B As illustrated, below the call status indicator 312, the user interface 302 includes controls for video calls, including a speaker control 314, a camera control 316, a mute control 318, a share control 320, and an end control 322. Figure 3BAs illustrated, at the bottom of the user interface 302, the user interface 302 includes additional controls for video calls, including a camera control 324, a focus control 326, and a camera adjustment control 328. Figure 3B At this point, the computer system 300 stops displaying the accept control 306 and the reject control 308 in response to detecting a tap input 305a.

[0064] Will Figure 3B The right side and Figure 3B The left side is compared, and even if the field of view 336 rotates within the physical environment 330, the computer system 300 is still included in the user 310 at the center of the user interface 302. In some embodiments, the computer system 300 continues to be used with the user 310, which is being rotated to change the view. Figure 3B The right-hand field of view 336 is occupied by a different camera to capture user 310. In some embodiments, the computer system 300 rotates the same camera (or rotates different cameras in the same motion) and partially captures user representation 334 (e.g., user 310), and user 310 is in... Figure 3B The view is displayed on the left, partially off-screen from the real-time view 304. In other embodiments, the real-time view 304 is not displayed when the computer system 300 captures the physical environment 330 and / or moves the camera, which includes the field of view 336, away from the user representation 334. In some embodiments, the computer system 300 does not return the user 310 to the center of the real-time view 304 until the field of view 336 is centered on the user representation 334.

[0065] exist Figure 3C In response to computer system 300 completing the capture (and / or generation of a static representation) of physical environment 330, computer system 300 initiates a video call with John Appleseed's device. Figure 3C As illustrated, computer system 300 continues to display user interface 302, which includes a live view 304 containing user 310. (As...) Figure 3C As illustrated, computer system 300 updates the display of call status indication 312 to indicate that computer system 300 is connecting a video call to John Appleseed's device (e.g., "Connecting...").

[0066] like Figure 3C As illustrated, device display 332 causes field of view 336 to return to user-facing display 334, as initially... Figure 3A exemplified. exist Figure 3CAt this point, computer system 300 returns field of view 336 to user-facing display 334 because computer system 300 has completed capturing physical environment 330. In some implementations, computer system 300 returns field of view 336 to user-facing display 334 to capture media from user 310 for use in video calls.

[0067] like Figure 3DAs illustrated, after computer system 300 returns field of view 336 to user-facing representation 334, computer system 300 initiates a video call. In some embodiments, initiating a video call includes transmitting both a static representation of the physical environment 330 and media captured by the camera to John Appleseed's device. In some embodiments, the static representation of the physical environment 330 is transmitted to John Appleseed's device to provide a static (e.g., as described above), occluded (e.g., as described above), and / or currently outside the field of view 336 portion of the physical environment 330. In some embodiments, computer system 300 captures the main subject of the video call (e.g., user 310) during the video call, without capturing other portions of the physical environment 330. In other words, in some embodiments, computer system 300 transmits media captured by the camera that includes only the camera's current field of view. In some implementations, John Appleseed's device uses two datasets (e.g., a static representation of physical environment 330 and media captured by a camera) (e.g., as described above) to generate a live view of physical environment 330 to be displayed by John Appleseed's device. For example, after John Appleseed's device receives the static representation of physical environment 330 and the media of physical environment 330 within field of view 336, John Appleseed's device uses the media and / or static representation of physical environment 330 to generate and display the live view. In such an example, in response to John Appleseed's device being moved to an orientation corresponding to an area outside the media of physical environment 330 captured by field of view 336, John Appleseed's device uses the static representation of physical environment 330 to generate and display the live view, which includes at least a portion of the static representation (and optionally, at least a portion of the media of physical environment 330 captured by field of view 336). In some embodiments, in response to John Appleseed's device being moved to an orientation corresponding to an area outside the media of the physical environment 330 captured by field of view 336, computer system 300 moves its camera to capture the area outside the media of the physical environment 330, while John Appleseed's device displays a static representation. In other embodiments, in response to John Appleseed's device being moved to an orientation corresponding to an area outside the media of the physical environment 330 captured by field of view 336, computer system 300 does not move its camera but continues to capture the first user 310.In such implementations, the orientation corresponding to the area outside the media of the physical environment 330 captured by the field of view 336 is displayed by John Appleseed's device using a static representation, without using the media of the physical environment 330 captured by the field of view 336.

[0068] In some implementations, computer system 300 transmits a static representation of the physical environment 330 and media captured by a camera to John Appleseed's device using different transmission methods. For example, in some implementations, different communication channels and / or bit rates are used to transmit the static representation of the physical environment 300 and the media captured by the camera.

[0069] In some implementations, computer system 300 continues to transmit media captured by the camera without continuing to transmit still representations. In other implementations, computer system 300 continues to update still representations and transmits them to John Appleseed's device as they are updated.

[0070] like Figure 3D As illustrated, device display 332 maintains field of view 336 facing user display 334. In some embodiments, in response to detecting movement of user 310, computer system 300 moves to maintain user display 334 in field of view 336 for video calls.

[0071] Similarly, Figure 3D As illustrated, computer system 300 displays a live view 304 at different locations on user interface 302. Figure 3D At this point, computer system 300 displays a live view 304 in the lower right corner of user interface 302. Similarly... Figure 3D As illustrated, computer system 300 displays camera control 324 and changes camera control 328 and stops displaying focus control 326. In some embodiments, computer system 300 maintains the display of live view 304 at the center of user interface 302, which includes camera control 324 and changes camera control 328, as well as focus control 326 (e.g., as described above in...). Figure 3B (As described in the text).

[0072] Similarly, Figure 3D As illustrated, computer system 300 displays a user interface 302 including a live view 340. For example... Figure 3D As illustrated, live view 340 includes a representation of John Appleseed 342. In some implementations, live view 340 is received from John Appleseed's device.

[0073] Figure 4This is a flowchart illustrating a method for capturing content (e.g., method 400) according to some implementation schemes. Some operations in method 400 may be optionally combined, the order of some operations may be optionally changed, and some operations may be optionally omitted.

[0074] As described below, Method 400 provides an intuitive way to capture content. Method 400 reduces the cognitive load on the user, thereby creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to interact with such devices faster and more efficiently, saving power and increasing the time between battery charging.

[0075] In some embodiments, method 400 is performed at a first computer system (e.g., 300) that communicates with mobile components (e.g., actuators (e.g., pneumatic actuators, hydraulic actuators, and / or electric actuators), movable bases, rotatable components, motors, lifts, levels, and / or rotatable bases) and one or more cameras (e.g., telephoto cameras, wide-angle cameras, and / or ultra-wide-angle cameras) via wired and / or wireless communication. In some embodiments, the computer system communicates with one or more input devices (e.g., depth sensors, microphones, hardware input mechanisms, rotatable input mechanisms, heart monitors, temperature sensors, and / or touch-sensitive surfaces). In some embodiments, the first computer system is a watch, telephone, tablet, fitness tracker, processor, head-mounted display (HMD) device, public utility, media device, speaker, television, and / or personal computing device.

[0076] The first computer system communicates via a communication session (e.g., as described above relative to...). Figures 3A to 3D The first computer system connects (402) to a second computer system (e.g., 300) that is different from the first computer system (as described above) (e.g., a call, video conferencing, and / or audio conferencing). In some embodiments, the first computer system connects to the second computer system via a communication session in response to the detection of a communication session via one or more input devices. In some embodiments, the second computer system is a watch, phone, tablet, fitness tracker, processor, head-mounted display (HMD) device, public utility, media device, speaker, television, and / or personal computing device. In some embodiments, the first computer system connects to the second computer system in response to the detection of a request to connect to the second computer system via an input device (e.g., a camera, depth sensor, microphone, hardware input mechanism, rotatable input mechanism, heart monitor, temperature sensor, and / or touch-sensitive surface) communicating with the first computer system.

[0077] In (404), in conjunction with a connection to a second computer system (e.g., 300) via a communication session (e.g., before, after, in response to, and / or during the connection), the first computer system physically moves (406) (e.g., translates, tilts, rotates, and / or changes the physical position) one or more cameras (e.g., as described above relative to) a moving component. Figure 3B (As described). In some implementations, the first computer system physically moves one or more cameras via a mobile component, in conjunction with a second computer system connected via a communication session.

[0078] In (404), in conjunction with a connection to a second computer system via a communication session, while the camera is physically moved, the first computer system (e.g., 300) captures (408) a first representation (e.g., an image and / or map) of a portion of the environment (e.g., a physical environment and / or a virtual environment) via one or more cameras (e.g., as described above relative to...). Figure 3B As described, the physical environment 330 is a static representation and / or media captured by a camera. In some embodiments, the first computer system physically moves the camera in a scanning motion, thereby capturing a portion of the environment larger than a portion captured by a stationary camera.

[0079] Following the capture of a first representation of a portion of the environment (e.g., simultaneously with and / or in response to the capture) (and / or after a call is established), the first computer system transmits (410) a second representation of the portion of the environment (e.g., a 3D model of a static portion and / or a background portion of the environment) to the second computer system via a communication session, wherein the second representation is based on the first representation of a portion of the environment (e.g., the second representation is a transformation of the first representation and / or the second representation includes the removal of at least a portion of the first representation (such as the portion including the user)), and wherein the second representation differs from the first representation (e.g., includes content different from the first representation) (e.g., is not an encoded version of the first representation) (e.g., as described above relative to...). Figure 3B and Figure 3D As described, the physical environment 330 is a static representation and / or media captured by a camera. In some embodiments, a second representation of a portion of the environment is transmitted to a second computer system as part of a communication session.

[0080] In some embodiments, a first computer system (e.g., 300) communicates with one or more input devices (e.g., a camera, depth sensor, microphone, hardware input mechanism, rotatable input mechanism, heart monitor, temperature sensor, and / or touch-sensitive surface). In some embodiments, before connecting to a second computer system via a communication session, the first computer system detects input (e.g., 305a) corresponding to a request for connection (e.g., cellular call, Wi-Fi call, video conferencing, audio call, social media audio call, and / or video call) between the first computer system (e.g., 300) and the second computer system (e.g., establishing a communication session between the first and second computer systems) via one or more input devices (e.g., tap input and / or non-tap input (e.g., speech input, audible request, audible command, audible statement, swipe input, press and drag input, gaze input, air gesture, mouse movement, and / or mouse click)), wherein physical movement of one or more cameras occurs in response to detecting input corresponding to a connection request (e.g., as described above relative to...). Figure 3A (As described). In some implementations, a connection request is a request to call a user corresponding to a second computer system.

[0081] In some implementations, before connecting to the second computer system via a communication session, the first computer system detects (and / or receives) a connection request (e.g., 300) between the first computer system and the second computer system (e.g., establishing a communication session between the first and second computer systems) (e.g., 305a), wherein physically moving one or more cameras occurs in response to the detection of the connection request (e.g., as described above relative to...). Figure 3B (As described).

[0082] In some implementations, physically moving one or more cameras includes translating, tilting, rotating, or any combination thereof (e.g., 1 degree to 360 degrees) via a moving component (e.g., relative to the above). Figure 3B (As described).

[0083] In some implementations, physically moving one or more cameras includes changing the position of a first computer system (e.g., 300) from a first position (e.g., coordinates) to a second position different from the first position via a moving component (e.g., in cases where the orientation of the first computer system is changed or not changed) (e.g., as described above relative to...). Figure 3B (As described).

[0084] In some implementations, physically moving one or more cameras includes: based on determining that the environment (e.g., 330) is in a first state (e.g., including a set of one or more features (e.g., objects and / or users) and / or a set of one or more features performing one or more specific actions), and the computer system cannot determine parts of the environment (e.g., gaps in the environment (e.g., due to obstructions blocking a part of the environment)), physically moving one or more cameras in a first direction (e.g., along a rotation axis (e.g., clockwise, counterclockwise, forward, backward, left, and / or right)) (e.g., as described above relative to...). Figure 3B As described above); and physically moving one or more cameras in a second direction (and / or not moving them in the first direction) in a second direction different from the first direction, depending on the determined environment (e.g., 330) being in a second state different from the first state (e.g., relative to the first direction). Figure 3B (As described).

[0085] In some implementations, the first representation includes a portion not included in the second representation of the environment (e.g., non-static, real-time, and / or foreground portions of the environment (e.g., users of the first computer system and / or animals in the environment)) (e.g., as described above relative to...). Figure 3D As described, the physical environment 330 is a static representation and / or media captured by a camera. In some embodiments, after capturing a first representation of a portion of the environment, the first computer system generates a second representation of the portion of the environment. In some embodiments, generating a second representation of a portion of the environment includes removing portions not included in the second representation of the environment (and / or omitting portions not included in the second representation of the environment).

[0086] In some embodiments, one or more cameras have (and / or capture) a first field of view. In some embodiments, a first representation of a portion of the environment has (and / or captures) a field of view larger than the first field of view (e.g., the first representation includes media captured by one or more cameras at different locations (e.g., while physically moving the cameras), such that the first representation includes more of the environment that can be captured by one or more cameras without physically moving the cameras) (e.g., as described above relative to...). Figure 3B (As described). In some implementations, the first representation is a combination of multiple media segments (e.g., spliced ​​together and / or otherwise combined) to cover a larger area than one or more cameras can capture without moving.

[0087] In some implementations, in conjunction with transmitting a second representation of a portion of the environment to a second computer system via a communication session (e.g., before, after, in response to, and / or during the transmission), the first computer system transmits a third representation of a portion of the environment via the communication session that differs from the first and second representations (e.g., non-static, real-time, and / or foreground portions of the environment (e.g., the user of the first computer system and / or animals in the environment)). Figure 3D As described, the physical environment 330 is a static representation and / or media captured by a camera. In some embodiments, the first representation includes the third representation. In some embodiments, the second representation does not include the third representation. In some embodiments, the third representation is removed from the first representation to generate the second representation. In some embodiments, a third representation of a portion of the environment is captured via one or more cameras in conjunction with the second representation of a portion of the environment transmitted to a second computer system via a communication session (e.g., before, after, in response to, and / or during the transmission).

[0088] In some implementations, a second representation of a portion of the environment is transmitted via a first data transmission method (e.g., a communication channel, protocol (e.g., Bluetooth and / or Wi-Fi) and / or bit rate). In some implementations, a third representation of a portion of the environment is transmitted via a second data transmission method different from the first data transmission method (e.g., as described above relative to...). Figure 3D (As described).

[0089] It should be noted that the above text regarding method 400 (for example, Figure 4 The details of the process described herein also apply in a similar manner to the other methods described herein. For example, method 600 may optionally include one or more features of the various methods described above with reference to method 400. For example, the orientation of the input device toward the user in method 600 may occur during the transmission of the second representation of method 400. For the sake of brevity, these details will not be repeated below.

[0090] Figures 5A to 5E Exemplary user interfaces for capturing content while in motion using a computer system, according to some embodiments, are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 6 The process in.

[0091] like Figures 5A to 5EAs illustrated on the right, physical environment 530 includes different locations of computer system 300. Physical environment 530 includes device representation 532 (which represents the location of computer system 300 within physical environment 530), user representation 534 (which represents the location of user 510 within physical environment 530), and object representation 536 (which represents the location of balloon 512 within physical environment 530). Figures 5A to 5E As illustrated, the physical environment 530 also includes a field of view 538, which represents the orientation of the computer system 300 within the physical environment 530. In some embodiments, the field of view 538 represents the field of view of a camera of the computer system 300.

[0092] In some implementations, computer system 300 translates, tilts, moves, and / or rotates a camera (and / or a portion of computer system 300) to capture a portion and / or all of the physical environment 530. For example, computer system 300 can move a field of view 538 by activating components of computer system 300, such as actuators that are part of computer system 300, where movement of the actuators alters and / or displaces the field of view 538. In such examples, the actuators can move a portion of computer system 300, including the camera, to different orientations and / or directions, thereby altering the field of view 538. As another example, computer system 300 can move the field of view 538 by activating components remote from and / or separate from computer system 300, such as actuators physically and / or magnetically coupled to a mount of computer system 300. In such examples, computer system 300 can communicate with the mount and control the positioning of the mount by transmitting control messages to the mount. It should be recognized that, in addition to or in place of the techniques described above, other techniques known to those skilled in the art may be used to achieve the movement of the field of view 538.

[0093] like Figures 5A to 5E As illustrated on the left, computer system 300 displays user interface 502. Figures 5A to 5E At this point, user interface 502 is the user interface displayed by computer system 300 when computer system 300 is recording video. Figures 5A to 5E At this point, computer system 300 is recording a video commentary of a balloon. In some embodiments, user interface 502 is displayed in response to computer system 300 detecting a request to capture media. It should be understood that user interface 502 is merely one example of a user interface used with the techniques described herein, and other types of user interfaces may also be used. For example, although the following example describes video recording, similar embodiments can be used to take photographs.

[0094] like Figures 5A to 5EAs illustrated, user interface 502 includes a live view 504 positioned at the center of user interface 502. In some embodiments, the live view 504 is displayed across the entire user interface 502. In some embodiments, the live view 504 is displayed at the bottom or top of user interface 502. In some embodiments, the live view 504 is a representation (e.g., an image or video) of the physical environment 530 captured by a camera communicating with computer system 300. Figures 5A to 5E As illustrated, this representation includes user 510.

[0095] like Figures 5A to 5E As illustrated, computer system 300 is pointing the camera toward the subject (e.g., Figures 5C to 5D balloon 512 or Figures 5A to 5B and Figure 5E (User 510 in the middle). Figures 5A to 5E Orienting the camera towards a subject includes following the subject. For example, the computer system 300 orients the camera towards the subject by guiding the camera to move to the subject and maintaining a threshold distance from the subject. In some embodiments, orienting the camera towards the subject includes moving towards the subject. For example, the computer system 300 guides the camera to a location near the subject. In another example, the computer system 300 guides the camera to move in the direction of the subject's current location. In some embodiments, orienting towards the subject includes the computer system 300 moving the camera towards a location near the subject. For example, the location of the subject could be a location adjacent to the subject.

[0096] In some embodiments, orienting the camera toward the subject includes maintaining a continuous view of the subject. For example, computer system 300 keeps the subject within the camera's field of view 538 by guiding the camera to a corresponding position. In some embodiments, the subject becomes obstructed from the camera's field of view, such as by an obstacle, when orienting the camera toward it. In such embodiments, computer system 300 may guide the camera to move around the obstacle to maintain and / or restore the subject in the live view 504.

[0097] In some implementations, computer system 300 includes components that enable computer system 300 to physically move within physical environment 530 (e.g., as described above). Figure 3A (Components described herein). For example, computer system 300 may include components that enable computer system 300 to move and / or enable the camera of computer system 300 to move.

[0098] like Figure 5A As illustrated, user representation 534 is within field of view 538 of physical environment 330, and object representation 536 is outside field of view 538 of physical environment 330. Figure 5AAt this point, device 532 indicates that the computer system 300's camera is being pointed towards the user 534. In some embodiments, the computer system 300 is pointing the camera towards the user 510 in response to the computer system 300 detecting input indicating that the camera is being pointed towards the user 510. For example, the user 510 may provide voice input (e.g., "Please follow me"). In another example, the user 510 provides a gesture (e.g., waving) to point the computer system 300's camera towards the user 510. In yet another example, the user 510 provides a tap input on user interface 502 to point the computer system 300's camera towards the user 510. Figure 5A At this point, user 534 is moving left toward object 536. In some embodiments, computer system 300 identifies user 510 as the subject the camera is facing. In some embodiments, computer system 300 identifies user 510 as the subject by determining that the audio input originates from user 510. In some embodiments, computer system 300 identifies user 510 as the subject by determining that user 510 is looking toward computer system 300 (e.g., the focus direction of user 510).

[0099] like Figure 5A As illustrated, when computer system 300 points the camera at user 510, computer system 300 captures media from user 510 and simultaneously displays that media (e.g., in live view 504). Figure 5A As illustrated, user 510 is located at the center of live view 504. In some embodiments, when computer system 300 points the camera towards user 510, computer system 300 changes the framing and / or viewing angle of user 510 in live view 504. In some embodiments, computer system 300 changes the user's framing and / or viewing angle to zoom out and / or display user 510 to the right of the framing to illustrate the movement. Figure 5A Since user 510 is moving to the left, computer system 300 displays the corresponding video of user 510, which is being captured by the camera of computer system 300 and is moving to the left, on live view 504.

[0100] exist Figure 5B In response to user 510 moving to the left, computer system 300 continues to point the camera at user 510 and moves with user 510. For example... Figure 5B As illustrated, user representation 534 remains within field of view 538 in physical environment 330, and object representation 536 is within field of view 538.

[0101] exist Figure 5BIn response to user 510 moving to the left, computer system 300 moves the camera toward balloon 512 along with user 510. In some implementations, computer system 300 maintains the composition of user 510 in live view 504. For example, in Figure 5A In this context, user 510 is positioned at the center of live view 504. In some embodiments, computer system 300 guides the camera to maintain framing by moving it with user 510 to keep it centered in live view 504. In some embodiments, maintaining camera framing includes maintaining video framing. For example, if computer system 300 detects that the framing is an establishment shot, computer system 300 positions the camera to maintain an establishment shot framing. In another example, if computer system 300 detects that the framing is a close-up shot, computer system 300 positions the camera to maintain a close-up shot framing.

[0102] exist Figure 5B At this point, computer system 300 determines that user 510 intends to change the subject from user 510 to balloon 512. In some embodiments, such determination is performed based on detecting user 510's gaze toward balloon 512. For example, computer system 300 may detect user 510 looking at balloon 512 for a threshold time period (e.g., 1 second to 10 seconds). As another example, computer system 300 may detect a threshold number of times user 510 looks toward balloon 512 (e.g., 2 times or more). As yet another example, computer system 300 may detect that user 510's gaze on balloon 512 meets a threshold intensity, such as by opening eyelids more and / or moving user 510's head toward balloon 512 while gazing at balloon 512. In other embodiments, such determination is performed based on detecting touch input from user 510 while user 510's gaze is toward balloon 512. In other embodiments, such determination is performed based on detecting voice input referring to balloon 512. Figure 5B In response to determining that user 510 intends to change the subject from user 510 to balloon 512, computer system 300 stops pointing the camera at user 510 and points the camera at balloon 512.

[0103] exist Figure 5C In response to the computer system pointing the camera toward balloon 512, the computer system 300 guides the camera to move so that balloon 512 is centered in the real-time view 504. Figure 5C When the computer system 300 points the camera toward the balloon 512, the computer system 300 (e.g., as represented by device representation 532) moves the camera to the left so that the field of view 538 is centered on object representation 536 rather than user representation 534.

[0104] exist Figure 5CIn response to the computer system 300 guiding the camera to move toward the balloon 512, the computer system 300 updates the user interface 502 to display an updated view centered on the balloon 512, with the user 510 facing the right side of the user interface 502. In some embodiments, the computer system 300 guides the camera to maintain composition by guiding its movement to center itself on the balloon 512 in the live view 504. For example, in Figure 5C At this point, the computer system 300 guides the camera to maintain the zoom level and relative size of the balloon 512 so that it is the same size as the user 510 when the computer system 300 detects that the user 510 is looking at the balloon 512. Figure 5C Then, user 510 moved to the left of balloon 512.

[0105] exist Figure 5D In response to user 510 moving to the left of balloon 512, computer system 300 maintains the camera facing balloon 512. Figure 5D Because the computer system 300 is pointing the camera toward the balloon 512, the computer system 300 (e.g., as indicated by the device representation 532) does not guide the camera to move when the user representation 534 moves within the physical environment 530.

[0106] exist Figure 5D In response to the computer system 300 pointing the camera toward the balloon 512, the computer system 300 guides the camera to remain in place and positions the balloon 512 at the center of the field of view 538. Figure 5D As illustrated, computer system 300 displays user 510 on the left side of user interface 502, thereby reflecting the updated location of user 510 in physical environment 530.

[0107] like Figure 5D As illustrated, user 510 no longer focuses on balloon 512 in the physical environment 530. Figure 5D At this point, computer system 300 detects that user 510 no longer intends to shift the focus from user 510 to balloon 512 by detecting that user 510's gaze is no longer directed towards balloon 512. In some embodiments, this detection occurs when user 510 removes their gaze from balloon 512 for a threshold time period (e.g., 1 second to 10 seconds). In some embodiments, this detection occurs when computer system 300 detects multiple gazes not directed towards balloon 512. In some embodiments, this detection occurs when user 510's gaze on balloon 512 no longer meets a threshold intensity. Figure 5D In response to detecting that the user 510's gaze is no longer directed toward the balloon 512, the computer system 300 stops pointing the camera toward the balloon 512 and points the camera toward the user 510.

[0108] In some implementations, computer system 300 stops pointing the camera at balloon 512 in response to detecting verbal input (e.g., "follow me") from user 510. In some implementations, computer system 300 stops pointing the camera at balloon 512 in response to detecting tap input on user interface 502.

[0109] exist Figure 5E In response to detecting that user 510's gaze is no longer directed at balloon 512, computer system 300 turns the camera toward user 510. Figure 5E Located on the right, in physical environment 530, computer system 300 moves the camera toward user representation 534. Figure 5E On the left, user 510 is located in the center of live view 504.

[0110] In some implementation schemes, Figure 5E An alternative implementation is illustrated in which user 510's gaze is not directed toward balloon 512 (e.g., user 510 looks at balloon 512 for less than a threshold time period), and computer system 300 determines that the user does not intend to change the subject from user 510 to balloon 512. In some implementations, in Figure 5B and Figure 5E Between these events, user 510 moved to the left of balloon 512. Figure 5E In response to the computer system 300 detecting that the user 510's gaze is not directed towards the balloon 512, the computer system 300 continues to point the camera towards the user 510. Figure 5E Being on the right, because user 510 moved to the left of balloon 612, user representation 534 moved to the left of object representation 536. Figure 5E Because the computer system 300 directs the camera toward the user 510, the device representation 532 moves to keep the user representation 534 centered in the field of view 538. Figure 5E Located on the left, in response to the computer system 300 detecting that the user 510's gaze is not directed toward the balloon 512 and the user 510 moves to the left, the computer system 300 displays the user 510 in the center of the user interface 502.

[0111] Figure 6 This is a flowchart illustrating a method (e.g., method 600) for capturing content during movement, according to some implementation schemes. Some operations in method 600 may be optionally combined, the order of some operations may be optionally changed, and some operations may be optionally omitted.

[0112] As described below, Method 600 provides an intuitive way to capture content while on the move. Method 600 reduces the cognitive load on the user, thereby creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to interact with such devices faster and more efficiently, saving power and increasing the time between battery charging.

[0113] In some embodiments, method 600 is performed at a computer system (e.g., 300) that communicates (e.g., with) moving components (e.g., actuators (e.g., pneumatic actuators, hydraulic actuators, and / or electric actuators), movable bases, rotatable components, motors, lifts, levels, and / or rotatable bases) and one or more input devices (e.g., cameras, depth sensors, microphones, hardware input mechanisms, rotatable input mechanisms, heart monitors, temperature sensors, and / or touch-sensitive surfaces) via wired and / or wireless communication. In some embodiments, the computer system is a watch, telephone, tablet, fitness tracker, processor, head-mounted display (HMD) device, public utility, media device, speaker, television, and / or personal computing device. In some embodiments, the computer system communicates with a mounting base. In some embodiments, the computer system is (e.g., physically and / or magnetically) coupled to the mounting base. In some embodiments, the mounting base includes one or more input devices. In some embodiments, the mounting base includes moving components.

[0114] When (602) media content (e.g., one or more images, videos, and / or HMD media) is captured via one or more input devices (e.g., 504), the computer system (e.g., 300) directs one or more input devices toward (604) via a moving component (e.g., as described above relative to...). Figures 5A to 5E As described, computer system 300 directs the camera toward (e.g., automatically and / or in response to a user's request (e.g., an explicit request toward the computer system and / or user input)) a user (e.g., 510) (e.g., the user of the computer system and / or a different user captured by one or more input devices) (e.g., moving toward and / or following the user). In some embodiments, directing one or more input devices toward the user includes transmitting one or more instructions to the mount to physically move toward the user via a moving component (e.g., panning, tilting, rotating, and / or changing physical positioning). In some embodiments, the computer system initiates the capture of media content in response to the computer system detecting a request to capture media content via one or more input devices (e.g., an explicit request toward the computer system and / or user input) (e.g., pointing toward the computer system).

[0115] When capturing media content via one or more input devices (602), while the one or more input devices are oriented toward a user (e.g., 510), the computer system detects (606) the user's (e.g., 510) gaze toward a location (e.g., the location the user is looking at and / or looking toward in a physical and / or virtual environment) via the one or more input devices (e.g., as described above relative to...). Figure 5B (As described).

[0116] While capturing media content via one or more input devices at (602), in response to (608) detecting a user's (e.g., 510) gaze at the location, based on determining that the user's (e.g., 510) gaze (and / or at the location) does not meet one or more criteria (e.g., gaze intensity, number of gazes, gaze detected using additional input (e.g., speech, touch, air gesture, tap and / or click), and / or gaze location exceeds a threshold time period), the computer system continues at (610) to move one or more input devices toward the user via a moving component (e.g., as described above relative to the location). Figure 5E (As described).

[0117] When capturing media content via one or more input devices at (602), in response to detecting a user's gaze at the location at (608), based on determining that the user's (e.g., 510) gaze (and / or the location) meets one or more criteria, the computer system orients one or more input devices toward (612) the location (and / or stops orienting one or more input devices toward the user) (e.g., as described above relative to...). Figures 5C to 5D (As described). In some embodiments, orienting one or more input devices toward the location includes transmitting one or more commands to the mount to physically move (e.g., translate, tilt, rotate, and / or change physical positioning) toward the location via a moving component.

[0118] In some implementations, orienting one or more input devices toward a user (e.g., 510) includes: physically moving (e.g., translating, tilting, rotating, and / or changing the physical position) one or more input devices via a moving component (and / or transmitting one or more commands to the mount to physically move them) (and / or moving them to a position corresponding to the user (e.g., within a predetermined and / or automatic distance from the user, and / or in a predetermined and / or automatic orientation of the user (e.g., translating, tilting, and / or rotating))) (e.g., as described above relative to... Figures 5A to 5E(As described). In some embodiments, when one or more input devices are oriented toward the user and based on the determination that the user is currently moving, the computer system moves one or more input devices toward the user and / or maintains the user within the detection field (e.g., field of view or audible area) of one or more input devices. In some embodiments, when one or more input devices are oriented toward the user and based on the determination that the user is currently moving, the computer system transmits one or more instructions to the mounting base to move one or more input devices toward the user and / or maintain the user within the detection field (e.g., field of view or audible area) of one or more input devices. In some embodiments, when one or more input devices are oriented toward the user and based on the determination that the user is not currently moving, the computer system abandons moving one or more input devices. In some embodiments, when one or more input devices are oriented toward the user and based on the determination that the user is not currently moving, the computer system abandons transmitting one or more instructions to the mounting base to move one or more input devices.

[0119] In some implementations, physically moving one or more input devices includes: physically moving one or more input devices (e.g., translating, tilting, rotating, and / or changing their physical position) via a moving component (and / or sending instructions to the mount to physically move them) to a position that is not obstructed by the object and that allows the one or more input devices to face the user (e.g., as described above relative to the user) based on an object (e.g., in a physical and / or virtual environment) that prevents the one or more input devices from facing the user (e.g., as described above relative to the user). Figure 5D As described above); and based on the determination that the object does not prevent one or more input devices from facing the user (e.g., 510), abandoning (e.g., towards that location) physical movement (and / or abandoning the transmission of one or more instructions to the mount to physically move) of one or more input devices (and / or continuing to face one or more input devices towards the user) (e.g., as described above relative to the user). Figure 5D (As described).

[0120] In some implementations, orienting one or more input devices toward the user (e.g., 510) includes maintaining the user's continuous view (e.g., via one or more input devices and / or captured by one or more input devices) (e.g., as described above relative to...). Figures 5A to 5E (As described). In some implementations, maintaining a user's continuous view includes: the computer system capturing the user within a portion of the field of view of one or more input devices while changing the orientation of the computer system relative to the user (e.g., tilt, translation, rotation, and / or positioning).

[0121] In some implementations, continuing to orient one or more input devices toward the user (e.g., 510) includes: based on determining that the user (e.g., 510) is in a first state (e.g., performing an action (e.g., gesture and / or movement) and / or providing input corresponding to an object and / or location), changing the user's view (e.g., framing and / or perspective captured via one or more input devices) via a motion component (e.g., as described above relative to...). Figure 5A As described above); and in some embodiments, changing the user's view includes: the computer system moving via a moving component from a first location (e.g., a specific location and / or orientation) to a second location, the first location capturing the user in a first portion of the field of view of one or more input devices, the second location capturing the user in a second portion of the field of view of one or more input devices, the second portion being different from the first portion of the field of view of one or more input devices. Based on determining that the user (e.g., 510) is in a second state different from the first state, the change of the user's view (e.g., as described above relative to...) is abandoned. Figure 5A (As described).

[0122] In some embodiments, orienting one or more input devices toward the location includes: physically moving (e.g., translating, tilting, rotating, and / or changing the physical positioning) one or more input devices via a moving component (and / or transmitting instructions to the mount to move) such that the one or more input devices reach a positioning (e.g., a specific position and / or orientation) corresponding to the location (e.g., reaching a positioning corresponding to the location (e.g., within a predetermined and / or automatic distance from the location, and / or in a predetermined and / or automatic orientation (e.g., translating, tilting, rotating) relative to the location)) (e.g., orienting toward the location) (e.g., as described above relative to the location). Figures 5A to 5E (As described).

[0123] In some implementations, the location corresponds to an object (e.g., a physical object, an object captured in the field of view of one or more input devices) in the physical environment (e.g., 530) of the computer system (e.g., 300).

[0124] In some implementations, after one or more input devices are oriented toward the location, the computer system detects (e.g., via one or more input devices and / or by receiving notification of events from another computer system different from the computer system) events (e.g., input (e.g., tap input and / or non-tap input (e.g., verbal input, audible request, audible command, audible statement, swipe input, press and drag input, gaze input, air gesture, mouse movement and / or mouse click)), requests, expiration of time thresholds and / or the current time (e.g., corresponding to the user and / or a location different from that location) (e.g., as described above relative to the location). Figure 5E (As described above). In some implementations, in response to detecting an event, the computer system directs one or more input devices (and / or transmits one or more instructions to the mount for physical movement) to the user via a moving component (e.g., 510) (e.g., as described above relative to...). Figure 5E (As described).

[0125] It should be noted that the above text regarding method 600 (for example, Figure 6 The details of the process described herein also apply in a similar manner to the methods described herein. For example, method 400 may optionally include one or more features of the various methods described herein with reference to method 600. For example, a second representation of the transmission environment of method 400 may occur during the capture of media content in method 600. For the sake of brevity, these details will not be repeated below.

[0126] For purposes of explanation, the foregoing description has been presented with reference to specific examples. However, the illustrative discussion above is not intended to be exhaustive or to limit this disclosure to the precise form disclosed. Many modifications and variations are possible based on the teachings above. These examples were chosen and described in order to best explain the principles of these techniques and their practical applications. Others skilled in the art will thus be able to best utilize these techniques and the various examples with modifications suitable for the specific intended use.

[0127] While this disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of this disclosure and examples as defined by the claims.

[0128] As described above, one aspect of the present invention is the collection and use of data available from various sources to improve content capture. This disclosure contemplates that, in some instances, the collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. Such personal information may include photographs, videos, demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifying information.

[0129] This disclosure recognizes that the use of such personal information data in the techniques of this invention can benefit users. For example, personal information data can be used to capture user content and / or capture content for users. Therefore, the use of such personal information data enables better content capture. Furthermore, this disclosure also anticipates other uses of personal information data that benefit users.

[0130] This disclosure also envisions that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and measures that are recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. For example, personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate purposes. Furthermore, such collection should only be conducted after receiving informed consent from users. Additionally, such entities should take any necessary steps to safeguard and protect access to such personal information data and ensure that others with access to such personal information data comply with their privacy policies and procedures. Moreover, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and privacy practices.

[0131] Regardless of the foregoing, this disclosure also contemplates implementation schemes for users to selectively block the use or access to personal information data. That is, this disclosure contemplates providing hardware and / or software components to prevent or block access to such personal information data. For example, with regard to image capture, the inventive technology can be configured to allow users to choose to "join" or "opt out" from participating in the collection of personal information data during service registration.

[0132] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not become inoperable due to the absence of all or part of such personal information data.

Claims

1. A method, the method comprising: At the first computer system communicating with the moving component and one or more cameras: Connect to a second computer system different from the first computer system via a communication session; Connecting to the second computer system via the communication session: The camera in one or more cameras is physically moved via the moving component; as well as While the camera is physically moved, a first representation of a portion of the environment is captured via the one or more cameras; as well as After capturing the first representation of the portion of the environment, a second representation of the portion of the environment is transmitted to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

2. The method of claim 1, wherein the first computer system communicates with one or more input devices, the method further comprising: Before connecting to the second computer system via the communication session, input corresponding to a request to connect the first computer system and the second computer system is detected via the one or more input devices, wherein the physical movement of one or more cameras occurs in response to the detection of the input corresponding to the connection request.

3. The method according to any one of claims 1 to 2, further comprising: Before connecting to the second computer system via the communication session, a request to connect the first computer system and the second computer system is detected, wherein the physical movement of one or more cameras occurs in response to the detection of the connection request.

4. The method according to any one of claims 1 to 3, wherein physically moving one of the one or more cameras includes translating, tilting, rotating, or any combination thereof via the moving component.

5. The method according to any one of claims 1 to 4, wherein physically moving one of the one or more cameras includes changing the position of the first computer system from a first position to a second position, the second position being different from the first position, via the moving component.

6. The method according to any one of claims 1 to 5, wherein physically moving one or more of the cameras comprises: Based on determining that the environment is in a first state, the camera in one or more cameras is physically moved in a first direction; as well as Based on determining that the environment is in a second state different from the first state, the camera among the one or more cameras is physically moved in a second direction different from the first direction.

7. The method according to any one of claims 1 to 6, wherein the first representation includes a portion of the environment not included in the second representation.

8. The method according to any one of claims 1 to 7, wherein the one or more cameras have a first field of view, and wherein the first representation of the portion of the environment has a larger field of view than the first field of view.

9. The method according to any one of claims 1 to 8, further comprising: The second representation of the environment is transmitted to the second computer system via the communication session, and a third representation of the environment is transmitted via the communication session, the third representation being different from the first and second representations.

10. The method of claim 9, wherein the second representation of the portion of the environment is transmitted by a first data transmission method, and wherein the third representation of the portion of the environment is transmitted by a second data transmission method, the second data transmission method being different from the first data transmission method.

11. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a first computer system in communication with a mobile component and one or more cameras, the one or more programs including instructions for performing the method according to any one of claims 1 to 10.

12. A first computer system configured to communicate with a mobile component and one or more cameras, the first computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 10.

13. A first computer system configured to communicate with a mobile component and one or more cameras, the first computer system comprising: Components for performing the method according to any one of claims 1 to 10.

14. A computer program product comprising one or more programs configured to be executed by one or more processors of a first computer system in communication with a mobile component and one or more cameras, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 10.

15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first computer system in communication with a mobile component and one or more cameras, the one or more programs including instructions for: Connect to a second computer system different from the first computer system via a communication session; Connecting to the second computer system via the communication session: The camera in one or more cameras is physically moved via the moving component; as well as While the camera is physically moved, a first representation of a portion of the environment is captured via the one or more cameras; as well as After capturing the first representation of the portion of the environment, a second representation of the portion of the environment is transmitted to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

16. A first computer system configured to communicate with a mobile component and one or more cameras, the first computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: Connect to a second computer system different from the first computer system via a communication session; Connecting to the second computer system via the communication session: The camera in one or more cameras is physically moved via the moving component; as well as While the camera is physically moved, a first representation of a portion of the environment is captured via the one or more cameras; as well as After capturing the first representation of the portion of the environment, a second representation of the portion of the environment is transmitted to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

17. A first computer system configured to communicate with a mobile component and one or more cameras, the first computer system comprising: Components for connecting to a second computer system different from the first computer system via a communication session; Connecting to the second computer system via the communication session: A component for physically moving one or more cameras via the moving component; as well as A component for capturing a first representation of a portion of the environment via the one or more cameras while the cameras are physically moved; as well as A component for transmitting a second representation of a portion of the environment to the second computer system via the communication session after capturing a first representation of that portion of the environment, wherein the second representation is based on the first representation of that portion of the environment, and wherein the second representation is different from the first representation.

18. A computer program product comprising one or more programs configured to be executed by one or more processors of a first computer system in communication with a mobile component and one or more cameras, the one or more programs comprising instructions for performing the following operations: Connect to a second computer system different from the first computer system via a communication session; Connecting to the second computer system via the communication session: The camera in one or more cameras is physically moved via the moving component; as well as While the camera is physically moved, a first representation of a portion of the environment is captured via the one or more cameras; as well as After capturing the first representation of the portion of the environment, a second representation of the portion of the environment is transmitted to the second computer system via the communication session, wherein the second representation is based on the first representation of the portion of the environment, and wherein the second representation is different from the first representation.

19. A method comprising: At a computer system that communicates with mobile components and one or more input devices: When capturing media content via the one or more input devices: Orienting the one or more input devices toward the user via the moving component; When the one or more input devices are oriented toward the user, the user's gaze toward the location is detected via the one or more input devices; as well as In response to detecting the user's gaze at the location: Based on the determination that the user's gaze does not meet a set of one or more criteria, the one or more input devices continue to be directed toward the user via the mobile component; as well as The one or more input devices are oriented toward the location based on the set of criteria that determine that the user's gaze meets one or more standards.

20. The method of claim 19, wherein moving the one or more input devices toward the user includes physically moving the one or more input devices via the moving component.

21. The method according to any one of claims 19 to 20, wherein physically moving the one or more input devices comprises: Based on the determination that an object obstructs the one or more input devices from facing the user, the one or more input devices are physically moved via the moving component to a position where they are not obstructed by the object and are facing the user. as well as If the determined object does not prevent the one or more input devices from facing the user, then physical movement of the one or more input devices is abandoned.

22. The method according to any one of claims 19 to 21, wherein oriented the one or more input devices toward the user comprises: Maintain the user's continuous view.

23. The method according to any one of claims 19 to 22, wherein continuing to orient the one or more input devices toward the user comprises: Based on determining that the user is in a first state, the view of the user is changed via the mobile component; as well as Based on the determination that the user is in a second state different from the first state, the view of the user is not changed.

24. The method of any one of claims 19 to 23, wherein orienting the one or more input devices toward the location comprises physically moving the one or more input devices to a position corresponding to the location via the moving component.

25. The method according to any one of claims 19 to 24, wherein the location corresponds to an object in the physical environment of the computer system.

26. The method according to any one of claims 19 to 25, the method further comprising: An event is detected after the one or more input devices are oriented toward the position; as well as In response to the detection of the event, the one or more input devices are directed toward the user via the mobile component.

27. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a mobile component and one or more input devices, the one or more programs including instructions for performing the method according to any one of claims 19 to 26.

28. A computer system configured to communicate with a mobile component and one or more input devices, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 19 to 26.

29. A computer system configured to communicate with a mobile component and one or more input devices, the computer system comprising: Components for performing the method according to any one of claims 19 to 26.

30. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a mobile component and one or more input devices, the one or more programs comprising instructions for performing the method according to any one of claims 19 to 26.

31. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a mobile component and one or more input devices, the one or more programs including instructions for performing the following operations: When capturing media content via the one or more input devices: Orienting the one or more input devices toward the user via the moving component; When the one or more input devices are oriented toward the user, the user's gaze toward the location is detected via the one or more input devices; as well as In response to detecting the user's gaze at the location: Based on the determination that the user's gaze does not meet a set of one or more criteria, the one or more input devices continue to be directed toward the user via the mobile component; as well as The one or more input devices are oriented toward the location based on the set of criteria that determine that the user's gaze meets one or more standards.

32. A computer system configured to communicate with a mobile component and one or more input devices, the computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: When capturing media content via the one or more input devices: Orienting the one or more input devices toward the user via the moving component; When the one or more input devices are oriented toward the user, the user's gaze toward the location is detected via the one or more input devices; as well as In response to detecting the user's gaze at the location: Based on the determination that the user's gaze does not meet a set of one or more criteria, the one or more input devices continue to be directed toward the user via the mobile component; as well as The one or more input devices are oriented toward the location based on the set of criteria that determine that the user's gaze meets one or more standards.

33. A computer system configured to communicate with a mobile component and one or more input devices, the computer system comprising: When capturing media content via the one or more input devices: A component for orienting the one or more input devices toward the user via the moving component; A component for detecting the user's gaze toward a location via the one or more input devices when the one or more input devices are oriented toward the user; as well as In response to detecting the user's gaze at the location: A component for continuing to orient the one or more input devices toward the user via the moving component based on a set of one or more criteria determined by the user's gaze; as well as A component for orienting one or more input devices toward the location based on a set of criteria that determine that the user's gaze meets one or more standards.

34. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a mobile component and one or more input devices, the one or more programs comprising instructions for performing the following operations: When capturing media content via the one or more input devices: Orienting the one or more input devices toward the user via the moving component; When the one or more input devices are oriented toward the user, the user's gaze toward the location is detected via the one or more input devices; as well as In response to detecting the user's gaze at the location: Based on the determination that the user's gaze does not meet a set of one or more criteria, the one or more input devices continue to be directed toward the user via the mobile component; as well as The one or more input devices are oriented toward the location based on the set of criteria that determine that the user's gaze meets one or more standards.

Images