Techniques for coordinating electronic device activities

Abstract

The present disclosure generally relates to coordinating device activities amongst multiple electronic devices.

Description

TECHNIQUES FOR COORDINATING ELECTRONIC DEVICE ACTIVITIESCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to U.S. Provisional Patent Application Serial No. 63 / 572,721, entitled “TECHNIQUES FOR COORDINATING ELECTRONIC DEVICE ACTIVITIES,” filed April 1, 2024, and to U.S. Provisional Patent Application Serial No. 63 / 540,035, entitled “TECHNIQUES FOR COORDINATING ELECTRONIC DEVICE ACTIVITIES,” filed September 22, 2023, which are hereby incorporated by reference in their entireties for all purposes.FIELD

[0002] The present disclosure relates generally to computer user interfaces, and more specifically to techniques for coordinating device activities amongst multiple electronic devices.BACKGROUND

[0003] Electronic devices often issue notifications when events occur. Such notifications can indicate when operations are performed and / or input is detected.SUMMARY

[0004] Current techniques for coordinating devices amongst multiple devices are sub- optimal in that they may involve complex processes, for example requiring many keystrokes or other user input. This disclosure provides more effective and / or efficient techniques for coordinating between devices using various devices including examples of a smart phone and a smart watch. In addition, techniques optionally complement or replace other techniques for coordinating between devices in a multi-device system such as an internet-of-things system.

[0005] In some examples, a method that is performed at a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device is described. In some examples, the method comprises: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria,causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0006] In some examples, a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device is described. In some examples, the one or more programs includes instructions for: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0007] In some examples, a transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device is described. In some examples, the one or more programs includes instructions for: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0008] In some examples, a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device is described. In some examples, the first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device comprises one or more processors and memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programsincludes instructions for: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0009] In some examples, a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device is described. In some examples, the first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device comprises means for performing each of the following steps: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0010] In some examples, a computer program product is described. In some examples, the computer program product comprises one or more programs configured to be executed by one or more processors of a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device. In some examples, the one or more programs include instructions for: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

[0011] In some examples, a method that is performed at a first device that is in communication with one or more input devices is described. In some examples, the method comprises: sending, to a plurality of other devices, a request to perform a first operation,wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0012] In some examples, a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices is described. In some examples, the one or more programs includes instructions for: sending, to a plurality of other devices, a request to perform a first operation, wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0013] In some examples, a transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices is described. In some examples, the one ormore programs includes instructions for: sending, to a plurality of other devices, a request to perform a first operation, wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0014] In some examples, a first device configured to communicate with one or more input devices is described. In some examples, the first device comprises one or more processors and memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs includes instructions for: sending, to a plurality of other devices, a request to perform a first operation, wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0015] In some examples, a first device configured to communicate with one or more input devices is described. In some examples, the first device comprises means for performing each of the following steps: sending, to a plurality of other devices, a request to perform a first operation, wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0016] In some examples, a computer program product is described. In some examples, the computer program product comprises one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices. In some examples, the one or more programs include instructions for: sending, to a plurality of other devices, a request to perform a first operation, wherein the plurality of other devices does not include the first device; in conjunction with sending the request to perform the first operation, capturing, via the one or more input devices, media content of an environment; detecting, via the one or more input devices, a first input corresponding to a request to perform a second operation; in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a first response to the request to perform the first operation, performing the second operation using first data; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response to the request to perform the first operation, performing the second operation using second data different from the first data.

[0017] In some examples, a method that is performed at a first device that is in communication with one or more input devices is described. In some examples, the method comprises: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data, performing a second operation corresponding to the third device.

[0018] In some examples, a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices is described. In some examples, the one or more programs includes instructions for: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data, performing a second operation corresponding to the third device.

[0019] In some examples, a transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices is described. In some examples, the one or more programs includes instructions for: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data, performing a second operation corresponding to the third device.

[0020] In some examples, a first device configured to communicate with one or more input devices is described. In some examples, the first device comprises one or more processors and memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs includes instructions for: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the mediacontent and the second sensor data, performing a second operation corresponding to the third device.

[0021] In some examples, a first device configured to communicate with one or more input devices is described. In some examples, the first device comprises means for performing each of the following steps: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data, performing a second operation corresponding to the third device.

[0022] In some examples, a computer program product is described. In some examples, the computer program product comprises one or more programs configured to be executed by one or more processors of a first device that is in communication with one or more input devices. In some examples, the one or more programs include instructions for: capturing, via the one or more input devices, media content of an environment; sending, to a plurality of other devices, a request for sensor data, wherein the plurality of other devices does not include the first device; after sending the request for sensor data: receiving, from a second device of the plurality of other devices, first sensor data; and receiving, from a third device, different from the second device, of the plurality of other devices, second sensor data; and after capturing the media data, after receiving the first sensor data, and after receiving the second sensor data: in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data, performing a first operation corresponding to the second device; and in accordance with a determination that a second setof one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data, performing a second operation corresponding to the third device.

[0023] One or more computer-readable storage medium can include executable instructions for performing these techniques. One or more computer program product can include executable instructions for performing these techniques. Some examples of such computer-readable storage mediums are non-transitory. Some examples of such computer- readable storage mediums are transitory. Some examples of such computer program products are non-transitory. Some examples of such computer program products are transitory. Such computer-readable storage medium and / or other computer program product are configured for execution by one or more processors.DESCRIPTION OF THE FIGURES

[0024] Embodiments described below in the Detailed Description may reference the following drawings.

[0025] FIG. 1 A is a block diagram illustrating an electronic device in accordance with some examples.

[0026] FIG. IB illustrates an exemplary electronic device in accordance with some examples.

[0027] FIGS. 2A-2B are block diagrams illustrating a scenario where one or more subject characteristics is detected via an ecosystem of electronic devices in accordance with some examples.

[0028] FIGS. 3A-3B are a flow diagram illustrating a method for coordinating devices based on detecting one or more subject characteristics via the ecosystem in accordance with some examples.

[0029] FIGS. 4A-4C are block diagrams illustrating a scenario where an electronic device determines an identity corresponding to a requested operation and then performs the requested operation based on the identity in accordance with some examples.

[0030] FIGS. 5A-5C are block diagrams illustrating a scenario where an electronic device receives a notification corresponding to a specific user and then identifies a device near the specific user to output the notification in accordance with some examples.

[0031] FIG. 6 is a flow diagram illustrating a method for performing an operation based on an identified user in accordance with some examples.

[0032] FIG. 7 is a flow diagram illustrating a method for receiving a notification for a specific user and outputting the notification at a device nearest to the specific user in accordance with some examples.DETAILED DESCRIPTION

[0033] The following detailed description describes one or more different techniques that can be implemented as methods, devices, systems, computer readable mediums, and the like. This detailed description is intended to assist in understanding the scope of the present disclosure. It is not, however, intended to limit the scope of the present disclosure. For example, some of the phrases and terminology used below are for the purpose of providing examples rather than to limit the scope of the present disclosure.

[0034] Turning to FIGS. 1 A-1B, an example electronic device 100 is illustrated. Electronic device 100 is a non-limiting example of an electronic device that can be used to perform functionality described herein. With reference to the block diagram of FIG. 1 A, electronic device 100 includes processor subsystem 110 communicating with memory 120 (e.g., a system memory) and EO interface 130 via interconnect 150 (e.g., a system bus, one or more memory locations, or other communication channel for connecting multiple components of electronic device 100). In addition, I / O interface 130 is communicating with (e.g., wired or wirelessly) EO device 140. In some examples, EO interface 130 is included with I / O device 140 such that the two are a single component. It should be recognized that there can be one or more EO interfaces, with each EO interface communicating with one or more EO devices. In some examples, multiple instances of processor subsystem 110 can be communicating via interconnect 150.

[0035] Electronic device 100 can be any of various types of devices, including, but not limited to, a system on a chip, a server system, a personal electronic device, a smart phone, a smart watch, a wearable device, a tablet, a laptop computer, a fitness tracking device, a head-mounted display (HMD) device, a desktop computer, an accessory (e.g., switch, light, speaker, air conditioner, heater, window cover, fan, lock, media playback device, television, and so forth), a controller, a hub, and / or a sensor. In some examples, a sensor includes one or more hardware components that detect information about a physical environment in proximity of (e.g., surrounding) the sensor. In some examples, a hardware component of a sensor includes a sensing component (e.g., an image sensor or temperature sensor), a transmitting component (e.g., a laser or radio transmitter), and / or a receiving component (e.g., a laser or radio receiver). Examples of sensors include an angle sensor, a breakage sensor such as a glass breakage sensor, a chemical sensor, a contact sensor, a non-contact sensor, a flow sensor, a force sensor, a gas sensor, a humidity or moisture sensor, an image sensor (e.g., a RGB camera and / or an infrared sensor), an inertial measurement unit, a leak sensor, a level sensor, a metal sensor, a microphone, a motion sensor, a particle sensor, a photoelectric sensor (e.g., ambient light and / or solar), a position sensor (e.g., a global positioning system), a precipitation sensor, a pressure sensor, a proximity sensor, a radiation sensor, a range or depth sensor (e.g., RADAR, LiDAR), a speed sensor, a temperature sensor, a time-of-flight sensor, a torque sensor, and an ultrasonic sensor, a vacancy sensor, an voltage and / or current sensor, and / or a water sensor. In some examples, sensor data is captured by fusing data from one sensor with data from one or more other sensors. Although a single electronic device is shown in FIGS. 1A-1B, electronic device 100 can also be implemented as two or more electronic devices operating together.

[0036] In some examples, processor subsystem 110 includes one or more processors or processing units configured to execute program instructions to perform functionality described herein. For example, processor subsystem 110 can execute an operating system and / or one or more applications.

[0037] Memory 120 can include a computer readable medium (e.g., non-transitory or transitory computer readable medium) usable to store (e.g., configured to store, assigned to store, and / or that stores) program instructions executable by processor subsystem 110 to cause electronic device 100 to perform various operations described herein. For example, memory 120 can store program instructions to implement the functionality associated with method 300 described below.

[0038] Memory 120 can be implemented using different physical, non-transitory memory media, such as hard disk storage, optical drive storage, floppy disk storage, removable diskstorage, removable flash drive, storage array, a storage area network (e.g., SAN), flash memory, random access memory (e g., RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, and / or RAMBUS RAM), and / or read only memory (e.g., PROM and / or EEPROM).

[0039] VO interface 130 can be any of various types of interfaces configured to communicate with other devices. In some examples, VO interface 130 includes a bridge chip (e.g., Southbridge) from a front-side bus to one or more back-side buses. VO interface 130 can communicate with one or more VO devices (e.g., VO device 140) via one or more corresponding buses or other interfaces. Examples of VO devices include storage devices (e.g., as described above with respect to memory 120), network interface devices (e.g., to a local or wide-area network), sensor devices (e.g., as described above with respect to sensors), a physical user-interface device (e.g., a physical keyboard, a mouse, and / or a joystick), and an auditory and / or visual output device (e.g., speaker, light, screen, and / or projector). In some examples, the visual output device is referred to as a display generation component. The display generation component is configured to provide visual output, such as display via an LED display or image projection. As used herein, “displaying” content includes causing to display the content (e.g., video data rendered or decoded by a display controller) by transmitting, via a wired or wireless connection, data (e.g., image data or video data) to an integrated or external display generation component to visually produce the content.

[0040] In some examples, VO device 140 includes one or more camera sensors (e.g., one or more optical sensors and / or one or more depth camera sensors), such as for recognizing a subject and / or a subject’s gestures (e.g., hand gestures and / or air gestures) as input. In some examples, an air gesture is a gesture that is detected without the user touching an input element that is part of the device (or independently of an input element that is a part of the device) and is based on detected motion of a portion of the user’s body through the air including motion of the user’s body relative to an absolute reference (e.g., an angle of the user’s arm relative to the ground or a distance of the user’s hand relative to the ground), relative to another portion of the user’s body (e.g., movement of a hand of the user relative to a shoulder of the user, movement of one hand of the user relative to another hand of the user, and / or movement of a finger of the user relative to another finger or portion of a hand of the user), and / or absolute motion of a portion of the user’s body (e.g., a tap gesture that includes movement of a hand in a predetermined pose by a predetermined amount and / or speed, or ashake gesture that includes a predetermined speed or amount of rotation of a portion of the user’s body).

[0041] In some examples, I / O device 140 is integrated with other components of electronic device 100. In some examples, I / O device 140 is separate from other components of electronic device 100. In some examples, I / O device 140 includes a network interface device that permits electronic device 100 to communicate with a network or other electronic devices, in a wired or wireless manner. Exemplary network interface devices include Wi-Fi, Bluetooth, NFC, USB, Thunderbolt, Ethernet, Thread, UWB, and so forth. With reference to FIG. IB, example electronic device 100 can house one or more processors 150, memory 152, input devices 154, sensors 156, and display 158.

[0042] Attention is now directed towards techniques for coordinating an operation to be performed at a first device based on detecting one or more characteristics of a subject (e.g., a user, a person, and / or an animal) via a second device in an environment. In some examples, the second device is or includes one or more sensors, such as one or more cameras, inertial measurement units, antennas, radio frequency receivers and / or transmitters, ambient light sensors, microphones, etc. For example, a wearable device, such as a fitness tracking device and / or smart watch, can be caused to output one or more notifications and / or store data by another device, such as a smart phone and / or a smart hub, that has determined that one or more characteristics of the subject, detected via one or more sensors, meets certain criteria. In some examples, a determination of whether one or more characteristics of the subject meets certain criteria is based on detecting whether the subject is in a particular position, moving more than a certain amount, moving faster than a certain amount, performing a particular activity, etc.

[0043] FIGS. 2A-2B are block diagrams illustrating a scenario where one or more subject characteristics is detected via an ecosystem of electronic devices in accordance with some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 3A-3B.

[0044] FIG. 2A illustrates electronic device 200 on the left, electronic device 216 in the middle, and a schematic of location 206 on the right. Location 206 includes space 208 and space 210. In some examples, Location 206 incudes one or more spaces other than spaces 208 and / or 210. Examples of locations include areas of a building, an office, an event center,a home, an outdoor patio, so forth. Examples of spaces include rooms in such locations or other defined spaces, such as a foyer, an atrium, a closet, so forth.

[0045] At FIG. 2A, person 212 is in space 208 at position 214. No user is illustrated in FIG. 2A as being in space 210. In this example, sensor 222 is located in space 210. In some examples, sensor 222 is physically coupled to electronic device 200. In some examples, sensor 222 is not physically coupled, but rather wirelessly coupled, to electronic device 200. In some examples, sensor 222 is a stand-alone device. In some examples, sensor 222 is integrated with an electronic device with capabilities beyond sensing. While the examples below are described with reference to sensor 222, it should be understood than location 206 can contain one or more other smart devices, such as smart speakers, smart televisions, smart window coverings, and / or smart locks, and / or one or more other sensors, such as inertial measurement units, antennas, radio frequency receivers and / or transmitters, ambient light sensors, and / or microphones. These types of smart devices are sometimes referred to smart due to their ability to connect over a network, and coordinate their activities with, other networked devices. Sensor 222 can have one or more of visual (e.g., camera), aural (e.g., microphone), light (e.g., ambient), and / or motion (e.g., IMU) sensing capabilities.

[0046] As illustrated in FIG. 2A, electronic device 200 includes display 202. In FIG. 2A, electronic device 200 is currently displaying a user interface that includes viewfinder 204. Viewfinder 204 includes a representation of the field-of-view of the camera located in space 210. As illustrated in FIG. 2A, no user is illustrated in viewfinder 204 because no user is currently in space 210. FIG. 2B illustrates user 212 located in space 210 of location 206 at position 220. While person 212 is located in space 210, sensor 222 is capturing images of person 212. FIG. 2B illustrates a captured image of person 212, which is visible via viewfinder 204 of electronic device 200. In some examples, viewfinder 204 is a live feed of sensor 222.

[0047] In some examples, and subject to user privacy considerations as discussed further below, electronic device 200 sends data captured via one or more smart devices, such as sensor 222, of location 206 upon a determination that the activity of person 212 satisfies one or more respective criteria. At FIG. 2B, electronic device 200 determines that person 212 is more than likely working out. Electronic device 200 makes this determination via the one or more images captured by sensor 222. In some examples, to make the determination that person 212 is working out, electronic device 200 detects that user is performing certainmovements, such as jumping, running, stretching, and / or other exercises, in the field-of-view of sensor 222. In some examples, the camera of space 210 makes this determination based on one or more processors in communication with the camera, such as one or more processors that are separate from the processor of electronic device 200. In some examples, one or more portions of images are sent to an electronic device having more processing power than electronic device 200, to make the determination and sends an indication of the determination to electronic device 200. In some examples, the different electronic device is a controller device and / or a smart hub for accessory devices located in location 206.

[0048] At FIG. 2B, in response to electronic device 200 receiving an indication that person 212 is working out, one or more instructions are sent to electronic device 216 to begin recording a workout along with the heart rate of person 212. As illustrated in FIG. 2B, in response to receiving a command to begin recording a workout, notification 218 is displayed, which includes the current heart rate of person 212. In some examples, notification 218 could provide the current heart rate of person 212, such as a message that states the person’s current heart rate is 132 beats per minute. In some examples, notification 218 is displayed, in addition to or in lieu of, a haptic notification (e.g., a vibration and / or shaking of a device) and / or a sound notification (e.g., a ding, a beep, and / or a voice audio output from electronic device 216) being output.

[0049] In some examples, an indication of person 212 working out is sent to electronic device 216. In some examples, data is derived from one or more images captured by sensor 222, such as how many steps the user has taken, how many breaks the user has taken, and / or how many repetitions (e.g., of jumping jacks) the user has performed. In accordance with user permissions, the data derived from the one or more images may be sent and / or stored by electronic device 216. In some examples, the data derived from the one or more images that is sent and / or stored by electronic device 216 does not include the one or more images. This metadata can correspond to insights that are determined from the images. In some examples, subject to user permissions, stored data can be included in the data displayed by electronic device 200 by accessing a bridge application corresponding to electronic device 216. In some examples, the bridge application on the smart phone communicates with electronic device 216 to obtain information. Electronic device 200 utilizes the bridge application to view the data that electronic device 216 detected. In some examples, the bridge application includes health data collected with permission from person 212, such as heart rate, activities, bodytemperature, blood oxygen saturation levels, and sleep patterns. That is, the data derived from the one or more images detected by sensor 222 can be viewed historically on electronic device 200 or electronic device 216 via the bridge application. In some examples, electronic device 216 can use the derived data to provide its functionality, such as using one or more steps and / or heart rate derived from data captured via sensor 222 to enhance workout data (e.g., include steps captured by sensor 222 for a workout that electronic device 216 begins recording after the steps were captured).

[0050] While the example of FIG. 2B describes sending data to electronic device 216 in response to detecting that person 212 has been working out and / or causing electronic device 216 to initiate recording a workout, similar techniques can be applied to other user cases. For example, electronic device 200 can send data to electronic device 216 in response to making a determination that person 212 has gone to the kitchen, which could apply to scenarios that involve whether the user has washed their hands for sanitary food preparation. As another example, electronic device 200 can send data to electronic device 216 in response to making a determination that the user is eating in order for electronic device 216 to consider the user’s eating habits when determining the user’s wellness. As yet another example, electronic device 200 can send data to electronic device 216 in response to making a determination that the user is meditating in order for electronic device 216 to initiate and / or capture a meditation session for person 212. In some examples, electronic device 200 detects subject in a particular room and either sends a notification relevant to the particular room to electronic device 216 or sends an identifier associated the particular room to the electronic device 216 so that the electronic device 216 can know where electronic device 216 is relative to location 206. In some examples, providing the electronic device 216 with information concerning the particular location of where the user is in the home can allow electronic device 216 more accurately identify the activities by person 212. In another set of examples, electronic device 200 detects subject in environment and obtains data sensed by electronic device 216 to decide how to proceed. For example, electronic device 200 (1) detects subject in environment, (2) obtains a heart rate of the subject via electronic device 216 (e.g., by sending a request to electronic device for the heart rate), (3) determines that the subject is exercising based on the heart rate, and (4) causes a smart television to play a workout video.

[0051] In some examples, in order for one or more of these determinations to be made, the characteristics of person 212 captured via the camera in the space 210 must meet one ormore respective criteria. As alluded to above, the one or more respective criteria are based on determining whether person 212 is performing and / or demonstrating a threshold amount of a characteristic, such as movement, heart rate, temperature, etc. In some examples, the one or more respective criteria includes a criterion that is satisfied when person 212 has above a threshold amount of a characteristic for a threshold amount of time (e.g., exercising for an amount of time). For example, if the camera detects person 212 laying down for a threshold amount of time accompanied with a consistently low movement, low resting heart rate, and increased body temperature reported by electronic device 216, electronic device 200 can infer that person 212 is sleeping and communicate this information to electronic device 216 accordingly. In some examples, the one or more respective criteria are based on detecting that person 212 has occupied a particular area for more than a threshold amount of time. In some examples, the one or more respective criteria are based on detecting that the attention and / or gaze of person 212 is directed to a particular area of the environment for more than the threshold amount of time.

[0052] In some examples, electronic device 200 only sends the data and / or causes electronic device 216 to perform an operation when electronic device 216 is operating in a certain mode (e.g., a fitness recording and / or health recording mode). For example, if electronic device 216 is operating in a do not disturb mode, electronic device 200 would not cause electronic device 216 to start recording a workout based on the activity user detected via sensor 222. However, if electronic device 216 is not operating in the do not disturb mode, electronic device 200 would cause electronic device 216 to start recording a workout based on the activity user detected via sensor 222. As another example, if electronic device 216 is in a fitness recording mode, electronic device 200 would not cause electronic device 216 to start a meditation exercise based on a user lying down in space 210 (e.g., detected via the camera). However, if electronic device 216 is not in the fitness recording mode, electronic device 200 would cause electronic device 216 to start the meditation exercise. Yet another example includes electronic device 200 communicating with electronic device 216 to determine whether electronic device 216 is in a locked or unlocked state. In some examples, electronic device 216 is only caused to execute operations only if in an unlocked state in response to determining that the activity of person 212 satisfies one or more respective criteria.

[0053] In some examples, in response to person 212 exhibiting a characteristic different from a previously exhibited characteristic, electronic device 200 performs an operationwithout causing electronic device 216 to perform an operation. For example, electronic device 200 can cause one or more other smart devices in location 206 to change states, such as turning a set of smart lights off in the room in which person 212 is sleeping (e.g., determined via sensor 222) and / or causing a smart television to play a workout video when a determination is made that person 212 is going to start working out (e.g., determined via sensor 222). Thus, in some examples, electronic device 200 can cause one or more smart devices to change state proactively or reactively based on one or more images detected by sensor 222.

[0054] While the examples described above involve electronic device 200 detecting one or more characteristics of person 212 via one or more images (and / or data derived from the one or more images) captured by a camera to cause electronic device 216 to perform one or more actions, it should be understood that the examples provided above are merely exemplary and are being provided for ease of discussion. The detection of different types of data from one or more other types of devices can be used to detect one or more characteristics of person 212 (and / or the physical environment) in order to cause electronic device 216 to perform one or more actions. In accordance with necessary user permissions, for example, electronic device 200 can use an array of data captured by different types of devices to detect one or more characteristics to cause electronic device 216. In some examples, the one or more characteristics includes motion characteristics, frequency characteristics (e.g., to detect Wi-Fi reflection and / or to detect whether a subject device is connected to Bluetooth or WiFi), environmental light characteristics, temperature characteristics, sound characteristics, etc. In some examples, the different types of devices include one or more sensors, such as one or more inertial measurement units, one or more antennas, one or more radio frequency receivers and / or transmitters, one or more ambient light sensors, one or more microphones, etc. In some examples, one or more of the different types of devices are included in location 206 described above. In some examples, the reflection of Wi-Fi signals in location 206 can be detected. In some examples, data corresponding to the reflected Wi-Fi signals can be used to generate a model (e.g., skeleton) of location 206. In some examples, the generated model of location 206 and / or data derived from the generated model can be sent to electronic device 216 via electronic device 200 to cause electronic device 216 to perform one or more operations, such as recommending or recording a particular type of workout by determining the size of the room in which person 212 is located.

[0055] The operations described above can be performed using various ecosystems of devices. Conceptually, a source device obtains and delivers data representing the environment to a decision controller. In the foregoing examples, for instance, an accessory device in the form of a camera acts as a source device by providing camera output about location 206. The camera output can be provided to a controller device with sufficient computation power to process the incoming information and generate instructions for other devices in a system (e.g., location 206 and / or one or more devices in communication with the controller device). Examples of electronic devices having sufficient computational power to act as controllers include a smart phone, a tablet, a laptop, and / or a desktop computer. Controller functionality can also be integrated into devices that have other primary functionalities, such as a media playback device, a smart speaker, a tabletop dock or smart screen, a television, and so forth. Source devices, such as the camera, can in some instances have sufficient computational power to act as controller devices. It should be appreciated that computational power generally represents a design choice that is balanced with power consumption, packaging, and cost. For example, a source device that is wired to main electricity can be more likely to take on controller device functionality than a battery- powered device, even though both are possible. The controller device, upon determining a decision based on the obtained sensor output, provides instructions to be processed by one or more other accessory devices in location 206. In the foregoing examples, the controller device instructs electronic device 216 to record, track, and / or initiate a meditation session for person 212.

[0056] The various ecosystem of devices described above can connect and communicate with one another using various communication configurations. Some exemplary configurations involve direct communications such as device-to-device connections. For example, a source device (e.g., camera) can capture images of location 206, determine a position of a subject and, acting as a controller device, determine to send an instruction to an accessory device to change states. The connection between the source device and the accessory device can be wired or wireless. The connection can be a direct device-to-device connection such as Bluetooth. Some exemplary configurations involve mesh connections. For example, a source device can use a mesh connection such as Thread to connect with other devices in environment 200. Some exemplary configurations involve local and / or wide area networks and can employ a combination of wired (e.g., Ethernet) and wireless (e.g., Wi-Fi, Bluetooth, Thread, and / or UWB) connections. For example, a camera can connect locallywith a controller hub in the form of a smart speaker, and the smart speaker can relay instructions remotely with a smart phone, over a cellular and / or Internet connection.

[0057] The description herein provides for the use of data from various possible sources, such as a system on a chip, server, personal computer system, smart phone, smart watch, wearable device, tablet, laptop computer, fitness tracking device, HMD device, desktop computer, accessory (e.g., switch, light, speaker, air conditioner, heater, window cover, fan, lock, media playback device, television, and so forth), controller, hub, and / or sensor (e.g., visual, aural, light, motion) to improve interactions with connected devices. In some instances, these sources can include electronic devices situated in an enclosed space. These connected, smart devices offer potential benefit to users. For example, security systems often incorporate cameras and other sensors. Accordingly, the use of smart devices enables users to have calculated control of benefits, including better coordination between devices, in their environment. Implementers should consider and adhere to prevailing privacy standards and policies in maintaining user privacy. These standards and policies should meet industry, government, and / or other legal requirements.

[0058] As one example, implementers can take steps to anonymize sensor data. For instance, sensors may operate at low resolution for automatic object detection, and capture at higher resolutions upon explicit user instruction. Additionally or alternatively, sensor data that is captured in the enclosed space can also be restricted to devices that are located in the enclosed space, and not be transmitted and stored in the cloud. Sensor data may be limited to devices that are in close proximity to one another, such that sensor data from one environment is not physically transmitted beyond the environment. Peer to peer communication protocols of limited range may be used to limit communication to trusted and / or local devices. Additionally or alternatively, identification information can be removed from (or otherwise not become associated with) the data. During set up, users may be allowed to selectively block the use of sensor information, such that users are allowed to, for example, opt-in or opt-out of certain features. For example, users may be allowed to enable ambient light detection while disabling high resolution camera imaging.

[0059] FIGS. 3A-3B is a flow diagram illustrating a method for coordinating between devices based on detecting one or more subject characteristics via an ecosystem of electronic devices in accordance with some examples, including those discussed with reference to FIGS. 2A-2B. In some examples, one or more operations of method 300 is performed by acontroller device as described above with reference to FIGS. 2A-2B. In some examples, the controller device is or is integrated with a device such as device 100 described above with reference to FIG. 1 A and / or electronic device 200 as described above with reference to FIGS. 2A-2B. Some operations in method 300 are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted. In some examples, operations of method 300 are shared between a first electronic device and a different second electronic device.

[0060] Method 300 provides an intuitive way for coordinating between devices based on detecting one or more subject characteristics via an ecosystem of electronic devices. A first electronic device (e.g., 100 and / or 200) detects (302), via one or more input devices, one or more characteristics (e.g., a position, an orientation, a status, a state, and / or a current activity) (e.g., one or more current characteristics and / or one or more physical characteristics) of a subject (e.g., 212) (e.g., a user, a person, a device, and / or an electronic device) (e.g., in a physical environment including the first electronic device, the one or more input devices, and / or the second electronic device) (e.g., the subject has woken up and / or the subject has gone to certain room in home) (e.g., as described above in relation to FIGS. 2A-2B, 212, and / or 214). In some examples, the first electronic device is in communication with the one or more input devices (e.g., a physical input mechanism, a camera, a touch-sensitive display, and / or a microphone). In some examples, the one or more input devices includes one or more sensors, such as one or more cameras, one or more inertial measurement units (e.g., used to detect motion), one or more antennas, one or more radio frequency receivers and / or transmitters (e.g., used to detect radio frequencies), one or more ambient light sensors (e.g., used to detect light in an environment), and one or more microphones (e.g., used to detect sound). In some examples, the one or more input devices are not physically coupled to the first electronic device. In some examples, the one or more input devices are stand-alone devices, such as a stand-alone camera. In some examples, the first electronic device is an accessory, a controller, a speaker, a smart light, a laptop, a smart phone, a smart watch, a fitness tracking device, a tablet, a head-mounted display (HMD) device, an electronic device, and / or another type of electronic device.

[0061] In response to (304) detecting the one or more characteristics of the subject (e.g., 212) (e.g., as described above in relation to FIGS. 2A-2B, 212, and / or 214) and in accordance with a determination that the one or more characteristics of the subject (e.g., 212) satisfies afirst set of one or more criteria (e.g., that the subject is performing a particular activity and / or action), the first electronic device causes (306) a second electronic device (e.g., 216) to perform a first operation (e.g., as described above in relation to FIG. 2B). In some examples, the first operation is performed based on the one or more characteristics. In some examples, causing the second electronic device to perform the first operation includes sending an indication of the one or more characteristics to the second electronic device. In some examples, the first set of one or more criteria includes a first criterion that is satisfied when a determination is made that the one or more characteristics of the subject includes a first type of characteristic (e.g., performing a health-oriented activity and / or not generally using, such as touching and / or providing input to, the second electronic device). In some examples, the first set of one or more criteria includes a second criterion (e.g., the first criterion or another criterion different from the first criterion) that is not satisfied when a determination is made that the one or more characteristics of the subject includes a second type of characteristic different from the first type of characteristic. In some examples, the first electronic device is in communication with the second electronic device (e.g., an accessory, a controller, a speaker, a smart light, a laptop, a smart phone, a smart watch, a fitness tracking device, a tablet, a head-mounted display (HMD) device, an electronic device, and / or another type of electronic device) different from the first electronic device (e.g., 200). In some examples, the second electronic device is different from the one or more input devices. In some examples, the second electronic device is a different type of device than the first electronic device. In some examples, a physical input mechanism is a hardware input mechanism, a rotatable input mechanism, a crown, a knob, a dial, a physical slider, and / or a hardware button. In some examples, causing the second electronic device to perform the first operation when the one or more characteristics of the subjects includes the first type of characteristic and forgoing causing the second electronic device to perform the first operation when the one or more characteristics of the subject includes the second type of characteristic enables the first electronic device to automatically cause the second electronic device to perform operations based on the type of detected characteristics, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0062] In response to (304) detecting the one or more characteristics of the subject and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, the first electronic device forgoes causing (308)the second electronic device (e.g., 216) to perform the first operation (e.g., as described above in relation to FIG. 2B). Causing the second electronic device to perform the first operation when one or more characteristics of the subject satisfies the first set of one or more criteria and forgoing causing the second electronic device to perform the first operation when the one or more characteristics of the subject do not satisfy the first set of one or more criteria enables the first electronic device to automatically cause the second electronic device to perform operations based on detected characteristics of the subject, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0063] In some examples, the first set of one or more criteria includes a third criterion that is satisfied when a determination is made that a first threshold amount (e.g., a non-zero amount) of a first characteristic of the subject (e.g., 212) (e.g., a particular characteristic) (e.g., length of time that the subject had and / or performed the first characteristic) (e.g., a degree of which that the subject has the first characteristic) is detected. In some examples, the first set of one or more criteria includes a fourth criterion (e.g., the third criterion or another criterion different from the third criterion) that is not satisfied when a determination is made that the first threshold amount of the first characteristic of the subject is not detected (and / or that a third threshold amount of the first characteristic of the subject is detected) (e.g., as described above in relation to FIGS. 2A-2B). In some examples, the third threshold amount is a non-zero amount. Causing the second electronic device to perform the first operation when the first threshold amount of the first characteristic is detected and forgoing causing the second electronic device to perform the first operation when the first threshold amount of the first characteristic is not detected enables the first electronic device to automatically cause the second electronic device to perform operations based on an amount of a characteristic being detected, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0064] In some examples, the first set of one or more criteria includes a fifth criterion that is satisfied when a determination is made that a second threshold amount of a second characteristic of the subject (e.g., 212) is detected in less than a predetermined period of time (e.g., as described above in relation to FIGS. 2A-2B). In some examples, the first set of oneor more criteria includes a sixth criterion (e.g., the fifth criterion or another criterion different from the fifth criterion) that is not satisfied when a determination is made that the second threshold amount of the second characteristic of the subject is not detected in less than the predetermined period of time (e.g., as described above in relation to FIGS. 2A-2B). Causing the second electronic device to perform the first operation when the second threshold amount of the second characteristic is detected in less than the predetermined period of time and forgoing causing the second electronic device to perform the first operation when the second threshold amount of the second characteristic is not detected in less than the predetermined period of time enables the first electronic device to automatically cause the second electronic device to perform operations based on an amount of a characteristic being satisfied within a predetermined period of time, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0065] In some examples, the first set of one or more criteria includes a seventh criterion that is satisfied when a determination is made that the second electronic device (e.g., 216) is operating in a first mode (e.g., fitness tracking, health tracking, and / or navigation mode) (e.g., an operational mode, such as a non-sleep mode and / or a non-do-no-disturb mode) (e.g., a locked or unlocked mode) (e.g., as described above in relation to FIGS. 2A-2B). In some examples, the first set of one or more criteria includes an eighth criterion (e.g., the seventh criterion or another criterion different from the seventh criterion) that is not satisfied when a determination is made that the second electronic device (e.g., 216) is operating in a second mode different from the first mode (e.g., as described above in relation to FIGS. 2A-2B). In some examples, the first mode and the second mode are both operational and / or active modes. Causing the second electronic device to perform the first operation when the second electronic device is operating in the first mode and forgoing causing the second electronic device to perform the first operation when the second electronic device is operating in the second mode enables the first electronic device to automatically cause the second electronic device to perform operations based on a particular mode of the second electronic device, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0066] In some examples, in response to (304) detecting the one or more characteristics of the subject (e.g., 212) and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, the first electronic device causes (310) the second electronic device (e.g., 216) to perform a second operation different from the first operation (e.g., as described above in relation to FIGS. 2A- 2B). In some examples, the second operation is performed based on the one or more characteristics of the subject. In some examples, the second operation is the same type of operation as the first operation but is based on and / or uses one or more different characteristics. In some examples, causing the second electronic device to perform the second operation includes sending an indication of the one or more characteristics of the subject to the second electronic device. Causing the second electronic device to perform the second operation in response to detecting the one or more characteristics of the subject and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria enables the first electronic device to react differently to detecting different characteristics, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0067] In some examples, the one or more characteristics is detected via one or more cameras (e.g., the one or more characteristics is detected in one or more images captured by the one or more cameras) (e.g., as described above in relation to FIGS. 2A-2B).

[0068] In some examples, causing the second electronic device (e.g., 216) to perform the first operation includes causing the second electronic device (e.g., 216) to output a notification (e.g., 218) (e.g., display a notification, output a haptic corresponding to notification, and / or output a sound corresponding to a notification). Causing the second electronic device to perform an operation including causing the second electronic device to output the notification based on whether one or more characteristics of the subject satisfy the first set of one or more criteria, enables the first electronic device to automatically notify a user based on detected characteristics of the first electronic device, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0069] In some examples, causing the second electronic device (e.g., 216) to perform the first operation includes causing the second electronic device to retain (e.g., collect, store, and / or send to another electronic device, such as the first electronic device or a different electronic device than the first electronic device) data detected via one or more sensors of the second electronic device (e.g., as described above in relation to FIG. 2B). Causing the second electronic device to retain data detected via the one or more sensors enables the first electronic device to cause different data to be retained in different situations, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0070] In some examples, the first electronic device (e.g., 200) is in communication with a display generation component. In some examples, after causing the second electronic device (e.g., 216) to retain the data detected via the one or more sensors of the second electronic device, the first electronic device displays, via the display generation component, a representation of the data detected via the one or more sensors of the second electronic device (e.g., as described above in relation to FIG. 2B). In some examples, the representation of the data is and / or includes the data. In some examples, the data is displayed (e.g., via the display generation component and / or by the second electronic device) in a user interface of an application corresponding to the second electronic device (e.g., the user interface corresponding to the second electronic device and / or the application corresponds to the second electronic device). In some examples, before and / or after causing the second electronic device to retain the data detected via the one or more sensors of the second electronic device, the first electronic device establishes a communication channel with the second electronic device. In some examples, after the first electronic device establishes the communication channel, the first electronic device receives, via the communication channel, the data detected via the one or more sensors of the second electronic device. In some examples, after receiving the data detected via the one or more sensors of the second electronic device, the data is displayed via the display generation component. The first electronic device displaying the representation of the data detected via the one or more sensors of the second electronic device enables the first electronic device to cause data to be detected by another electronic device that is then displayed by the first electronic device, thereby reducing the number of inputs needed to perform an operation, performing anoperation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0071] In some examples, the second electronic device (e.g., 216) is a wearable device (e.g., a head-mounted display device, a smart watch, and / or smart ring). In some examples, the first electronic device is not a wearable device.

[0072] In some examples, the second electronic device (e.g., 216) is logged into a same user account as the first electronic device (e.g., 200). The second electronic device being logged into the same user account as the first electronic device enables different electronic devices logged into the same account to work together for performing operations, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0073] In some examples, the first electronic device (e.g., 200) includes the one or more input devices. In some examples, the one or more input devices are separate from the second electronic device (e.g., 216).

[0074] In some examples, in response to detecting the one or more characteristics of the subject (e.g., 212) and in accordance with a determination that the one or more characteristics of the subject satisfies a second set of one or more criteria different from the first set of one or more criteria, the first electronic device causes the second electronic device to perform a third operation different from the first operation (e.g., as described above in relation to FIG 2B). In some examples, in response to detecting the one or more characteristics of the subject and in accordance with a determination that the one or more characteristics of the subject do not satisfy the second set of one or more criteria, the first electronic device forgoes causing the second electronic device to perform the third operation. Causing the second electronic device to perform the third operation when the second set of one or more criteria are satisfied enables the first electronic device to automatically cause the second electronic device to perform different operations based on different characteristics, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0075] In some examples, the one or more characteristics of the subject (e.g., 212) is one or more first characteristics of the subject. In some examples, after causing the second electronic device (e.g., 216) to perform the first operation, the first electronic device detects (312), via the one or more input devices, one or more second characteristics of the subject (e.g., 212) different from the one or more first characteristics of the subject. In some examples, in response to detecting the one or more second characteristics of the subject (and / or in accordance with a determination that the one or more second characteristics of the subject satisfy a third set of one or more criteria), the first electronic device causes (314) the second electronic device to perform a fourth operation different from the first operation (e.g., as described above in relation to FIG. 2B). In some examples, the third set of one or more criteria is the second set of one or more criteria and / or another set of one or more criteria different from the second set of one or more criteria. In some examples, in response to detecting the one or more second characteristics of the subject and in accordance with a determination that the one or more second characteristics of the subject do not satisfy the third set of one or more criteria, the first electronic device forgoes causing the second electronic device to perform the fourth operation. Causing the second electronic device to perform a fourth operation when the one or more second characteristics of the subject satisfy the third set of one or more criteria enables the first electronic device to automatically perform different operations for different characteristics, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and providing improved feedback to the user.

[0076] In some examples, the one or more characteristics of the subject (e.g., 212) is one or more first characteristics of the subject. In some examples, the first electronic device detects (316), via the one or more input devices, one or more third characteristics of the subject different from the one or more first characteristics of the subject. In some examples, in response to (318) detecting the one or more third characteristics of the subject and in accordance with a determination that the one or more third characteristics of the subject satisfy a fourth set of one or more criteria (e.g., as described above in relation to the first set of one or more criteria), the first electronic device causes (320) the second electronic device (e.g., 216) to perform a fourth operation of a first type (e.g., as described above in relation to FIG. 2B). In some examples, in response to (318) detecting the one or more third characteristics of the subject and in accordance with a determination that the one or more third characteristics of the subject satisfy a fifth set of one or more criteria (e.g., as describedabove in relation to the first set of one or more criteria) different from the fourth set of one or more criteria, the first electronic device causes (322) the second electronic device (e.g., 216) to perform a fifth operation of a second type, wherein the fifth operation is different from the fourth operation, and wherein the second type is different from the first type (e.g., as described above in relation to FIG. 2B). Causing the second electronic device to perform operations of different types based on different criteria being satisfied enables the first electronic device to automatically perform different types of operations that match particular criteria, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0077] In some examples, causing the second electronic device (e.g., 216) to perform the first operation includes sending, to the second electronic device, an indication (e.g., represented by 218) (e.g., an instruction, an image, text, and / or one or more symbols) (e.g., the subject has moved a certain amount, the subject has moved at a certain speed, the subject has gone into a particular area of a building, such as a kitchen and / or a bathroom, and / or the subject is using a particular device and / or type of device, such as a refrigerator and / or a blood pressure cuff) of the one or more characteristics. Causing the second electronic device to perform an operation including sending the indication of the one or more characteristics based on whether one or more characteristics of the subject satisfies the first set of one or more criteria and forgoing causing the second electronic device to perform the first operation based on one or more characteristics of the subject do not satisfy the first set of one or more criteria enables the first electronic device to automatically send the characteristics based on detected characteristics of the first electronic device, thereby reducing the number of inputs needed to perform an operation, performing an operation when a set of conditions has been met without requiring further user input, and / or providing improved feedback to the user.

[0078] Note that details of the processes described above with respect to method 300(e.g., FIG. 3) are also applicable in an analogous manner to other methods described herein. For example, method 700 optionally includes one or more of the characteristics of the various methods described above with reference to method 300. For example, performing the operation of method 300 at the selected device of method 700. For brevity, these details are not repeated herein.

[0079] Attention is now directed towards techniques for disambiguating between users and / or devices in an environment. Some techniques use sensor data detected by the devices to correlate to different states of objects in media content of the environment. For example, it can be determined using the media content that an operation is being requested by an object that is moving. In such an example, sensor data received from the devices can indicate that one device is being worn by a user that is moving while another device is being worn by a user that is not moving. Accordingly, an identity associated with the device being worn by the user that is moving can be used to perform the operation. Other techniques use visual aspects of an environment to determine where users and / or devices are located in the environment. For example, it can be determined that an operation should be performed with respect to a particular user. In such an example, a device of the particular user can be determined to be outputting particular content. In response to detecting a location of output of the particular content, it can be determined where the particular user is located.

[0080] FIGS. 4A-4C are block diagrams illustrating a scenario where an electronic device (e.g., first device 400) determines an identity corresponding to a requested operation and then performs the requested operation based on the identity. The scenario in these figures is used to illustrate the processes described below, including the processes in FIGS. 6-7.

[0081] As illustrated in FIGS. 4A-4C, first device 400 is in an environment (e.g., a room, an office, an outside area, and / or a building) with second device 402 and third device 404. In some examples, first device 400, second device 402, and / or third device 404 includes one or more components and / or features described above in relation to electronic device 100 and / or electronic device 200. It should be recognized that techniques described herein can be used with more than three devices.

[0082] As illustrated in FIGS. 4A-4C, second device 402 and third device 404 are within a field of view of first device 400 (e.g., as illustrated by second device 402 and third device 404 being within dotted lines 406). In some examples, the field of view of first device 400 corresponds to and / or is a field of view of one or more cameras of first device 400.

[0083] At FIG. 4A, first device 400 detects voice command 405a. In one example, voice command 405a is a request to “Play My Favorite Song”. It should be recognized that other operations can be used with techniques described herein. In some examples, voice command 405a originates from a user that first device 400 is unable to identify. In such examples, therequest cannot be properly performed until first device 400 determines an identity of a source of voice command 405a (e.g., because different users in the environment can have different favorite songs). It should be recognized that voice command 405a is just one example of a type of input that can be used with techniques herein and that other types of inputs can also be used, such as a text input, a touch input, and / or an air gesture.

[0084] In response to being unable to determine the identity of the source of voice command 405a, first device 400 initiates a disambiguation process. In some examples, the disambiguation process includes first device 400 determining a direction (e.g., a general direction, such as in front of and / or to the left front of first device 400, as illustrated in FIG. 4A) and / or a location from which voice command 405a originated. For example, a direction of the sound of voice command 405a can be determined using known techniques for acoustic location. For another example, movement of a user corresponding to voice command 405a (such as movement of lips) can be determined using known techniques for image and / or video analysis. In such examples, the direction and / or the location can be used to identify which object in media content of the environment corresponds to voice command 405a. In some examples, as part of or separate from determining the object from which voice command 405a originated, the disambiguation process includes first device 500 capturing first media content (e.g., an image and / or a video) of the environment. At FIG. 4A, based on one or more operations discussed in this paragraph, first device 400 determines that the source of voice command 405a is an object to the left in the field of view of first device 300.

[0085] As illustrated in FIG. 4B, third device 404 has moved closer to second device 402 as compared to FIG. 4A. As also illustrated in FIG. 4B, second device 402 has not moved between FIG. 4A and FIG. 4B. At FIG. 4B, first device 400 continues the disambiguation process by capturing a second media content of the environment. In some examples, by comparing the first media content and the second media content, first device 400 determines that the source of voice command 405a is moving. In other examples, by analyzing an appearance of one or more objects in the second media content, first device 400 determines that the source of voice command 405a is moving. In other examples, by analyzing an appearance of one or more objects in the second media content, a heart rate can be predicted for users in the second media content using known techniques for identifying heart rate using media content.

[0086] At FIG. 4B, in conjunction with (e.g., after and / or while) capturing the second media content, first device 400 sends a request for sensor data and / or source data to second device 402 and third device 404. Examples of the sensor data include location data (e.g., Global Positioning System (GPS) data, Wi-Fi- Positioning System (WPS) data, Cellular Network positioning data, Bluetooth Low Energy data and / or Sensor Fusion data), orientation data (e.g., IMU data, gyroscope data, and / or accelerometer data), and / or biometric data (e.g., a user’s heart rate, calorie count, and / or step count). In some examples, different sensor data is requested from each device (e.g., as a result of different devices having different sensors, such as heart rate from one device and step count from another device). Examples of the source data include IP address, MAC address, domain or other name, and / or certificate, which may each be encrypted. In response to receiving the request, second device 402 and third device 404 send sensor data and / or source data to first device 400. For example, second device 402 and third device 404 can each send heart rate data and source data to first device 400.

[0087] In some examples, first device 400 determines an identity of the source of voice command 405a by comparing sensor data from second device 402 and sensor data from third device 404. For example, the sensor data from third device 404 can indicate that the heart rate of a user of third device 404 (sometimes referred to as the third user) has increased over time and / or is elevated while the sensor data from second device 402 can indicate that the heart rate of a user of second device 402 (sometimes referred to as the second user) has decreased over time and / or is not elevated. In such an example, first device 400 uses the information that a user corresponding to voice command 405a is moving and that the third user is determined to be moving (e.g., by the heart rate increasing over time and / or being elevated) to infer (e.g., determine) the source of voice command 405a as the third user. In other examples, first device 400 uses the predicted heart rate as compared to the heart rate received from third device 404 to infer the source of voice command 405a as the third user. In some examples, as part of inferring the source of voice command 405a and after determining which device is the source of voice command 405a, first device 400 uses the source data received from third device 404 to determine the identity to be used when performing the operation.

[0088] In some examples, instead of or in addition to requesting sensor data, first device 400 requests that second device 402 and third device 404 each output unique content, such as one or more colors or a unique sequence of infrared light. In such examples, first device 400determines the source of voice command 405a by detecting which unique content is output by the object in the second media content that is determined to correspond to the source of voice command 405a.

[0089] In some examples, instead of or in addition to requesting sensor data, first device 400 requests information on what is currently being output by second device 402 and third device 404. In such examples, first device 400 determines the source of voice command 405a by detecting what content is output by the object in the second media content that is determined to correspond to the source of voice command 405a.

[0090] As illustrated in FIG. 4C, after determining the source of voice command 405a, first device 400 outputs audio content 408 (e.g., audio book, radio station, podcast, and / or song). In some examples, audio content 408 is a song that is designated as the favorite song of the third user. It should be recognized that audio content 408 is just one example of output that can correspond to a user and that techniques described herein can be used to disambiguate between users for purposes in addition to and / or other than outputting audio, such as outputting notifications, displaying content, and / or communicating with other devices.

[0091] FIGS. 5A-5C are block diagrams illustrating a scenario where an electronic device (e.g., first device 400) receives a notification corresponding to a specific user and then identifies a device near the specific user to output the notification. The scenario in these figures is described in the same environment as discussed above with respect to FIGS. 4A-4C and is used to illustrate the processes described below, including the processes in FIGS. 6-7.

[0092] At FIG. 5A, first device 400 receives notification 502. In some examples, notification 502 corresponds to the second user though first device 400 does not know where the second user is located. In some examples, notification 502 is received from an application operating on first device 400 such as a calendar application or a messaging application. In other examples, notification 502 is received from another device such as a server not in the environment.

[0093] At FIG. 5A, after receiving notification 502, first device 400 initiates a disambiguation process to identify a position of users and / or devices (e.g., second device 402 and third device 404) in the environment so that, in some examples, an operationcorresponding to the notification can be performed by a device that is located near the intended recipient of the operation.

[0094] At FIG. 5 A, first device 400 sends a request to second device 402 to perform a first operation and / or a request to third device 404 to perform a second operation. In some examples, the second operation is the same as the first operation and such requests are sent at different times in order to differentiate the devices. In other examples, the second operation is different from the first operation.

[0095] In some examples, after sending the request to second device 402 and / or third device 404, first device 400 captures media content of the environment. As illustrated in FIG. 5B, as a result of receiving the request from first device 400, second device is outputting content 506 and third device 404 is outputting content 504. Such output of each device can be captured in the media content such that first device 400 can determine which device is which and / or where a particular device is located in the environment.

[0096] As illustrated in FIG. 5C, after identifying locations of second device 402 and third device 404, first device 400 causes audio notification 508 to be output by a device (such as a speaker) near second device 402 and an audio notification to not be output by one or more devices that are not near (e.g., within a predetermined distance from) the user of second device 402.

[0097] FIG. 6 is a flow diagram illustrating a method (e.g., method 600) for performing an operation based on an identified user in accordance with some examples. Some operations in method 600 are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted.

[0098] As described below, method 600 provides an intuitive way for performing an operation based on an identified user. In some examples, method 600 is performed at a first device (e.g., 400) (e.g., a watch, a phone, a tablet, a fitness tracking device, a processor, a head-mounted display (HMD) device, a communal device, a media device, a speaker, a television, and / or a personal computing device) that is in communication with (and / or including) one or more input devices (e.g., a camera, a depth sensor (e.g., a stereo sensor, a Time-of-Flight (ToF) sensor, and / or a Structured Light sensor), a radar, a light Detection and Ranging (LiDAR) sensor, infrared sensors, an inertial measurement sensor, a microphone, a hardware input mechanism, a rotatable input mechanism, and / or a temperature sensor).

[0099] The first device sends (602), to a plurality of other devices (e.g., 402 and / or 404) (e.g., a watch, a phone, a tablet, a fitness tracking device, a processor, a head -mounted display (HMD) device, a communal device, a media device, a speaker, a television, and / or a personal computing device) (e.g., in a physical environment) (e.g., via a short-range communication channel) (e.g., via Bluetooth or Wi-Fi), a request to perform a first operation (e.g., display content, turn on and / or off a light, authenticate a user, perform facial recognition, output audio, and / or output haptic signal), wherein the plurality of other devices does not include the first device. In some examples, a second device of the plurality of other devices is a different type of device than a third device of the plurality of other devices.

[0100] In conjunction with (e.g., in response to and / or after) sending the request to perform the first operation, the first device captures (604), via the one or more input devices, media content (e.g., an image, a video, and / or a sound recording) of an environment (e.g., as indicated by field of view of 406) (e.g., a physical environment, such as a room or an outside area). In some examples, the plurality of other devices are included in the environment.

[0101] The first device detects (606), via the one or more input devices, a first input (e.g., 405a) (e.g., a user input) (e.g., a verbal input (e.g., a verbal utterance, a sound, an audible request, an audible command, and / or an audible statement) and / or a non-verbal input (e.g., a swipe input, a hold-and-drag input, a gaze input, an air gesture, and / or a mouse click)) corresponding to a request to perform a second operation (e.g., directed to an application and / or a personal assistant executing on the first device) (e.g., answer a question and / or provide information, such as “what is the weather,” “when is my next meeting,” “do I have any text messages,” and / or “how much traffic is there to work”). In some examples, sending the request to perform the first operation and / or capturing the media content of the environment is performed in conjunction with (e.g., before, in response to, and / or after) detecting the first input corresponding to the request to perform the second operation. In some examples, the first input is detected by a different input device that used to capture the media content.

[0102] In accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content of the environment (e.g., as indicated by field of view of 406) and a first response (e.g., by one or more devices of the plurality of other devices) to the request to perform the first operation (e.g., from a second device of the plurality of devices) (e.g., thefirst response is received from the second device and / or identified within the media content), the first device performs (608) the second operation using first data (e.g., 408) (e.g., corresponding to a first user that is located within the media content based on the first response to the request) (and / or performs the second operation in a first manner). In some examples, the first response corresponds to the second device. In some examples, the first device is in communication with and / or includes one or more output devices (e.g., a display generation component (e.g., a display screen, a projector, and / or a touch- sensitive display), an audio generation component (e.g., smart speaker, home theater system, soundbar, headphone, earphone, earbud, speaker, television speaker, augmented reality headset speaker, audio jack, optical audio output, Bluetooth audio output, and / or HDMI audio output), a speaker, a haptic output device, a display screen, a projector, and / or a touch-sensitive display). In some examples, in response to and / or after detecting the first input corresponding to the request to perform the second operation and in accordance with a determination that the first set of one or more criteria is satisfied, the first device outputs, via the one or more output devices, first content. In some examples, the second operation is performed using the first data in response to and / or after detecting the first input corresponding to the request to perform the second operation. In some examples, performing the second operation using the first data includes and / or results in outputting, via the one or more output devices, first content. In some examples, the first data corresponds to the second device. In some examples, in response to and / or after detecting the first input corresponding to the request to perform the second operation (and / or before performing the second operation (e.g., using the first data and / or the second data)) and in accordance with a determination that the first set of one or more criteria is satisfied, the first device selects, based on the media content of the environment (e.g., the media content including a response by a plurality of devices of the plurality of other devices), the first data to be used with respect to the second operation.

[0103] In accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and a second response (e.g., the first response and / or another response different from the first response) (e.g., by one or more devices of the plurality of other devices) to the request to perform the first operation (e.g., from the second device or a third device of the plurality of devices) (e.g., the second response is received from the second device or the third device and / or identified within the media content), the first device performs (610) the secondoperation using second data different from the first data (e.g., corresponding to a second user, different from the first user, that is located within the media content based on the second response to the request) (e.g., without using the first data) (and / or performs the second operation in a second manner different from the first manner). In some examples, the second operation is performed using the first data without using the second data. In some examples, the second response corresponds to the second device or the third device. In some examples, in response to and / or after detecting the first input corresponding to the request to perform the second operation and in accordance with a determination that the second set of one or more criteria is satisfied, the first device outputs, via the one or more output devices, second content, different from the first content, without outputting the first content. In some examples, the second operation is performed using the second data in response to and / or after detecting the first input corresponding to the request to perform the second operation. In some examples, performing the second operation using the second data includes and / or results in outputting, via the one or more output devices, second content different from the first content. In some examples, the second data corresponds to the third device of the plurality of other devices. In some examples, the third device is different from the first device and / or the second device. In some examples, in response to and / or after detecting the first input corresponding to the request to perform the second operation (and / or before performing the second operation (e.g., using the first data and / or the second data)) and in accordance with a determination that the second set of one or more criteria is satisfied, the first device selects, based on the media content of the environment (e.g., the media content including a response by a plurality of devices of the plurality of other devices), the second data to be used with respect to the second operation.

[0104] In some examples, in accordance with a determination that a third set of one or more criteria, different from the first set of one or more criteria and the second set of one or more criteria, is satisfied, wherein the third set of one or more criteria includes a criterion (e.g., identifying a user within the media content and / or a user of the first device, not identifying a user within the media content, and / or not exceeding a threshold of confidence in identifying a user within the media content) that is satisfied based on the media content of the environment (e.g., as indicated by field of view of 406), a third response (e.g., the first response, the second response, and / or another response different from the first response and the second response) to the request to perform the first operation (e.g., received from a device and / or identified within the media content), and a fourth response (e.g., the first response, thesecond response, and / or another response different from the first response and the second response) to the request to perform the first operation (e.g., received from a device and / or identified within the media content), the first device performs the second operation using third data (e.g., corresponding to the first user, the second user, or another user, different from the first user and the second user, that is located within the media content based on the third response and the fourth response) (e.g., without using the first data and / or the second data) (e.g., not corresponding to a particular user in the environment and / or corresponding to a user logged into the first device) (e.g., different from the first data and the second data), wherein the third response corresponds to (and / or is associated with, from and / or of) a second device of the plurality of other devices (e.g., 402 and / or 404), wherein the fourth response corresponds to (and / or is associated with, from and / or of) a third device (e.g., 404) of the plurality of other devices (e.g., without corresponding to the second device), and wherein the third device is different from the second device (e.g., 402). In some examples, performing the second operation using the third data includes outputting an alert that a user corresponding to the first input cannot be identified. In some examples, the third set of one or more criteria includes a criterion that is satisfied based on the media content of the environment and multiple responses to the request to perform the first operation (e.g., received from multiple different devices and / or identified within the media content).

[0105] In some examples, the first input (e.g., 405a) is (and / or includes) audio input (e.g., as illustrated by the text bubble of 405a) (e.g., a verbal utterance, a sound, an audible request, an audible command, and / or an audible statement) (e.g., captured via a microphone), visual input (e.g., a pose of a user or an air gesture) (e.g., captured via a camera), or a combination thereof. In some examples, the input is multimedia inputs. In some examples, the first input is a touch input (e.g., captured via a touch-sensitive surface).

[0106] In some examples, the request to perform the first operation corresponds to a request to output visual content (e.g., turn off and / or on a light (e.g., infrared and / or colored light), keep a light on for a predetermined of time, output a pattern of light, and / or output content (e.g., personalized data and / or biometric data)) (e.g., via a display component in communication with and / or of each device in the plurality of other devices). In some examples, the request to perform the first operation corresponds a request to output haptic output and / or audio output.

[0107] In some examples, the criterion of the first set of one or more criteria is satisfied when visual content (e.g., the first response) that corresponds to the request to output visual content is detected (and / or identified) within the media content at a first location (e.g., corresponding to a first user (e.g., a first user account) (and / or a device (e.g., of the plurality of devices) of the first user)). In some examples, the first data corresponds to (and / or is associated with) the first location (e.g., the first user (e.g., and not another user different from the first user) and / or a particular device). In some examples, the criterion of the second set of one or more criteria is satisfied when visual content (e.g., the second response) that corresponds to the request to output visual content is detected (and / or identified) within the media content at a second location (e.g., corresponding to a second user (e.g., a second user account) (and / or a device (e.g., of the plurality of devices) of the second user) (e.g., the location corresponding to the second user is different from the location corresponding to the first user) different from the first user) different from the first location. In some examples, the second data corresponds to the second location (e.g., the second user (e.g., and not another user different from the second user)). In some examples, the first response includes and / or is visual content within the media content when the visual content is located at location corresponding to a device of the plurality of devices that corresponds to the first data. In some examples, the second response includes and / or is visual output within the media content when the visual content is located at location corresponding to a device of the plurality of devices that corresponds to the second data.

[0108] In some examples, the criterion of the first set of one or more criteria is satisfied when visual content (e.g., the first response) that corresponds to the request to output visual content is detected (and / or identified) within the media content at a location corresponding to a second device (e.g., 402) of the plurality of devices (e.g., 402 and / or 404). In some examples, the first data corresponds to the second device (e.g., and not the third device or another device different from the second device). In some examples, the criterion of the second set of one or more criteria is satisfied when visual content (e.g., the second response) that corresponds to the request to output visual content is detected (and / or identified) within the media content at a location corresponding to a third device (e.g., 404) of the plurality of devices different from the second device of the plurality of devices. In some examples, the second data corresponds to the third device (e.g., and not the third device or another device different from the second device. In some examples, the first response includes and / or is visual content within the media content when the visual content is located at locationcorresponding to the second device). In some examples, the second response includes and / or is visual output within the media content when the visual content is located at location corresponding to the third device.

[0109] In some examples, performing the second operation using the first data (e.g., 408) includes playing a first media item (e.g., media content, such as a song and / or a video) corresponding to a first user (e.g., with a first profile). In some examples, performing the second operation using the second data includes playing a second media item, different from the first media item, corresponding to a second user (e.g., with a second profile different from the first profile) different from the first user.

[0110] In some examples, performing the second operation using the first data (e.g., 408) includes outputting first content (e.g., a notification, such as audio notification, visual notification, and / or haptic notification) in a first area (e.g., through the first device, one or more of the plurality of other devices, the second device, and / or another device not included in the plurality of other device such as an accessory device) corresponding to a location of a first user (and / or the second device) captured in the media content of the environment (e.g., as indicated by field of view of 406). In some examples, performing the second operation using the second data includes outputting second content (e.g., a notification, such as audio notification, visual notification, and / or haptic notification) (e.g., the first content or second content different from the first content) in a second area (e.g., through the first device, one or more of the plurality of other devices, the third device, and / or another device not included in the plurality of other device such as an accessory device), different from the first area, corresponding to a location of a second user (and / or the third device), different from the first user, captured in the media content of the environment.[OHl] In some examples, performing the second operation using the first data (e.g., 408) includes (1) selecting, using the media content of the environment (e.g., as indicated by field of view of 406), a second device (e.g., to output a notification (e.g., audio notification, image notification, and / or haptic notification)) of the plurality of other devices (e.g., 402 and / or 404) and (2) sending, to the second device, content for output. In some examples, performing the second operation using the second data includes (1) selecting, using the media content of the environment, a third device (e.g., to output a notification (e.g., audio notification, image notification, and / or haptic notification)) of the plurality of other devices different from thesecond device of the plurality of other device and (2) sending, to the third device, content for output (e.g., without sending, to the second device, content for output).

[0112] In some examples, sending, to the plurality of other devices (e.g., 402 and / or 404), the request to perform the first operation includes: sending, to a second device (e.g., 402) of the plurality of other devices, a request to output first content (e.g., a request to flash a green light); and sending, to a third device (e.g., 404) of the plurality of other devices different from the second device of the plurality of other devices, a request to output second content (e.g., a request to flash a red light) different from the first content (e.g., different devices are both asked to display content but the content asked to be displayed is different content so as to be able to differentiate between the different devices). In some examples, different devices are asked to perform different operations (e.g., output visual content, output audio content, respond with a location, and / or authenticate) so as to be able to differentiate between the different devices when the different devices have different capabilities, such as some devices might not be in communication with or include a display component.

[0113] In some examples, sending, to the plurality of other devices (e.g., 402 and / or 404), the request to perform the first operation occurs in response to a determination that a user (and / or a device) corresponding to the first input (e.g., 405a) is ambiguous (and / or unknown) (e.g., the request to perform the first operation is sent to disambiguate the user corresponding to the first input, such as when a voice corresponding to the first input is unknown and / or there are multiple users in the environment).

[0114] In some examples, before sending, to the plurality of other devices (e.g., 402 and / or 404), the request to perform the first operation, the first device detects multiple users (e.g., people) (and / or devices) in the environment (e.g., as indicated by field of view of 406), wherein sending to the plurality of other devices, the request to perform the first operation occurs in response to detecting the multiple users in the environment. In some examples, sending, to the plurality of other devices, the request to perform the first operation occurs in response to a determination that the environment includes multiple users (e.g., people) (and / or devices).

[0115] Note that details of the processes described above with respect to method 600 (e.g., FIG. 6) are also applicable in an analogous manner to other methods described herein. For example, method 700 optionally includes one or more of the characteristics of the variousmethods described above with reference to method 600. For example, performing the operation of method 600 at the selected device of method 700. For brevity, these details are not repeated herein.

[0116] FIG. 7 is a flow diagram illustrating a method (e.g., method 700) for outputting content at a selected device based on proximity to an identified user in accordance with some examples. Some operations in method 700 are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted.

[0117] As described below, method 700 provides an intuitive way for outputting content at a selected device based on proximity to an identified user. In some examples, method 700 is performed at a first device (e.g., 400) (e.g., a watch, a phone, a tablet, a fitness tracking device, a processor, a head-mounted display (HMD) device, a communal device, a media device, a speaker, a television, and / or a personal computing device) that is in communication with (and / or includes) one or more input devices (e.g., a camera, a depth sensor (e.g., a stereo sensor, a Time-of-Flight (ToF) sensor, and / or a Structured Light sensor), a radar, a light Detection and Ranging (LiDAR) sensor, infrared sensors, an inertial measurement sensor, a microphone, a hardware input mechanism, a rotatable input mechanism, and / or a temperature sensor).

[0118] The first device captures (702), via the one or more input devices, media content (e.g., an image, a video, and / or a sound recording) of an environment (e.g., as illustrated by the field of view of 406) (e.g., a physical environment, such as a room or an outside area). In some examples, the plurality of other devices are included in the environment and / or a portion of the environment.

[0119] The first device sends (704), to a plurality of other devices (e.g., 402 and / or 404) (e.g., a watch, a phone, a tablet, a fitness tracking device, a processor, a head -mounted display (HMD) device, a communal device, a media device, a speaker, a television, and / or a personal computing device) (e.g., in a physical environment) (e.g., via a short-range communication channel) (e.g., via Bluetooth or Wi-Fi), a request for sensor data, wherein the plurality of other devices does not include the first device. In some examples, before sending the request for sensor data, the first device detects, via the one or more input devices, a first input (e.g., user input) (e.g., a verbal input (e.g., a verbal utterance, a sound, an audible request, an audible command, and / or an audible statement) and / or a non-verbal input (e.g., aswipe input, a hold-and-drag input, a gaze input, an air gesture, and / or a mouse click)) corresponding to a first operation (e.g., directed to an application and / or a personal assistant executing on the first device) (e.g., answer a question and / or provide information, such as “what is the weather,” “when is my next meeting,” “do I have any text messages,” and / or “how much traffic is there to work”). In some examples, the first device sends the request for sensor data in conjunction with (e.g., before, in response to, while, and / or after) detecting the first input corresponding to the first operation. In some examples, the first device sends the request for sensor data in conjunction with (e.g., before, in response to, while, and / or after) capturing the media content of the environment.

[0120] After (706) sending the request for sensor data, the first device receives (708), from a second device (e.g., 402) of the plurality of other devices, first sensor data (e.g., 506) (e.g., a position (e.g., location and / or orientation) of the second device, a position (e.g., location and / or orientation) relative to the first device and / or another object in the environment, and / or a value of sensor data from a sensor of the second device).

[0121] After (706) sending the request for sensor data, the first device receives (710), from a third device (e.g., 404), different from the second device, of the plurality of other devices, second sensor data (e.g., 504) (e.g., a position (e.g., location and / or orientation) of the third device, a position (e.g., location and / or orientation) relative to the first device and / or another object in the environment, and / or a value of sensor data from a sensor of the third device) (e.g., different from the first sensor data). In some examples, the third device is a different type of device than the second device.

[0122] After (712) capturing the media data, after receiving the first sensor data, and after receiving the second sensor data, in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied based on the media content and the first sensor data (and / or the second sensor data), the first device performs (714) a first operation (e.g., 508) (e.g., display content, output audio, open an application, output haptic signal, and / or perform a function) corresponding to the second device (and / or a first user).

[0123] After (712) capturing the media data, after receiving the first sensor data, and after receiving the second sensor data, in accordance with a determination that a second set of one or more criteria, different from the first set of one or more criteria, is satisfied, wherein thesecond set of one or more criteria includes a criterion that is satisfied based on the media content and the second sensor data (and / or the first sensor data), the first device performs (716) a second operation (e.g., display content, output audio, open an application, output haptic signal, and / or perform a function) corresponding to the third device (and / or a second user different from the first user) (e.g., the third operation does not correspond to the second device and / or the first user). In some examples, the second operation is the same operation as the first operation except that the second operation is catered to, associated with, and / or corresponds to the second user instead of the first user.

[0124] In some examples, the first sensor data (e.g., 506) includes Inertial Measurement Unit (IMU) data (e.g., orientation, motion, acceleration, and / or speed data) (e.g., corresponding to the second device (e.g., wearable device such as fitness tracking watch, fitness tracking ring, and / or a head-mounted display (HMD)) and / or corresponding to an IMU in communication with and / or included in the second device). In some examples, the second sensor data (e.g., 504) includes IMU data (e.g., corresponding to the third device (e.g., wearable device such as fitness tracking watch, fitness tracking ring, and / or a head-mounted display (HMD)) and / or corresponding to an IMU in communication with and / or included in the second device).

[0125] In some examples, the first sensor data (e.g., 506) includes biometric data (e.g., heart rate, facial scan, iris scan, and / or fingerprints) (e.g., corresponding to a user of the second device). In some examples, the second sensor data (e.g., 504) includes biometric data (e.g., heart rate, facial scan, iris scan, and / or fingerprints) (e.g., corresponding to a user of the third device).

[0126] In some examples, after sending the request for sensor data, the first device receives, from the second device (e.g., 402) (e.g., in conjunction with, such as before, while, in the same message, in a separate message, and / or after, receiving the first sensor data), a first identifier (e.g., of a user and / or the second device such as User ID, IP address, account information, and / or serial number) (e.g., corresponding to the second device and / or a user of the second device), wherein data used to perform the first operation (e.g., 508) corresponds to (e.g., is accessed by and / or is identified using) the first identifier (e.g., the first device performs the first operation using data corresponding to the first identifier). In some examples, after sending the request for sensor data, the first device receives, from the third device (e.g., 404) (e.g., in conjunction with, such as before, while, in the same message, in aseparate message, and / or after, receiving the second sensor data), a second identifier (e.g., of a user and / or the third device such as User ID, IP address, account information, and / or serial number) (e.g., corresponding to the third device and / or a user of the third device) different from the first identifier, wherein data used to perform the second operation corresponds to (e.g., is accessed by and / or is identified using) the second identifier (e.g., and does not correspond to the first identifier) (e.g., the first device performs the second operation using data corresponding to the second identifier).

[0127] In some examples, the first sensor data (e.g., 506) includes audio data (e.g., a verbal utterance, a sound, an audible request, an audible command, and / or an audible statement) (e.g., captured via a microphone), visual input (e.g., a pose of a user or an air gesture) (e.g., captured via a camera), or a combination thereof. In some examples, the second sensor data (e.g., 504) includes audio data (e.g., a verbal utterance, a sound, an audible request, an audible command, and / or an audible statement) (e.g., captured via a microphone), visual input (e.g., a pose of a user or an air gesture) (e.g., captured via a camera), or a combination thereof. In some examples, the first sensor data and / or the second sensor data includes multimedia data.

[0128] In some examples, the criterion of the first set of one or more criteria is satisfied in accordance with a determination that a state of (and / or a body position of) a user in the media content matches the first sensor data (e.g., 506) (e.g., a determination is made that the user and the first sensor data correspond to each other). In some examples, the criterion of the second set of one or more criteria is satisfied in accordance with a determination that the state of (and / or a body position of) the user in the media content matches the second sensor data (e.g., 504) (e.g., a determination is made that the user and the first sensor data correspond to each other). In some examples, the state of the user matches the first sensor data when the first sensor data indicates movement (e.g., heart rate is above a threshold) and the state of the user in the media content is moving. In some examples, the state of the user matches the second sensor data when the second sensor data indicates movement (e.g., heart rate is above a threshold) and the state of the user in the media content is moving. In some examples, the state of the user matches the first sensor data when the first sensor data indicates standing still (e.g., heart rate is below a threshold) and the state of the user in the media content is standing still. In some examples, the state of the user matches the second sensor data when the secondsensor data indicates standing still (e.g., heart rate is below a threshold) and the state of the user in the media content is standing still.

[0129] In some examples, performing the first operation (e.g., 508) corresponding to the second device (e.g., 402) includes outputting media content (e.g., 508) corresponding to a user of (e.g., using and / or being within a predetermined proximity to) the second device (e.g., and not a user of the third device). In some examples, performing the first operation corresponding to the third device includes outputting media content corresponding to a user of (e.g., using and / or being within a predetermined proximity to) the third device (e.g., and not a user of the second device).

[0130] In some examples, performing the first operation (e.g., 508) corresponding to the second device (e.g., 402) includes outputting media content (e.g., 508) at a location within proximity to a user of the second device (e.g., and not within proximity to a user of the third device). In some examples, performing the first operation corresponding to the third device includes outputting media content at a location within proximity to a user of the third device (e.g., and not within proximity to a user of the second device). In some examples, performing the first operation corresponding to the second device includes outputting content at a device (e.g., out of the plurality of other devices) most proximate to the user of the second device.

Claims

CLAIMSWhat is claimed is:

1. A method, comprising: at a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device: detecting, via the one or more input devices, one or more characteristics of a subject; and in response to detecting the one or more characteristics of the subject: in accordance with a determination that the one or more characteristics of the subject satisfies a first set of one or more criteria, causing the second electronic device to perform a first operation; and in accordance with a determination that the one or more characteristics of the subject do not satisfy the first set of one or more criteria, forgoing causing the second electronic device to perform the first operation.

2. The method of claim 1, wherein the first set of one or more criteria includes a criterion based on one or more of a location, position, orientation, and movement of the subject.

3. The method of any one of claims 1-2, wherein the one or more input devices comprises one or more sensors, wherein the one or more sensors comprises one or more of a visual, aural, light, and motion sensors.

4. The method of any one of claims 1-3, wherein causing the second electronic device to perform the first operation includes causing the second electronic device to output a notification in place of the first electronic device outputting the notification.

5. The method of any one of claims 1-4, wherein causing the second electronic device to perform the first operation includes causing the second electronic device to output a notification and wherein the method further comprises causing the first electronic device to output a notification, wherein the notification output by the first electronic device and the notification output by the second electronic device have common content.

6. The method of any one of claims 1-5, wherein the second electronic device is in communication with a display generation component, the wherein causing the first operation comprises: displaying, via the display generation component, a representation of the data detected using the one or more input devices of the first electronic device.

7. The method of any one of claims 1-6, wherein the second electronic device is a wearable or handheld device.

8. The method of any one of claims 1-7, wherein the second electronic device is logged into a same user account as the first electronic device.

9. The method of any one of claims 1-8, wherein causing the second electronic device to perform the first operation includes sending, to the second electronic device, an indication of the one or more characteristics.

10. The method of any one of claims 1-9, wherein the first set of one or more criteria includes a criterion that is satisfied when a determination is made that a second threshold amount of a second characteristic of the subject is detected in less than a predetermined period of time, and wherein the first set of one or more criteria includes a criterion that is not satisfied when a determination is made that the second threshold amount of the second characteristic of the subject is not detected in less than the predetermined period of time.

11. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device, the one or more programs including instructions for performing the method of any one of claims 1-10.

12. A first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device, comprising: one or more processors; andmemory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 1-10.

13. A first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device, comprising: means for performing the method of any one of claims 1-10.

14. A computer program product, comprising one or more programs configured to be executed by one or more processors of a first electronic device in communication with one or more input devices and a second electronic device different from the first electronic device, the one or more programs including instructions for performing the method of any one of claims 1-10.

Images