Method and system for audio bridging with an output device

Abstract

The present disclosure relates to methods and systems for audio bridging with output devices. Disclosed herein is a method performed by a first electronic device comprising a first speaker, the method comprising: receiving, via a network, a representation of audio content; while a second electronic device is playing back the audio content through a second speaker, determining that the first electronic device is moving away from the second electronic device; and in response to determining that the first electronic device is moving away from the second electronic device, playing back the audio content through the first speaker using the representation of the audio.

Description

Technical Field

[0001] One aspect of this disclosure relates to a system for bridging audio playback between one or more playback devices and an output device for a user. Other aspects are also described. Background Technology

[0002] A headset is an audio device that includes a pair of speakers, each placed in the user's ear when worn on or around the user's head. Similar to a headset, headphones (or in-ear headsets) are two separate audio devices, each with a speaker that inserts into the user's ear. Headsets and headphones are typically wired to a separate playback device, such as a digital audio player, which drives each speaker of the device with an audio signal to generate sound (e.g., music). Headsets and headphones provide a convenient way for users to listen to audio content privately without having to broadcast it to others nearby. Summary of the Invention

[0003] One aspect of this disclosure is a method performed by a first electronic device, such as headphones, including a first speaker. The first device receives a representation of audio content via a computer network (e.g., the Internet). While a second electronic device is playing back the audio content through a second speaker, the first device determines that it is moving away from the second electronic device. In response to determining that the first electronic device is moving away from the second electronic device, the representation of the audio content is used to play back the audio content through the first speaker.

[0004] In one aspect, the representation of the audio content includes playback data indicating a playback state of the audio content at a second electronic device, and playing back the audio content using the representation includes using the playback data to synchronize the playback of the audio content performed by the first electronic device with the playback state. In another aspect, the method further includes determining the acoustic time of flight (ToF) of the sound generated by a second speaker, the playback state including a timestamp of the portion of the audio content to be played back by the second electronic device, and using the playback data to synchronize the playback includes playing back that portion of the audio content via the first speaker according to the timestamp while taking into account the acoustic ToF, such that the sound of that portion of the audio content generated by the second speaker of the second electronic device and the sound of that portion of the audio content generated by the first speaker of the first electronic device are synchronized when perceived by a user of the first electronic device. In some aspects, the first device determines the acoustic time of flight of the sound generated by the second speaker, wherein that portion of the audio content is played back according to the timestamp while taking into account the acoustic time of flight.

[0005] In one aspect, the first electronic device plays back the audio content after the second electronic device plays it back. In another aspect, the playback by both the first and second electronic devices is perceived as synchronous by the user holding or wearing the first electronic device, while the first and second electronic devices play back the audio content asynchronously.

[0006] In one aspect, the first device determines a target sound level of the audio content based on a representation of the audio content, and determines the sound level of the audio content played back by the second electronic device at the microphone of the first electronic device, wherein playing back the audio content through a first speaker using the representation of the audio content includes: playing back the audio content through the first speaker at a level that satisfies the target sound level based on the sound level. In some aspects, playing back the audio content through the first speaker at a level that satisfies the target sound level includes: adjusting the level that satisfies the target sound level to compensate for the change in sound level as the first electronic device moves away from the second electronic device, based on the determination that the sound level of the audio content at the microphone has changed. In another aspect, adjusting the level that satisfies the target sound level includes: applying a volume adjustment to the first electronic device based on the difference between the sound level and the change in sound level. In one aspect, increasing the level that satisfies the target sound level as the first electronic device moves away from the second electronic device.

[0007] In one aspect, based on determining that the first electronic device is moving toward the second electronic device, the first device reduces the sound output level of the first speaker. In another aspect, using the audio representation to play back audio content includes: driving the first speaker with an audio signal having the audio content, wherein reducing the sound output level of the first speaker includes: attenuating the signal level of the audio signal at the first speaker based on changes in the sound level of the audio content played back by the second electronic device at the microphone of the first electronic device as the first electronic device moves toward the second electronic device. In some aspects, based on determining that the first electronic device has moved within a threshold distance of the second electronic device, the first device stops the playback of audio content through the first speaker by ceasing to drive the first speaker with the audio signal.

[0008] In one aspect, based on determining that the first electronic device is moving toward a third electronic device that is playing audio content through a third speaker, the first device reduces the sound output level of the first speaker. In some aspects, the first electronic device determines the position of a second electronic device relative to the first electronic device; and based on that position, spatially renders audio content to generate a virtual sound source including the audio content through the first speaker. In another aspect, the first electronic device is communicatively coupled to the second electronic device via a wireless connection, and determining that the first electronic device is moving away from the second electronic device includes: identifying the position of the first electronic device relative to the second electronic device based on a Received Signal Strength Indicator (RSSI) of the wireless connection; and determining that the first electronic device is moving away from that position based on changes in the RSSI. In some aspects, the first device determines the sound level of the audio content played back by the second electronic device at the microphone of the first electronic device, and determining that the first electronic device is moving away from the second electronic device includes: detecting that the sound level is decreasing at a specific rate.

[0009] In one aspect, the first electronic device is a wearable device. In another aspect, the wearable device is a pair of smart glasses, and the first speaker is an over-ear speaker. In another aspect, the first electronic device is a pair of headphones. In some aspects, the second electronic device is a smart speaker. In another aspect, the second electronic device is a television. In one aspect, the representation of the audio content includes the audio content. In another aspect, the representation of the audio content includes an identifier of the audio content. In some aspects, playing back the audio content using the representation of the audio content includes: using the identifier of the audio content to retrieve an audio signal from a remote electronic server or the local memory of the first electronic device, wherein the audio signal includes the audio content; and using the audio signal to drive the first speaker to produce sound of the audio content.

[0010] The above overview does not constitute an exhaustive list of all aspects of this disclosure. It is contemplated that this disclosure encompasses all systems and methods that can be practiced by all suitable combinations of the aspects outlined above and those disclosed in the detailed embodiments below and specifically pointed out in the claims. Such combinations may have specific advantages not specifically set forth in the foregoing summary. Attached Figure Description

[0011] Multiple aspects are illustrated in the accompanying drawings by way of example rather than limitation, and similar reference numerals in the drawings indicate similar elements. It should be noted that references to "a" or "an" aspect in this disclosure do not necessarily refer to the same aspect, and each refers to at least one. Furthermore, for the sake of brevity and to reduce the total number of drawings, a single drawing may be used to illustrate features of more than one aspect, and for a particular aspect, not all elements in that drawing may be necessary.

[0012] Figure 1 The system is illustrated in several stages, in which the output device is operating as an audio bridging device that is playing back the same audio content being played back by the playback device, so as to maintain the sound level of the audio content heard by the user when the user moves away from the playback device.

[0013] Figure 2 A system comprising a playback device and an output device that are communicatively coupled to each other is shown according to one aspect.

[0014] Figure 3 A block diagram is shown of an output device that is being bridged with a playback device for audio playback.

[0015] Figure 4 This is a flowchart of one aspect of the process of bridging audio playback between the output device and the playback device when the output device moves away from the playback device.

[0016] Figure 5 This is a flowchart of one aspect of the process of bridging audio playback between the output device and the playback device as the output device moves toward the playback device.

[0017] Figure 6 This is a flowchart of one aspect of the process of bridging audio playback between output and playback devices.

[0018] Figure 7 It illustrates several stages according to one aspect, wherein the output device maintains the sound level heard by the user as the user moves between two separate playback devices that are playing back audio content. Detailed Implementation

[0019] Various aspects of this disclosure will now be explained with reference to the accompanying drawings. Unless the shape, relative position, and other aspects of the components described in any aspect are explicitly defined, the scope of this disclosure is not limited to the components shown, which are for illustrative purposes only. Furthermore, while numerous details have been set forth, it should be understood that some embodiments may be implemented without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of the description. Moreover, unless the meaning is explicitly contrary, all scopes shown herein are to be considered to include the endpoints of each scope.

[0020] Today, many consumer products exist that play back audio content (e.g., music, podcasts, etc.) into the surrounding environment. For example, products such as smart speakers can be linked to online music streaming platforms that allow the smart speaker to stream music. A person can purchase a smart speaker and position it in their home where the listener can best enjoy the music played back by the speaker (e.g., in the kitchen, living room, bedroom, etc.). However, the sound output may be limited to a specific range, which may be based on the smart speaker's equipment limitations (e.g., the size of the smart speaker's speaker drivers, power capacity, etc.) and / or the physical environment (e.g., the size and shape of the room in which the sound is played). For example, when placed in the kitchen, a listener may be able to hear the sound output while cooking, but may not be able to hear it (or may be able to hear it faintly) in the adjacent living room. Therefore, when a person moves around the house (e.g., between the kitchen and the adjacent living room), this person may hear the sound produced by the smart speaker intermittently, which may adversely affect their listening experience because they will only hear a portion of the audio content. This may be especially true when listening to podcasts or audiobooks, where listeners may miss important (or relevant) parts of the podcast or audiobook while going in or out of the kitchen.

[0021] To address this issue, this disclosure describes an output device (e.g., headphones) that bridges audio playback to a playback device (e.g., a smart speaker) to provide a consistent listening experience to a user. For example, while audio content is being played back by a smart speaker, the output device (which may be worn or held by a user near the playback device) can determine that it is moving away from the playback device. For instance, the output device may be wirelessly coupled to the playback device, and the determination of its movement can be based on a Received Signal Strength Indicator (RSSI) of the wireless connection. Alternatively, the determination may be based on a decreasing (or weakening) sound level (e.g., captured by the output device's microphone), indicating that the output device is moving away. In response to determining that the output device is moving away from the playback device, the output device can move away from the played-back audio content as well. In this case, the sound produced by the output device can compensate for the perceived decrease in sound produced by the playback device by the user of the output device, caused by the user moving away from the playback device. Therefore, the output device can maintain the user-perceived audio playback, resulting in a consistent and pleasant listening experience.

[0022] Figure 1The diagram illustrates three phases 1-3 of system 4, in which an output device 6 (e.g., audio) worn by user 10 operates as an audio bridging device. This audio bridging device is configured to play back the same audio content being played back by playback device 5, so as to maintain the sound level of the audio content heard by the user as the user moves away from playback device 5. As described herein, an “audio bridging device” can be any electronic device configured to play back the same (similar or different) audio content being played back (e.g., played back to the surrounding environment) by one or more playback devices (e.g., loudspeakers), so that the bridging device compensates for changes in the audio playback of playback device 5 perceived by the user as they move away from and / or toward playback device 5 (e.g., changes in the sound level of the audio content being played back by the playback devices). In other words, output device 6 compensates for changes in the apparent loudness of the played-back audio content perceived by the user. Further details on how output device 6 bridges audio playback are described herein.

[0023] As shown in the figure, each stage of this diagram illustrates a playback device 5, shown as (e.g., a standalone) loudspeaker, and a user 10 wearing an output device 6, shown as a headset (e.g., a backless headset) worn on the user's head. As shown, the playback device 5 is playing back audio content (e.g., the audio content is shown as a cable extending away from the device). Specifically, the playback device 5 may be using one or more audio signals to drive one or more speakers (e.g., integrated within the housing of the playback device 5) to generate (or project) sound from the audio signals (the audio content contained within the audio signals) into the surrounding environment (e.g., the room 7 where the playback device 5 is located), each of the one or more audio signals having at least a portion of the audio content. In one aspect, the audio content being played back by the speakers may be a piece of audio content desired by the user, such as musical works, podcasts, audiobooks, movie soundtracks, etc. In one aspect, the content may be “user-expected,” such that system 4 (e.g., its playback device 5) has received user input (e.g., via voice command, selection of a physical button, etc.) for (e.g., to begin) playing back audio content through the speakers of the playback device. In another aspect, playback device 5 may begin playback in response to an instruction received from another electronic device communicatively coupled to the device. For example, playback device 5 may receive an instruction from output device 6 for playing back audio content, which may have already received user input (e.g., via voice command). In one aspect, playback device 5 may be streaming audio content (e.g., via the Internet) and / or may be retrieving content from the device’s local memory or from a remote storage device (e.g., a remote server). Further details regarding how playback device 5 plays back audio content are described herein.

[0024] As shown in the figure, the headset includes a speaker 8 and a microphone 9 (which are part of or integrated into the left shell or earcup of the headset). The speaker is an "out-of-ear" speaker arranged to project sound into the surrounding environment. In one aspect, the headset may be arranged to allow a user to hear sounds from the surrounding environment and / or sounds produced by the out-of-ear speaker. Specifically, the headset may be designed to allow sound to pass through the earcups and enter the user's ears. For example, the headset may be a backless headset (e.g., having one or more openings) that allows sound from the surrounding environment to pass through the headset (e.g., its shell) and enter the user's ears.

[0025] On the other hand, output device 6 may perform one or more audio signal processing operations to allow a user to hear ambient sounds. In this case, speaker 8 may be an "internal" speaker disposed inside the housing (e.g., earmuffs) of output device 6 and arranged to project sound into (or toward) the user's ears. Output device 6 may perform a transparency function, wherein the sound reproduced by one or more internal speakers of output device 6 is a reproduction of ambient sounds captured by the device's microphone in a "transparent" manner (e.g., as if the user were not wearing output device 6). Output device 6 (e.g., its controller, such as...) Figure 2 The output device 6 (shown) can process at least one microphone signal captured by a microphone and filter the signal through a transparency filter. This reduces acoustic blockage caused by the audio output device 6 being located on, in, or above the user's ear, while preserving spatial filtering effects based on the wearer's anatomical features (e.g., head, auricle, shoulders, etc.). The filter also helps preserve timbre and spatial cues associated with actual ambient sounds. In one aspect, the filter for the transparency function can be user-specific, based on specific measurements of the user's head. For example, the output device 6 can determine the transparency filter based on the head-related transfer function (HRTF) or the equivalent head-related impulse response (HRIR) based on the user's anthropometric measurements. Thus, the user can hear the sound produced by the playback device 5 and / or the speaker 8 via at least a portion of the output device 6.

[0026] Furthermore, each stage shows several sound levels (e.g., sound pressure level (SPL) dB) of the sound produced by the two devices as perceived by the user. Specifically, each stage shows the sound level 11 of the playback device 5 as heard by the user 10 (or by a listener at the listener's location) and the sound level 12 of the output device 6 as heard by the user 10. In one aspect, these two levels represent the sound pressure at (or near) the user's ears (or both ears) of the sound produced by the two devices. In another aspect, these levels may represent the sound pressure level measured (or perceived) by one or more microphones (e.g., microphone 9) of the output device 6. In yet another aspect, these levels represent the amount (e.g., percentage) of the sound produced by the respective device as perceived by the user 10. In some aspects, the sound level 12 may be the same as the sound output level of the speaker 8. In yet another aspect, the sound level 12 may be less than the sound output level of the speaker because the speaker 8 is at a distance from one or more of the user's ears. In this case, the speaker's sound output level can be higher than the sound output level perceived by the user in order to compensate for the distance between the user's ear and the speaker (e.g., its diaphragm).

[0027] Phase 1 illustrates a user 10 wearing output device 6 near playback device 5 (e.g., within the threshold distance of playback device 5) and primarily listening to sound being played back by playback device 5 in room 7. Specifically, the user listens only (or primarily) to playback device 5, while output device 6 is not producing any (or producing very little) sound (e.g., the audio content being played back by playback device 5). This is indicated by the high sound level 11 of playback device 5 (e.g., at the maximum sound level threshold) and the low sound level 12 (e.g., below the minimum sound level threshold). In one aspect, sound level 12 in this phase may indicate that output device 6 is not producing any sound of the audio content being played back by playback device 5. In another aspect, while output device 6 may not be playing back audio content, the device may alternatively produce other sounds.

[0028] In one aspect, sound level 11 can be the target sound level perceived by user 10. Specifically, this can be the level at which the listener expects to hear the sound being produced by playback device 5. In another aspect, the target sound level can be defined when playback device 5 begins audio playback. For example, the target sound level can correspond to the volume level of playback device 5 when the device begins outputting sound. In another aspect, the target sound level can be a sound level measured from a microphone signal captured by microphone 9. For example, the sound level can be measured once playback device 5 begins playback, as described herein. Furthermore, the sound level can be measured based on user input (e.g., at output device 6). Further details regarding the target sound level are described herein.

[0029] Phase 2 illustrates that user 10 has moved away from playback device 5 (e.g., beyond a threshold distance), but both remain in the same room (e.g., the user may be moving towards a door to leave the room). Specifically, the user is moving away from playback device 5, while playback device 5 continues to play back audio content. Due to the increasing distance, the sound level 11 decreases (e.g., drops to 25% of the level it was at in Phase 1). In one aspect, when the distance between playback device 5 and the user is doubled, the sound pressure level from the point source can be reduced by at least 50%. For example, if the distance between the user and playback device 5 is doubled between Phase 1 and Phase 2, the sound level can be reduced by at least 6 dB.

[0030] In one aspect, when it is determined that output device 6 (and / or the user) is moving away from playback device 5, output device 6 can be configured to (e.g., begin) play back audio content through speaker 8. Specifically, output device 6 can play back the same audio content as playback device 5 so that the combined sound output of playback device 5 and output device 6 maintains the sound level 11 (e.g., its target sound level) in the first stage 1. To achieve this, the sound produced by playback device 5 and the sound produced by speaker 8 can be synchronized when perceived by user 10 of output device 6. In this case, the user may not be able to distinguish or differentiate the sound produced by playback device 5 and / or the sound produced by output device 6, but will perceive the sounds from both devices as originating from (e.g., the same) sound source. This could be due to constructive interference of the sounds produced by the two devices at the listener's location (or nearby) (or more specifically, at the user's ear).

[0031] As described herein, output device 6 can play back audio content to compensate for the decrease in sound level 11. As shown in the figure, as the user moves away from playback device 5, sound level 11 has decreased compared to when the user was closer to the device in the first stage. In this case, output device 6 adjusts the sound output level of the sound produced by speaker 8 based on the change in sound level 11. For example, output device 6 can (e.g., start audio playback and / or) apply volume adjustment (e.g., increase the volume) to increase the sound output, as shown by the curve emitted from speaker 8 in the figure. Therefore, sound level 12 has been increased from the lower level shown in the first stage 1. In one aspect, the increase can be based on the difference between the target sound level 11 in stage 1 and the (current or new) sound level 11 in the second stage 2. In particular, output device 6 increases sound level 12 proportionally as sound level 11 decreases. Therefore, the combination of sound levels 12 and 11 in the second stage 2 is equal to (or approximately) the sound level 11 in stage 1. Therefore, the user will not perceive a change in (apparent) sound level as the user moves away from playback device 5. This article describes more about how output device 6 compensates for audio output.

[0032] Phase 3 illustrates that the user is no longer located within room 7, which includes playback device 5 (e.g., has moved beyond a threshold distance). Specifically, the user has moved outside building 13, which houses playback device 5, which is continuously playing back audio content. Because the user has moved far from playback device 5, the user cannot hear (or only vaguely hears) the sound level 11 of playback device 5 (e.g., the user has moved beyond the acoustically audible range of playback device 5). Furthermore, the sound level 12 of output device 6 has been increased to compensate for the low sound level of playback device 5, which is shown in this figure as an increase in the number of curves emanating from speaker 8 compared to the number of lines in phase 2. Specifically, the sound level of output device 6 is now the same (or similar) to the sound level 11 in phase 1. Therefore, throughout phases 1-3, the combination of sound levels 11 and 12 is the same (e.g., defined by the target sound level in phase 1), and thus the user perceives a continuous and uninterrupted sound level of the audio content as the user moves away from playback device 5.

[0033] As described so far, when the user moves away from the playback device 5, the output device 6 can increase the sound output to compensate for the decrease in sound level 11. In one aspect, when the user moves toward the playback device 5, the output device 6 can decrease the sound output. In this case, when the user moves toward the playback device 5, the sound level 11 increases, and therefore the output device 6 can decrease the sound output level of the speaker to reduce the sound level 12 perceived by the user.

[0034] Figure 2 A system 4 is illustrated, comprising a playback device 5 and an output device 6 communicatively coupled to each other, according to one aspect. In one aspect, the playback device 5 can be any electronic device configured to play back audio content and / or perform networking operations. As shown, the playback device 5 is a loudspeaker. In another aspect, the playback device 5 can include a standalone speaker, a smart speaker, a home theater system (as part of it), or an infotainment system integrated into a vehicle. In yet another aspect, the playback device 5 can be a desktop computer, a laptop computer, a digital media player, a television, etc. In one aspect, the device 5 can be a portable electronic device (e.g., handheld), such as a tablet computer, a smartphone, etc.

[0035] As shown, the playback device 5 includes a controller 20, a network interface 22, and a speaker 21. In one aspect, the playback device 5 may include more or fewer components, such as having two or more speakers. In one aspect, the network interface 22 is configured to establish (e.g., wirelessly) a communication link (or connection) with one or more other electronic devices, such as an output device 6, to exchange digital data. In one aspect, the speaker 21 may be, for example, an electrically driven driver specifically designed for sound output in a particular frequency band, such as a woofer, tweeter, or midrange driver. In one aspect, the speaker 21 may be a “full-range” (or “full-frequency”) electrically driven driver that reproduces as much of the audible frequency range as possible. In one aspect, the speaker 21 is an external speaker configured to output sound to the surrounding environment. In one aspect, the speaker 21 may be an “in-device” speaker integrated into the playback device 5 (e.g., its housing). For example, when the playback device 5 is a television set, the device may include one or more speakers integrated into the television set.

[0036] Controller 20 may be a dedicated processor such as an application-specific integrated circuit (ASIC), a general-purpose microprocessor, a field-programmable gate array (FPGA), a digital signal controller, or a set of hardware logic structures (e.g., filters, arithmetic logic units, and dedicated state machines). The controller is configured to perform audio signal processing operations and / or networking operations. For example, controller 20 may be configured (e.g., via network interface 22 through network 23) to retrieve audio content (e.g., one or more audio signals including the audio content) and use the audio signals to drive speaker 21 to output sound of the audio content. On the other hand, the controller is configured to perform networking operations, such as communicating with output device 6 (via network 23). Further details regarding the operations performed by controller 20 are described herein.

[0037] like Figure 1As shown, output device 6 can be a headset designed to be worn on or by a listener (e.g., user 10) (e.g., on their head). On the other hand, output device 6 can be any electronic device including at least one speaker (and at least one microphone) and configured to play back audio content by driving the speaker with one or more audio signals. For example, device 6 can be a wireless headset (e.g., an in-ear headset or earbud) designed to be positioned on (or in) a user's ear and designed to output sound into the user's ear canal. In some aspects, the headset can be a sealed type with a flexible earpiece end designed to acoustically seal the entrance to the user's ear canal relative to the surrounding environment by blocking or occluding it within the ear canal. In this case, output device 6 can include a left earpiece for the user's left ear and a right earpiece for the user's right ear. In this case, each earpiece can be configured to output at least one audio channel of media content (e.g., the right earpiece outputs the right audio channel of a stereo recording (such as a musical work) with two-channel input and the left earpiece outputs the left audio channel). On the other hand, the output device 6 can be any electronic device that includes at least one speaker and is arranged to be worn by a user and to output sound by driving the speaker with an audio signal. Alternatively, the output device 6 can be any type of headset, such as over-ear (or over-ear) headphones, that at least partially covers the user's ears and is arranged to direct sound to the user's ears.

[0038] On the other hand, output device 6 can be any type of wearable electronic device configured to play back audio content. For example, output device 6 could be a pair of smart glasses or a smartwatch. On the other hand, output device 6 can be a device similar to those described with respect to playback device 5. For example, output device 6 could be a smartphone. On the other hand, output device 6 can be a hearing aid configured to generate amplified ambient sound into a user's ear (e.g., ear canal).

[0039] As shown in the figure, the output device 6 includes a controller 24, one or more sensors 26, including a microphone 9, a camera 28, an inertial measurement unit (IMU) 29, a speaker 8, and a display screen 27. In one aspect, the output device 6 may include more or fewer components. For example, the output device 6 may include more sensors (e.g., temperature sensors, accelerometers, proximity sensors, etc.). In another aspect, the output device 6 may include two or more components, such as having two or more microphones, speakers, and / or displays.

[0040] In one aspect, one or more sensors 26 are configured to detect an environment (e.g., where the output device 6 is located) and generate sensor data based on the environment. The microphone 9 can be any type of microphone (e.g., a differential pressure gradient microelectromechanical system (MEMS) microphone) configured to convert acoustic energy caused by sound waves propagating in the acoustic environment into a microphone signal. As described herein, the microphone 9 can be a (e.g., a reference) microphone arranged to sense ambient sound. In another aspect, the microphone 9 can be an error (or internal) microphone arranged to capture sound within a user's ear canal when the user wears the output device 6. In some aspects, the output device 6 may include at least one of two types of microphones.

[0041] In one aspect, camera 28 is a complementary metal-oxide-semiconductor (CMOS) image sensor capable of capturing digital images including image data representing the camera's field of view, where the field of view includes the scene of the environment in which device 6 is located. In some aspects, the camera may be a charge-coupled device (CCD) camera type. The camera is configured to capture still digital images and / or video represented by a series of digital images. In one aspect, the camera may be positioned anywhere around the device. In some aspects, the device may include multiple cameras (e.g., where each camera may have a different field of view). IMU 29 may be an electronic device designed to measure the position and / or orientation of output device 6.

[0042] Display screen 27 (or monitor) is designed to present (or display) digital images or videos of video (or image) data. In one aspect, display screen 27 may use liquid crystal display (LCD) technology, light-emitting polymer display (LPD) technology, or light-emitting diode (LED) technology, but in other aspects, other display technologies may be used. In some aspects, display screen 27 may be a touch-sensitive display screen configured to sense user input as an input signal. In some aspects, the display may use any touch sensing technology, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies.

[0043] As described herein, each of these devices may include one or more components. In one aspect, at least some of these components may be part of (or integrated into) the housing of each respective device. In another aspect, any device may include one or more components as described herein. For example, playback device 5 may include one or more displays, one or more microphones, and / or one or more cameras. In yet another aspect, instead of integrating components into each device, one or more components may be separate electronic devices communicatively coupled to the controller (e.g., via a network interface). For example, microphone 9 may be (e.g., wirelessly) communicatively coupled to controller 24, which transmits one or more microphone signals (as audio digital data) to the controller.

[0044] In one aspect, output device 6 can be configured to communicatively couple with playback device 5 via network 23, such that the two devices can be configured to communicate with each other. In one aspect, the network can be any type of computer network, such as a wide area network (WAN) (e.g., the Internet), a local area network (LAN), etc., through which the devices can exchange data with each other and / or with one or more other electronic devices, such as remote electronic servers. In another aspect, the network can be a wireless network such as a wireless local area network (WLAN), a cellular network, etc., for exchanging digital (e.g., audio) data. Regarding cellular networks, output device 6 can be configured to establish wireless (e.g., cellular) calls, wherein the cellular network may include one or more cell towers, which may be part of a communication network (e.g., a 4G Long Term Evolution (LTE) network) that supports data transmission (and / or voice calls) of electronic devices such as mobile devices (e.g., smartphones). In another aspect, the devices can be configured to wirelessly exchange data via other network connections such as a wireless personal area network (WPAN). For example, output device 6 can be configured to establish a wireless connection with playback device 5 via a wireless communication protocol (e.g., Bluetooth or any other wireless communication protocol). During the establishment of a wireless connection, the devices may exchange (e.g., transmit and receive) data packets (e.g., Internet Protocol (IP) packets) containing digital (e.g., audio) data, which may include a representation of the audio content being played back by the playback device 5.

[0045] As described herein, controllers 20 and / or 24 are configured to perform digital signal processing operations, such as audio signal processing operations and networking operations. In one aspect, the operations performed by the controllers may be implemented in software (e.g., as instructions stored in memory and executed by the controllers) and / or may be implemented by hardware logic structures as described herein.

[0046] Figure 3A block diagram is shown of an output device 6 that is bridging audio playback with playback device 5. Specifically, output device 6 is playing back audio via speaker 8 and is also playing back audio via playback device 5 (e.g., via speaker 21, as shown). Figure 2 The audio content (as shown) is used to maintain the (e.g., target) sound level of the audio content perceived by the user of the output device 6. In one aspect, the operations described herein can be performed when a user 10 holding or wearing the output device 6 (e.g., about to or currently) moves away from the playback device 5 (or toward the playback device).

[0047] As shown, playback device 5 is playing back audio content by driving one or more speakers (e.g., speaker 21) with one or more audio signals that include a segment of audio content. In one aspect, playback device 5 may be playing back audio content based on user instructions. For example, playback device 5 may have received user input (e.g., from user 10 of output device 6) to initiate playback. For example, playback device 5 may have received user input from one or more input devices such as one or more (e.g., physical) buttons of playback device 5. In another aspect, playback device 5 may receive a user's voice command for playing back audio content (e.g., captured by the microphone of playback device 5). In this case, the playback device (e.g., its controller 20) may analyze the microphone signal to detect speech contained therein. Once detected, the controller may determine whether the speech includes (e.g., a voice command for playing back audio content). If so, playback device 5 may begin playback. On the other hand, user input can be received by the user selecting a user interface (UI) item in the graphical user interface (GUI) displayed on the screen (not shown), which, when selected, transmits control signals to the controller to play back the audio content.

[0048] As described so far, playback device 5 may receive user input via one or more input devices coupled to playback device 5. Alternatively, user input may be received from another electronic device communicatively coupled to playback device 5. For example, output device 6 may receive user input to instruct playback device 5 (e.g., to start) audio playback. Returning to the previous example, a user may select a UI item in the GUI displayed on display screen 27. Once selected, output device 6 may transmit control messages (e.g., via network 23) to playback device 5, thereby instructing controller 20 to start (or resume) streaming audio content (e.g., via network 23) for playback.

[0049] Controller 24 includes one or more operation blocks for performing audio signal processing operations for bridging audio content playback with playback device 5. For example, the controller includes an echo canceller 31, a playback synchronizer 32, a sound level estimator 33, a content extractor 34, and an audio renderer 35. As shown, controller 24 is configured to receive playback data 30 from playback device 5 (via network 23). For example, when (or before) playback of audio content by playback device 5, the device may establish (e.g., wireless) a connection with output device 6 and transmit the playback data as one or more data packets (e.g., Internet Protocol (IP)). In one aspect, the playback data may be (or include) a representation of the audio content. In particular, the data may include metadata describing the audio content, such as an identifier for the audio content. For example, when the audio content is a musical work, the identifier may describe the work, such as including the name of the musical work, genre, artist, etc. In another aspect, the identifier may be a unique identifier that uniquely identifies the audio content.

[0050] On the other hand, playback data may include the (e.g., current) playback status of audio content being played back by playback device 5. In one aspect, the playback status may indicate whether audio content is currently being played by the playback device, or whether the audio content has been paused or stopped (e.g., based on user input). For example, when playback data indicates that the content has been paused or stopped, output device 6 may also pause or stop playback. On the other hand, the playback status may include one or more timestamps indicating the timing characteristics of the audio content being played back by playback device 5. For example, the playback status may include a content timestamp of a portion (or a future portion) of the audio content to be (or currently) played back by playback device 5. For example, the content timestamp may indicate the playback time relative to the entire playback duration of the audio content (e.g., the timestamp indicates that a portion of the audio content to be played back is at the two-minute mark of a musical piece with a three-minute playback duration).

[0051] On the other hand, the playback status may include a content start timestamp, which indicates the start time (e.g., the moment when playback device 5 and / or output device 6 begin or start playing back audio content). In some aspects, the start time may be relative to (or defined by) a shared clock between the two devices, which allows the two devices to synchronize playback (e.g., when perceived by one or more listeners, as described herein). In one aspect, the two devices may synchronize or share clocks (e.g., internally) via any time synchronization method. For example, to synchronize clocks, the devices may exchange synchronization messages using any time synchronization protocol (e.g., the IEEE 802.1AS protocol), which may be included within or separate from the playback data (e.g., included in the content start timestamp). On the other hand, the devices may synchronize their internal clocks using information obtained by both devices from a Network Time Protocol (NTP) server (e.g., via network 23). In some aspects, the devices may synchronize their clocks in response to playback device 5 receiving user input for initiating (or playing back) audio content.

[0052] In some aspects, the playback state may include a current playback timestamp that indicates the time along a shared clock when the playback device 5 will (or is) playing back a portion of the audio content. Specifically, the current playback timestamp may indicate the portion of the audio content that will be played back relative to the shared clock, which can be associated with the playback state. For example, the current playback state may be associated with a content timestamp along the shared clock, since the current playback timestamp indicates the portion of the audio content that will be played back along the shared clock, which is associated with one or more content timestamps. In one aspect, one or more timestamps in the timestamps described herein may allow the output device 6 to play back synchronously with the playback device 5 (e.g., when perceived by one or more listeners).

[0053] On the other hand, the playback status may indicate other characteristics of the audio content (and / or playback device 5). For example, it may include the volume level (or sound output level) of the audio content being played back by playback device 5. Specifically, the volume level may be a user-defined volume level at which the listener expects to hear the sound output of playback device 5. On the other hand, these characteristics may indicate audio signal processing operations being performed on the audio content being played back (e.g., one or more audio signals of the audio content), such as whether equalization or dynamic range compression is being performed. On the other hand, in addition to (or instead of) including at least some of the data described herein, playback data 30 may also include at least a portion of the audio content being (or will be) played back by playback device 5. For example, playback data may include one or more audio signals of audio content in any audio format (e.g., as digital audio data).

[0054] As described so far, playback data 30 can be received by output device 6 from playback device 5. For example, playback device 5 can begin transmitting playback data 30 once playback begins. In some aspects, playback device 5 can transmit playback data while playing back audio content. In other aspects, at least some of the playback data 30 can be received by the output (and / or playback device 5) through one or more other devices. For example, any device in the device can receive playback data from an electronic remote server that can be configured to stream audio content to the device. In this case, the server can transmit one or more timestamps, metadata about the audio content, and / or characteristics.

[0055] Content extractor 34 is configured to receive playback data 30 and to extract (or retrieve) audio content associated with the playback data. As described herein, the playback data may include an identifier associated with the audio content being played back by playback device 5 and may include a timestamp (e.g., content time) indicating the portion of the audio content to be played back by (or to be played back by) playback device 5. Content extractor 34 may use at least a portion of this information to retrieve the audio content (e.g., one or more audio signals thereof) being played back by (or to be played back by) playback device 5. In one aspect, content extractor 34 may retrieve audio signals from a remote electronic device (e.g., from a remote server via network 23) and / or from the local memory of a first electronic device. In another aspect, content extractor 34 may supply one or more audio signals of the retrieved audio content to audio renderer 35, which may use the one or more audio signals to drive speaker 8 to produce sound from the audio content. Further details regarding audio renderer 35 are described herein.

[0056] Echo canceller (or canceller) 31 is configured to receive at least one microphone signal from microphone 9, including ambient sound captured by the microphone (which may include sound from audio content generated by playback device 5), and is configured to reduce (or cancel) echo linear components from the microphone signal that may be caused by sound generated by speaker 8. As described herein, output device 6 may be configured to play back audio content through speaker 8. In addition to capturing sound generated by playback device 5, the microphone may also capture sound generated by speaker 8. Therefore, echo canceller 31 performs an acoustic echo cancellation process on the microphone signal using the audio signal (driver signal) used by audio renderer 35 to drive speaker 8 as a reference input to produce a linear echo estimate representing an estimate of how much of the driver signal (output by speaker 8) is in the microphone signal generated by microphone 9. In one aspect, canceller 31 determines a linear filter (e.g., a finite impulse response (FIR) filter) and applies the filter to the driver signal to generate an estimate of the linear echo, which is subtracted from the microphone signal. The resulting echo-cancelled signal may include the sound generated by the playback device 5. In some respects, the echo canceller 31 may use any echo cancellation method.

[0057] Playback synchronizer 32 is configured to synchronize playback from output device 6 with playback from playback device 5. Specifically, synchronizer 32 determines (or estimates) a timing alignment for the playback audio content such that the sound of the audio content produced by speaker 8 and the sound from playback device 5 arrive at the user's location at (or approximately) the same time, so that the playback from both devices is synchronized when perceived by the user of output device 6 (e.g., the sounds produced by the two devices mutually interfere with each other). Therefore, controller 24 can use the estimated timing alignment to synchronize (e.g., future) portions of the audio content played back by output device 6 with the same portions played back by playback device 5.

[0058] In one aspect, timing alignment takes into account the time it takes for sound generated by playback device 5 to reach (and / or be heard by) user 10 at output device 6. Specifically, this time is the acoustic time of flight (ToF), which is the time it takes for sound generated by playback device 5 to travel through its surroundings and reach output device 6 (e.g., its microphone 9). Therefore, output device 6 can play back audio content later than playback device 5 according to timing alignment, such that the sound from both devices reaches the user at (approximately) the same time. Thus, the listener perceives synchronized playback from the devices, even though the two devices are actually playing back audio content asynchronously. Further details regarding synchronized playback are described in this document.

[0059] In one aspect, synchronizer 32 is configured to receive at least a portion of playback data 30, which indicates the current playback state of audio content being played back by playback device 5. For example, the playback state may include a current playback timestamp indicating the time along a shared clock of the two devices during which playback device 5 is playing back a portion of the audio content (e.g., a long playback duration of the audio content). In another aspect, synchronizer 32 may receive at least a portion of retrieved audio content (e.g., as at least one audio signal) from content extractor 34. Specifically, playback synchronizer 32 may receive a portion of the audio content associated with the playback data (e.g., its current playback state). For example, the received audio content may be a portion to be played back by playback device 5 according to the current playback state. In one aspect, the received audio content may span a time period (e.g., one second, one minute, etc.) that includes the time along (or begins at) the playback duration of the audio content associated with the received playback data. Specifically, the received audio content may begin at a time associated with a content timestamp related to the current playback state of the playback data. On the other hand, synchronizer 32 may receive (echo-cancelled) microphone signals that include captured ambient sounds (e.g., sounds of the audio content generated by playback device 5).

[0060] In one aspect, synchronizer 32 uses at least some of the received data to determine (or estimate) the acoustic Time-of-Flight (ToF). Specifically, output device 6 may receive playback data 30 indicating that playback device 5 will immediately play back a portion of the audio content relative to a shared clock of the device (e.g., based on a playback state associated with the playback data). However, since the acoustic transmission time is greater than the transmission time over a network (e.g., via Bluetooth), the sound generated by playback device 5 may arrive at output device 6 later than the received playback data. In one aspect, synchronizer 32 may compare (e.g., echo-cancelled) microphone signals (e.g., their spectral content) with the audio signal of audio content retrieved by content extractor 34 to determine whether the spectral content of the audio signal matches (e.g., at least partially) the spectral content of the microphone signal. In one aspect, the match may be determined based on a match of at least partially (e.g., at least within a threshold value) between the compared spectral contents. Upon identification of a match, meaning that the sound generated by playback device 5 has now reached output device 6 (e.g., its microphone), synchronizer 32 can determine the current time of the shared clock. Using the current time, synchronizer 32 can determine the acoustic Time of First Hour (ToF) based on the difference between the current playback timestamp of the playback data and the current time of the shared clock. In one aspect, the acoustic ToF can be the determined difference. For example, the playback status can indicate that a portion of the audio content will be played back at T0 of the shared clock. At T1 after T0, output device 6 can determine that the sound of that portion of the audio content has reached output device 6 (e.g., based on a comparison of the microphone signal with the retrieved audio content). Therefore, in this example, the acoustic ToF can be (or based on) T1-T0.

[0061] On the other hand, the playback synchronizer 32 may determine (or estimate) the acoustic Time of Flight (ToF) using other methods. Specifically, the output device 6 may estimate the acoustic ToF based on a determined (or estimated) distance between the output device 6 and the playback device 5. In one aspect, the synchronizer 32 may determine this distance based on sensor data from one or more sensors 26. For example, the synchronizer 32 may acquire image data captured by a camera and perform object recognition on the image data to determine whether at least a portion of the playback device 5 is within the image data (e.g., within the camera's field of view). In response to determining that the playback device 5 is within the image data, the synchronizer may determine the distance based on the image data. On the other hand, the synchronizer may determine the distance to the playback device 5 based on motion data and / or position data (e.g., from the IMU 29). For example, the sensor 26 may include a Global Positioning System (GPS) sensor (not shown) that generates position data indicating the location of the output device 6. In one aspect, the playback data may include the position data of the playback device 5. In this case, the output device 6 may determine the distance between the devices based on the position data and estimate the acoustic ToF based on the distance. On the other hand, the distance between devices can be determined based on the wireless connection between the two devices. For example, output device 6 can determine its position relative to playback device 5 based on the Received Signal Strength Indicator (RSSI) of the wireless connection.

[0062] On the other hand, the Time of Effect (ToF) can be determined based on the difference between the sound level of the microphone signal and the (target) sound level of the playback data 30. As described herein, sound output can dissipate with respect to distance within the environment. Therefore, the synchronizer 32 can estimate the acoustic ToF based on the difference between the sound level of the playback data (e.g., the volume level of the playback device 5) and the (current) sound level of the sound produced by the playback device 5 captured by the microphone. On the other hand, the playback synchronizer 32 can determine the acoustic ToF using other methods.

[0063] In one aspect, the playback synchronizer 32 can use acoustic Time-of-Flight (ToF) to determine the time alignment for playing back audio content. In another aspect, the time alignment can be the same as the acoustic ToF. In yet another aspect, the time alignment can be based on ToF. For example, in addition to the distance between the microphone 9 and the speaker 8, the time alignment can also take acoustic ToF into account.

[0064] In one aspect, the sound level estimator 33 is configured to maintain a constant (or consistent) sound level (or apparent audio loudness) of the audio content perceived by the user of the output device 6. Specifically, the estimator 33 is configured to determine the target sound level of the audio content that the user will perceive. In one aspect, the estimator 33 may determine the target sound level based on playback data. For example, the estimator 33 may determine the target level as the volume level at which the playback device 5 is (currently) playing back the audio content.

[0065] On the other hand, the target level can be user-defined. For example, the user of output device 6 can define the target level based on user input (e.g., by defining a user-defined volume level). On the other hand, the target sound level can be defined based on the time when playback device 5 has started audio playback. For example, once playback device 5 starts audio playback (e.g., of a specific segment of audio content desired by the user), playback device 5 can transmit (e.g., initial) playback data. Based on this initial playback data, sound level estimator 33 can define the target sound level. On the other hand, the target sound level can be based on the time when playback device 5 has started a specific audio playback session (e.g., based on the time when playback device 5 has turned on and started audio playback).

[0066] On the other hand, the target sound level can be estimated based on the microphone signal from microphone 9. For example, when it is determined that playback has started, the sound level estimator 33 can define the target sound level based on the initial portion of the audio content played back by playback device 5 captured by the microphone. On the other hand, the target sound level can be based on the volume level of playback device 5 and the sound level of the microphone signal (e.g., the relationship between them).

[0067] The sound level estimator 33 receives a microphone signal captured by the microphone 9 (e.g., echo-cancelled) and determines a level adjustment based on the microphone signal and playback data 30. Specifically, the estimator 33 uses the microphone signal to determine the sound level of the audio content played by the playback device 5 and determines (estimates) a level (e.g., volume) adjustment for the output device 6 based on the determined sound level and the target sound level of the playback data. In particular, the estimator 33 may determine a volume adjustment that satisfies (e.g., maintains) the target sound level based on the determined sound level. For example, if the sound level is determined to be lower than the target sound level, the estimator 33 may determine that the volume of the output device 6 should be increased to compensate for the drop in sound level. In particular, the estimator 33 may determine a (e.g., scalar) gain to be applied to one or more audio signals that will be used to drive the audio content of the speaker 8.

[0068] On the other hand, the sound level estimator 33 can determine the volume level to be reduced based on the increase in the sound level of the microphone signal. For example, the estimator 33 can determine that the sound level at the microphone is increasing (e.g., it has increased compared to the previous estimate of the sound level), which may be due to the user moving closer to the playback device 5. Therefore, in order to maintain the target sound level, the estimator 33 can determine the reduction in volume level (e.g., attenuation of the audio signal of the audio content). Thus, the sound level estimator 33 can dynamically adjust the sound output level of the speaker 8 to maintain the target sound level heard by the user of the output device 6.

[0069] Audio renderer 35 is configured to receive audio content (e.g., one or more audio signals including the audio content) from content extractor 34 and is configured to drive speaker 8 using one or more audio signals such that the sound of the audio content and the sound of playback device 5 are simultaneously perceived by the user of output device 6. On the other hand, audio renderer 35 receives a time alignment from playback synchronizer and uses the time alignment to synchronize playback with playback device 5. Specifically, audio renderer 35 may delay the playback of audio content (e.g., and future audio content) for a period of time indicated by the time alignment (e.g., relative to a shared clock). For example, audio renderer 35 may receive an immediate portion of audio content (e.g., as indicated by playback data) played back by playback device 5 and play that portion back after the period of time indicated by the time alignment.

[0070] On the other hand, the audio renderer 35 is configured to receive level adjustment from the sound level estimator 33 and is configured to apply one or more audio signal processing operations to the audio content based on the level adjustment. For example, the audio renderer 35 may apply a scalar gain (or gain value) to at least a portion of the audio signal to adjust (e.g., decrease or increase) the level (or magnitude) of the audio signal. In one aspect, the renderer 35 may apply gain adjustment in the analog domain (e.g., when the signal is an analog signal). In another aspect, gain may be applied in the digital domain (e.g., when the signal is a digital audio signal). In one aspect, the audio renderer 35 may adjust certain portions of the audio signal, such as certain frequencies. In another aspect, the renderer 35 may apply one or more gain values ​​to portions of the input audio signal by performing audio compression operations such as dynamic range compression (DRC). In another aspect, the audio renderer 35 may apply other signal processing operations such as equalization operations (e.g., spectral shaping of the audio signal) to the audio signal based on the level adjustment.

[0071] In one aspect, the audio renderer 35 can render audio content spatially such that the sound generated by the output device 6 is perceived by the user of the device as originating from a location within space. In another aspect, the audio renderer 35 can be configured to determine spatial characteristics (e.g., azimuth, altitude, frequency, etc.) that indicate the location of the sound of the audio content to be reproduced in space (e.g., as a virtual sound source). In another aspect, the audio renderer 35 can determine spatial characteristics to reproduce the sound at the location of the playback device 5. Specifically, the audio renderer 35 can be configured to determine the position of the playback device 5 relative to the output device 6. For example, the renderer 35 can use data from playback data 30 (e.g., position data of the playback device 5) and / or position data determined by the controller 24 of the playback device 5 relative to the output device 6. Based on this data, the renderer 35 can determine (or estimate) the spatial characteristics and can use these characteristics to select one or more spatial filters, such as a head-related transfer function (HRTF) or an equivalent one or more head-related impulse responses (HRIR), which, when applied to the audio signal of the audio content, produce spatial audio (e.g., a binaural rendered audio signal). Therefore, renderer 35 can spatially render audio content based on the position of playback device 5 to generate a virtual sound source including the audio content via speaker 8. In one aspect, output device 6 may include at least one additional speaker, which output device 6 may use to drive the binaural rendered audio signal.

[0072] In some respects, the audio renderer 35 may perform other audio signal processing operations. For example, when the output device 6 includes two or more speakers, the audio renderer 35 may perform sound output beamforming operations to project one or more sounds toward a specific location in space. On the other hand, the renderer 35 may perform active noise cancellation (ANC) functionality to make the speaker 8 produce noise immunity, thereby reducing ambient noise leaking into the user's ears. The ANC function may be implemented as feedforward ANC, feedback ANC, or a combination thereof. Thus, the controller 24 may receive a reference microphone signal from a microphone that captures external ambient sound. On the other hand, the controller 24 may perform any ANC method to produce noise immunity.

[0073] On the other hand, the controller 24 may include a sound pickup beamformer that can be configured to process audio (or microphone) signals generated by two or more external microphones of the output device 6 to form a directional beam pattern (as one or more audio signals) for spatially selective sound pickup in certain directions, so as to be more sensitive to the location of one or more sound sources. For example, the controller may use the sound pickup beamformer to capture sound generated by the playback device 5.

[0074] Figures 4-6 The flowcharts for processes 70, 80, and 90 are respectively used to perform one or more operations to bridge audio playback between output device 6 and playback device 5, such that the sounds of the playback device and the output device are synchronized when perceived by the listener, and that the sound level heard by the listener is maintained as the user moves around playback device 5. In one aspect, at least some of the operations may be performed by one or more devices of system 4, such as... Figure 2 As shown. For example, at least some of the operations in one or more of these processes may be performed by output device 6 (e.g., its controller 24). On the other hand, at least some of the operations may be performed by playback device 5 and / or by another electronic device (e.g., a remote electronic server coupled via network 23) communicatively to either device.

[0075] Figure 4 This is a flowchart of one aspect of a process 70 in which output device 6 bridges audio playback with playback device 5 as output device 6 moves away from playback device 5. Process 70 begins by output device 6 (controller 24) determining (e.g., within the acoustic audible range of output device 6) that playback device 5 is playing back (or will play back) audio content (at box 71). In one aspect, this determination may be based on data obtained from electronic devices (e.g., a remote electronic server) communicatively coupled to both devices. For example, the remote server may obtain location data from one or more playback devices and / or output device 6 and determine whether output device 6 and playback device 5 are within a threshold distance (e.g., within the acoustic audible range). If so, the electronic device may transmit an acknowledgment message indicating that playback device 5 is within the audible range to output device 6. In another aspect, the remote server may transmit (e.g., similar) a message to playback device 5. Once the acknowledgment message is received, the two devices may establish a communication link (e.g., a wireless connection) to communicate with each other.

[0076] In one respect, the remote server can determine that the output device 6 is within the acoustic audible range of the playback device 5 associated with it. For example, the remote server can determine that the devices are within a specific threshold (e.g., corresponding to the acoustic audible range) and determine whether the two devices are associated with the same user or user account (e.g., a cloud-based service). If so, the remote server can communicate with the output device 6 to establish a connection with the playback device 5. In another respect, the remote server can transmit an acknowledgment message to the output device 6 when it determines that the playback device 5 is playing back audio content.

[0077] In one aspect, output device 6 can determine, based on sensor data, that playback device 5 is playing audio content within its acoustic audible range. For example, output device 6 can monitor ambient sounds (captured by microphone 9) to determine whether the sound of the audio content is contained within one or more microphone signals. If so, output device 6 can determine whether playback device 5 is present (e.g., within the acoustic audible range of output device 6). For example, output device 6 can transmit a request for location data of playback devices within range to a remote server. Upon receiving confirmation, output device 6 can establish a communication link with playback device 5. In another aspect, output device 6 can attempt to establish a connection with one or more devices within the area, and upon establishing a connection, determine whether the device is playback device 5, which is playing audio content.

[0078] In one respect, this determination can be made based on user input. For example, output device 6 can make this determination once the device is activated (or turned on) by the user. In another respect, the device can receive user instructions to perform this determination (e.g., based on user input).

[0079] Controller 24 receives a representation of the audio content (at box 72). Specifically, output device 6 may receive a representation from playback device 5. For example, when it is determined (e.g., associated with output device 6) that playback device 5 is playing back audio content, output device 6 establishes a connection with playback device 5 and receives a representation. In one aspect, the representation may be (or include) playback data (e.g., data 30) indicating the playback status of the audio content at playback device 5, as described herein. In another aspect, controller 24 may determine the representation based on sensor data from one or more sensors 26. For example, controller may be configured to capture sound from the environment as one or more microphone signals generated by microphone 9 and may be configured to use the microphone signals to determine the representation. For example, controller may perform spectral analysis on the microphone signals to determine the representation, such as identifying the sound as including musical works generated by the playback device. In another aspect, output device 6 may receive playback data from the playback device. In some aspects, playback data may be received from a different device (e.g., a remote server to which the output device is communicatively coupled via network 23).

[0080] Controller 24 retrieves audio content based on a representation of the audio content (in box 73). For example, content extractor 34 may retrieve at least a portion of the audio content based on playback data of the audio content received via network 23 from playback device 5 (and / or from a remote server). Controller 24 determines the target sound level of the audio content being played back by playback device 5 based on the representation of the audio content (in box 74). For example, sound level estimator 33 may determine the target sound level based on playback data 30 and / or on microphone signals captured by microphone 9.

[0081] The controller determines that output device 6 is moving away from playback device 5 (at box 75). In one aspect, output device 6 may determine that it is moving away based on sensor data. For example, output device 6 may receive motion data from IMU 29 indicating that it is moving. In another aspect, the determination may be based on location data. For example, output device 6 may determine that location data (e.g., from a GPS sensor of output device 6) is changing relative to location data received from playback device 5. In another aspect, output device 6 may determine that it is moving away based on image data obtained from camera 28. In some aspects, output device 6 may determine that it is moving away based on microphone signals captured by microphone 9. For example, output device 6 may determine that the sound level of the audio content being played back by playback device 5 is changing (e.g., decreasing at a specific rate), which could indicate that the devices are moving separately. In another aspect, output device 6 may determine that it is moving away based on a wireless connection established between the devices. For example, output device 6 can determine that it is moving away from playback device 5 by using RSSI identification of the device relative to playback device 5 based on the wireless connection, and by using changes in RSSI to determine that output device 6 is moving away from playback device 5. On the other hand, output device 6 can use any method to determine that it is moving away from playback device 5.

[0082] The controller determines playback characteristics associated with the audio content played back by playback device 5 (at box 76). Specifically, sound level estimator 33 determines the sound level of the sound being produced by playback device 5 at microphone 9 (e.g., using one or more microphone signals captured by microphone 9, e.g., echo-cancelled). In addition to (or instead of) determining the sound level, playback synchronizer 32 determines a timing alignment for synchronizing the playback of output device 6 with the playback of playback device 5. In particular, the playback synchronizer may determine the timing alignment for controller 24 based on playback data (e.g., one or more timestamps therein) and a comparison of microphone signals with the audio content, as described herein. In some aspects, the controller may determine one or more playback characteristics in response to determining that output device 6 has moved (e.g., based on motion data from an IMU). In particular, controller 24 may determine spatial characteristics associated with the audio content played back by playback device 5, such as determining the position of the device relative to the output device when output device 6 moves in space.

[0083] The controller plays back the audio content (at box 77) at a level that meets the target sound level (e.g., increased) based on determined playback characteristics such as sound level and according to time alignment. Specifically, since output device 6 is moving away from playback device 5, output device 6 can determine that the sound level at the microphone is lower than the target sound level. Therefore, in response, output device 6 can adjust the sound output level (e.g., grade) of output device 6 to compensate for the difference between the sound level and the target sound level. For example, to adjust the grade, output device 6 can apply volume adjustment based on the difference between the two grades. In particular, the sound output level can be adjusted by increasing the volume of output device 6. In one aspect, this is done by applying a scalar gain to one or more audio signals of the audio content while taking time alignment into account, and using the audio signals to drive speaker 8.

[0084] On the other hand, the controller can also be configured to spatially render audio content at the location of the playback device based on playback (e.g., spatial) characteristics (as described herein). In particular, the controller 24 can apply one or more spatial filters based on the user's position relative to a determined location of the playback device (e.g., based on IMU sensor data).

[0085] In one aspect, output device 6 can perform at least some of these operations as output device 6 moves away from playback device 5 to provide a consistent listening experience. As described herein, output device 6 can play back audio content through speaker 8 at a level that meets the target sound level. As output device 6 moves away, the sound level at the microphone signal may decrease. Therefore, upon determining that the sound level at the microphone has changed, output device 6 can adjust the output sound level to meet the target sound level to compensate for the change in sound level at the microphone. In other words, at least some of these operations can be performed continuously (e.g., over a period of time) to meet the target sound level as output device 6 moves away.

[0086] The controller determines whether the output device 6 has moved beyond a threshold distance (at decision box 78). In one aspect, this determination may be based on sensor data. For example, the controller may determine whether the output device 6 is outside the acoustically audible range (e.g., based on whether the microphone signal has a sound level below the sound level threshold). If so, this may mean that the user of the output device 6 cannot hear any sound being produced by the playback device 5. Therefore, the output device 6 can play back audio content at a target sound level (at box 79). Specifically, the output sound level of the output device 6 may be equal to the target sound level determined for the playback device 5. In one aspect, the output device 6 may maintain this sound level when it is outside the threshold distance (e.g., outside the acoustically audible range).

[0087] Figure 5This is a flowchart of one aspect of a process 80 in which the output device 6 bridges with the playback device 5 for audio playback as the output device 6 moves toward the playback device 5. In one aspect, at least some of the operations described in this process may be... Figure 4 The operation is performed after (or before) one or more operations described in process 70. For example, an operation in this process may be performed some time after process 70 has been performed. For example, process 70 may be performed by output device 6 when user 10 of the device moves away from playback device 5, as per [reference to...]. Figure 1 As shown and described. This process can be performed while the output device 6 is playing back audio content and a user 10 wearing or holding the output device 6 is moving (back) toward the playback device 5 (e.g., in room 7).

[0088] Process 80 begins with the controller 24 determining that the output device 6 is moving toward the playback device 5 (at block 81). Specifically, the controller may perform operations similar to those described herein to determine that the device is moving toward the playback device 5. In one aspect, the controller may perform operations similar to those described in block 75 of process 70. For example, the controller may receive location data (e.g., from the playback device 5 and / or from a remote server communicatively coupled to the device) and compare the location data with the location data of the output device 6. The controller determines playback characteristics associated with the audio content being played back by the playback device 5 (at block 82). In particular, the controller may perform operations similar to those described in block 76 of process 70 to determine the sound level and / or time alignment of the audio content being generated by the playback device 5 based on playback data from the playback device 5.

[0089] The controller plays back audio content (at box 83) at a level that meets the target sound level (e.g., reduced) based on playback characteristics such as the determined sound level and time alignment. Specifically, the controller determines that the sound level at the microphone has increased, and therefore the combination of the sound level produced by the playback device 5 and the sound output level of the speaker 8 may exceed the target sound level. Therefore, the controller can adjust the sound output level of the output device 6 to compensate for the overall increase in sound level. In particular, the controller can reduce the sound level of the speaker 8 based on the increase in the microphone sound level (e.g., based on a comparison of a previously determined sound level with the current sound level). On the other hand, the controller can reduce the sound level based on the difference between the combination of the sound level at the microphone and the output sound level of the speaker. Therefore, in response to determining that the output device 6 is moving toward the playback device 5, the sound output level of the speaker 8 is reduced.

[0090] In one aspect, the audio renderer 35 may perform one or more audio signal processing operations to reduce the output sound level of the speaker. For example, to reduce the sound output level, the audio renderer 35 may attenuate the signal level of the audio signal of the audio content based on the change in the sound level at the microphone of the output device 6 caused by the sound generated by the playback device 5 (e.g., by applying a scalar gain based on the sound level at the microphone to the audio signal of the audio content). In another aspect, the output device 6 may perform these operations while the device is moving toward the playback device 5. Thus, as the output device 6 moves closer to the playback device 5, the device may continue to attenuate the audio signal (e.g., proportionally). Therefore, the controller processes the audio signal by gradually reducing (or partially reducing) the sound generated by the speaker 8.

[0091] Controller 24 determines whether output device 6 is within the threshold distance of playback device 5 (at decision box 84). Specifically, the controller determines whether output device 6 is close enough to playback device 5 that the sound produced by playback device 5 meets the target sound level, and therefore output device 6 is no longer needed to produce the sound of the audio content. In one aspect, the controller may make this determination based on the sound level at the microphone. Specifically, the controller may determine whether the sound level of the sound played back by playback device 5 is equal to or exceeds the target sound level. If so, the controller may determine that output device 6 is within the threshold distance. In another aspect, the determination may be based on other data, as described herein. If output device 6 is within the threshold distance, the controller may stop playback of audio content through speaker 8 by stopping the use of audio signals from content extractor 34 to drive speaker 8 (at box 85).

[0092] Figure 6This is a flowchart of one aspect of a process 90 for bridging audio playback between output device 6 and playback device 5. Process 90 begins by the controller receiving a representation of audio content via a computer network (e.g., network 23) (at box 91). For example, the representation may include playback data received by one or more playback devices that are playing back the audio content. While a second electronic device (e.g., playback device 5) is playing back the audio content through a second speaker (e.g., speaker 21), the controller determines that a first electronic device (e.g., output device 6) is moving away from the second electronic device (at box 92). In response to determining that the first electronic device is moving away from the second electronic device, the controller uses the representation of the audio content to play back the audio content through the first speaker (at box 93). For example, the controller may use the playback data to synchronize the playback of the audio content performed by the first electronic device with the playback status of the audio content at the second electronic device. Specifically, the controller may play back the audio content according to a timestamp of the playback status (e.g., current playback) such that the sound produced by the second speaker and the sound produced by the first speaker are synchronized when perceived by the user 10 of output device 6. As described herein, the controller can play back audio content based on timestamps while taking into account acoustic Time-of-Flight (ToF). Therefore, the two devices can play back audio content asynchronously (e.g., output device 6 plays back audio content after playback device 5), and the sounds produced by the two devices reach the user (e.g., the user's ears) at (approximately) the same time, thus giving the user the perception that the sound is synchronized. In one aspect, the sound output by output device 6 can provide the user with a perception of the location of the combined sound source from the playback device. For example, the controller can spatially render the audio signal at a virtual sound source located (approximately) at the location of the playback device (e.g., using one or more HRTFs).

[0093] Some aspects are feasible Figures 4-6Variations of processes 70, 80, and / or 90 described herein. For example, at least some specific operations in these processes may not be performed in the exact order shown and described. The specific operation may not be performed in a consecutive series of operations, and different specific operations may be performed in different aspects. For example, as described so far, the controller may play back audio content at a level that meets the target level based on the determined sound level and in time alignment. In one aspect, the controller may perform these operations without user intervention (e.g., automatically). In another aspect, the controller may request user authorization (approval) before applying audio signal processing operations to meet the target level. For example, the controller may output a notification indicating that the target sound level is not met (e.g., an audible sound notification via speaker 8 and / or a visual (e.g., pop-up) notification via display screen 27). Specifically, the controller may indicate that the sound output level of speaker 8 is insufficient to compensate for the detected change in sound level at the microphone. Upon receiving user input (e.g., user selection of a user interface (UI) item on the display screen, voice command, etc.), the controller may continue playing back the audio content as described herein.

[0094] As described above, output device 6 is configured to bridge audio playback with one or more playback devices 5. Specifically, output device 6 can perform at least some of the operations described herein to compensate for audio playback performed by playback device 5. In another aspect, output device 6 can be configured to bridge audio playback that has already begun at output device 6 with a playback device. For example, output device 6 may be playing back audio content (e.g., based on user input from user 10). In this case, the user may be perceiving the audio content at a specific sound level. In one aspect, output device 6 can bridge playback with nearby playback devices (e.g., within acoustic audible range). For example, output device 6 can communicate with a remote electronic server (e.g., via network 23) to identify whether one or more playback devices are within acoustic audible range. If so, output device 6 can transmit playback data to playback device 5 and instruct playback device 5 to play back audio content. In one aspect, output device 6 can transmit instructions to playback device 5 to play back audio content at a specific sound level (e.g., target level). Therefore, output device 6 can perform one or more of the operations described herein in order to satisfy the sound level of playback device 5.

[0095] As described above, the controller 24 of the output device 6 can perform one or more operations to meet the target sound level of the playback device 5. On the other hand, the output device 6 can transmit playback data to the playback device 5, including one or more instructions for the playback device 5 to perform one or more of the operations described herein. For example, as the output device 6 moves closer to the playback device 5, the output device 6 can determine a volume adjustment for the playback device 5 based on determined playback characteristics and can transmit the volume adjustment to the playback device 5. Furthermore, the playback device 5 can adjust its sound output according to the volume adjustment. For example, as the output device 6 moves away from the playback device 5, the output device 6 can instruct the playback device 5 to increase the volume to compensate for the increased distance between the devices.

[0096] In some respects, because the output device can be stationary relative to the user, the output device 6 can also perform one or more audio signal processing operations. For example, when the output device 6 moves away from the playback device 5, the output device 6 can also apply volume adjustment to increase the overall volume of both devices.

[0097] As described in this article, output device 6 is configured to bridge audio playback with playback device 5, such as Figure 1 As shown. On the other hand, output device 6 can be configured to bridge audio playback with two or more playback devices, thereby enabling output device 6 to adjust the sound output based on the audio playback of the playback devices in order to provide a consistent listening experience to the user. Figure 7 An example of this is shown.

[0098] Figure 7 Three stages 50-52 according to one aspect are illustrated, wherein the output device 6 maintains the sound level heard by the user 10 as the user moves between two separate playback devices (first playback device 55 and second playback device 56) playing audio content. Specifically, each stage illustrates the first playback device 55 and the second playback device 56, both playing the same audio content (e.g., a musical piece), and the user 10 wearing the output device 6. Furthermore, each stage illustrates the sound level 11 of the first playback device 55, the sound level 57 of the second playback device 56, and the sound level 12 of the output device 6, wherein each level is heard by the user 10 (e.g., the microphone 9 of the output device 6 worn by the user). In one aspect, each sound level can be a loudness level (e.g., in dB SPL) at or near the user's ear (or ear canal).

[0099] In the first stage 50, the user 10 wearing the output device 6 is positioned close to the first playback device 55. In this situation, the user hears most (if not all) of the audio content from the first playback device 55, and hears little (or very little) of the content from the output device 6 and the second playback device 56. This is indicated by sound levels 12 and 57 being approximately zero (or below the threshold). In one aspect, sound level 11 at this stage can be defined (e.g., by system 4) as a target sound level. For example, in this stage, the output device 6 can perform the following (e.g., in...) Figure 4 At least some of the operations described in process 70 are used to determine a target sound level. In one aspect, output device 6 may perform these operations based on user input (e.g., user activation of output device 6, user selection of UI items in a graphical user interface (GUI) displayed on display screen 27, etc.). For example, when activated, output device 6 may determine that the sound level of microphone 9 (at this location) is the target sound level at which user 10 wants to hear the audio content. In one aspect, since the sound level 11 at this stage is the target sound level, output device 6 may not play back the audio content because the sound level at the microphone is equal to (or greater than) the target level 11. In another aspect, at this stage 50, a threshold distance (e.g., the distance between user 10 and the first playback device 55) may be defined for output device 6 to stop playing back the audio content.

[0100] Phase 51 shows that user 10 has moved away from the first playback device 55 and towards the second playback device 56. Specifically, as the user moves away from the first playback device 55, the sound level 11 perceived by user 10 decreases, while the sound level 57 of the second playback device 56 increases. For example, this could be because user 10 is moving in a room where the two playback devices are located on opposite sides of the room. As the user moves away from the first playback device 55, output device 6 begins to generate sound to meet the target sound level, as shown in phase 50. Furthermore, output device 6 can be configured to take sound level 57 into account. Specifically, upon detecting the sound from both devices, output device 6 can determine their respective sound levels and then adjust the sound output level of speaker 8 (e.g., applying a scalar gain to the audio signal used to drive the audio content of the speaker) to maintain the target sound level perceived by the user. Thus, as shown, the combination of sound levels 11, 12, and 57 is equal to (or approximately) sound level 11 in phase 50.

[0101] In one aspect, output device 6 may synchronize playback based on playback data received from and / or transmitted to either or both of playback devices 55 and 56. For example, output device 6 may receive playback data from both devices and determine one or more time alignments to be applied so that the sound generated by output device 6 is perceived by the user as synchronized with the sound of either or both of the playback devices. In one aspect, output device 6 may apply different time alignments to one or more audio signals of audio content. In another aspect, output device 6 may transmit playback data to synchronize playback. For example, output device 6 may transmit playback data to a second playback device 56 to apply one or more time alignments to delay playback so that the sound generated by the second playback device arrives at (approximately) the same time as the sound from the first playback device 55 (at microphone 9). Along with (or instead of) instructing the sound playback device 56 to delay playback, output device 6 may instruct the second playback device 56 to adjust its sound output to ensure that the target sound level is maintained as the user moves closer to the second playback device 56.

[0102] The third stage 52 shows that the user 10 has moved closer to the second playback device 56, so that the user can no longer hear the sound from the first playback device (and / or the sound level produced by the first playback device is below the threshold level at the user's location), as shown by the low sound level 11. In the case of the sound level 57 increasing due to the user moving closer to the second playback device 56, the output device 6 reduces the sound output level of the speaker 8. Specifically, as the user moves toward the second playback device 56, the output device 6 attenuates the speaker sound output and reduces the sound level 12 of the output device 6. As shown, in effect, the output device 6 has stopped playing back the audio content (as indicated by sound level 12). In one aspect, the output device 6 may have stopped playback based on the output device being within the threshold distance of the second playback device 56. In another aspect, the output device 6 may have stopped sound output based on the sound level at the microphone being at least (or having reached) the target sound level.

[0103] In one aspect, an electronic device (e.g., output device 6, which may be a headset and / or a wearable device such as a pair of smart glasses with an over-ear speaker) plays back audio content after a second electronic device (e.g., playback device 5, such as a smart speaker or a television) has played back the audio content (e.g., to synchronize the sound of the audio content as perceived by the user 10). In another aspect, the playback performed by both the first and second electronic devices is perceived as synchronized by the user holding or wearing the first electronic device, while the first and second electronic devices play back audio content asynchronously (e.g., output device 6 plays back the same audio content as playback device 5, but at a later time).

[0104] In some aspects, playing back audio content through a first speaker (e.g., speaker 8) at a target sound level includes: when the first electronic device is moving away from the second electronic device, adjusting the target sound level to compensate for the change in sound level based on the determination that the sound level of the audio content at the microphone has changed. In some aspects, adjusting the target sound level includes: applying volume adjustment to the first electronic device based on the difference between the sound level and the change in sound level. In another aspect, increasing the target sound level as the first electronic device moves away from the second electronic device.

[0105] In one aspect, based on the determination that the first electronic device has moved within a threshold distance of the second electronic device, the first electronic device stops playing back audio content (e.g., by stopping the use of audio signals to drive the first speaker).

[0106] In one aspect, a first electronic device is communicatively coupled to a second electronic device via a wireless connection, and determining that the first electronic device is moving away from the second electronic device includes: identifying the position of the first electronic device relative to the second electronic device (e.g., based on the RSSI of the wireless connection), and determining that the first electronic device is moving away from that position based on a change in the RSSI. In another aspect, playing back audio content using a representation of audio content includes: retrieving an audio signal from a remote electronic server or the local memory of the first electronic device using an identifier of the audio content, wherein the audio signal includes the audio content; and using the audio signal to drive a first speaker to produce sound from the audio content.

[0107] On the other hand, the playback state includes a timestamp of the portion of the audio content to be played back by the second electronic device. Using the playback data to synchronize the playback includes: playing back the portion of the audio content according to the timestamp, such that the sound produced by the second speaker of the second electronic device when playing back the portion of the audio content is synchronized with the sound produced by the first speaker of the first electronic device when playing back the portion of the audio content, as perceived by the user of the first electronic device.

[0108] As is widely recognized, the use of personally identifiable information should comply with privacy policies and practices that are generally accepted to meet or exceed industry or governmental requirements for protecting user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use, and the nature of authorized use should be clearly explained to users.

[0109] As previously described, one aspect of this disclosure may be a non-transitory machine-readable medium (such as microelectronic memory) on which instructions are stored, programming one or more data processing units (generally referred to herein as a "processor") to perform network operations and audio signal processing operations, as described herein. In other aspects, some of these operations may be performed by specific hardware components containing hard-wired logic. Alternatively, those operations may be performed by any combination of programmed data processing units and fixed hard-wired circuit components.

[0110] While certain aspects have been described and illustrated in the accompanying drawings, it should be understood that such aspects are merely illustrative and not limiting of the general disclosure, and that this disclosure is not limited to the specific structures and arrangements shown and described, as various other modifications will be apparent to those skilled in the art. Therefore, the description is to be regarded as exemplary and not restrictive.

[0111] In some aspects, this disclosure may include the language "[element A] and [element B] at least one". This language may refer to one or more of these elements. For example, "at least one of A and B" may refer to "A", "B", or "A and B". Specifically, "at least one of A and B" may refer to "at least one of A and at least one of B" or "at least either A or B". In some aspects, this disclosure may include the language "[element A], [element B], and / or [element C]". This language may refer to any of these elements or any combination thereof. For example, "A, B, and / or C" may refer to "A", "B", "C", "A and B", "A and C", "B and C", or "A, B, and C".

Claims

1. A method, the method comprising: While an electronic device including a first speaker is playing audio content and the headphones are being worn by a user, it is determined that the headphones are moving away from the electronic device. The audio content representation is received via a network at the headphones including a second speaker, wherein the audio content representation includes playback data indicating the playback status of the audio content at the electronic device; as well as In response to determining that the headphones are moving away from the electronic device, the audio content is played back by both the electronic device and the headphones, and the playback data is used to synchronize the playback of the audio content through the second speaker of the headphones with the playback status.

2. The method according to claim 1, further comprising: Determine the acoustic time of flight (ToF) of the sound produced by the first speaker. The playback status includes timestamps of the portion of the audio content that will be played back by the electronic device. The use of the playback data to synchronize the playback of the audio content includes: playing back the portion of the audio content through the second speaker according to the timestamp while taking into account the acoustic time of flight (ToF), such that the sound of the portion of the audio content produced by the first speaker of the electronic device and the sound of the portion of the audio content produced by the second speaker of the headphones are synchronized when perceived by the user.

3. The method according to claim 1, further comprising: The target sound level of the audio content is determined based on the representation of the audio content; as well as Determine the sound level of the audio content played back by the electronic device at the microphone of the headphones. The use of the playback data for synchronous playback includes: playing back the audio content through the second speaker based on the sound level at a level that meets the target sound level.

4. The method according to claim 1, The use of the playback data for synchronized playback includes: The second speaker is driven using an audio signal containing the aforementioned audio content. The method further includes attenuating the signal level of the audio signal at the headphones based on changes in the sound level of the audio content played back by the electronic device at the microphone of the headphones as the headphones move toward the electronic device.

5. The method according to claim 1, wherein the electronic device is a first electronic device, and the method further comprises: Based on the determination that the headphones are moving toward a second electronic device that is playing back the audio content through a third speaker, the sound output level of the second speaker is reduced.

6. The method according to claim 1, further comprising: Determine the position of the electronic device relative to the headset; as well as The audio content is rendered in space to generate a virtual sound source including the audio content at the location via the second speaker.

7. The method according to claim 1, further comprising: Determine the sound level of the audio content played back by the electronic device at the microphone of the headphones. Determining that the headphones are moving away from the electronic device includes detecting that the sound level is decreasing at a specific rate.

8. A headset, the headset comprising: First loudspeaker; One or more processors; as well as A memory containing instructions that, when executed by the one or more processors, cause the headset to: When an electronic device including a second speaker is playing audio content and the headphones are being worn by a user, it is determined that the headphones are moving away from the electronic device. The representation of the audio content received via a network includes playback data indicating the playback status of the audio content at the electronic device; as well as In response to determining that the headphones are moving away from the electronic device, both the electronic device and the headphones play back the audio content, and the playback data is used to synchronize the playback of the audio content through the first speaker with the playback state.

9. The headset of claim 8, wherein the memory has additional instructions for determining the acoustic time-of-flight (ToF) of the sound produced by the second speaker. The playback status includes timestamps of the portion of the audio content that will be played back by the electronic device. The instructions for using the playback data to synchronize the playback of the audio content include instructions for: playing back the portion of the audio content through the first speaker while taking into account the acoustic time of flight (ToF) and according to the timestamp, such that the sound of the portion of the audio content produced by the second speaker of the electronic device and the sound of the portion of the audio content produced by the first speaker of the headphones are synchronized when perceived by the user.

10. The headset of claim 8, wherein the memory has additional instructions for: The target sound level of the audio content is determined based on the representation of the audio content; and Determine the sound level of the audio content played back by the electronic device at the microphone of the headphones. The instructions for using the playback data to synchronize the playback of the audio content through the first speaker include instructions for: playing back the audio content through the first speaker based on the sound level at a level that satisfies the target sound level.

11. The headphones according to claim 8, The instructions for using the playback data to synchronize the playback of the audio content include instructions for driving the first speaker using an audio signal having the audio content. The memory wherein the memory has additional instructions for attenuating the signal level of the audio signal at the headphones based on changes in the sound level of the audio content played back by the electronic device at the microphone of the headphones as the headphones move toward the electronic device.

12. The headphones of claim 8, wherein the electronic device is a first electronic device, and wherein the memory has additional instructions for: reducing the sound output level of the first speaker based on determining that the headphones are moving toward a second electronic device that is playing back the audio content through a third speaker.

13. The headset of claim 8, wherein the memory has additional instructions for: Determine the position of the electronic device relative to the headset; and The audio content is rendered in space to generate a virtual sound source including the audio content at the location via the first speaker.

14. The headphones of claim 8, wherein the memory has additional instructions for: determining the sound level of the audio content played back by the electronic device at the microphone of the headphones. The instructions for determining that the headphones are moving away from the electronic device include instructions for detecting that the sound level of the sound is decreasing at a specific rate.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a headset, cause the headset to: While an electronic device including a first speaker is playing audio content and the headphones are being worn by a user, it is determined that the headphones are moving away from the electronic device. Receiving a representation of the audio content via a network at the headphones including a second speaker, wherein the representation of the audio content includes playback data indicating the playback status of the audio content at the electronic device; and In response to determining that the headphones are moving away from the electronic device, the audio content is played back by both the electronic device and the headphones, and the playback data is used to synchronize the playback of the audio content through the second speaker of the headphones with the playback status.

16. The non-transitory computer-readable storage device of claim 15, further comprising instructions for: determining the acoustic time-of-flight (ToF) of the sound generated by the first speaker. The playback status includes timestamps of the portion of the audio content that will be played back by the electronic device. The instructions for using the playback data to synchronize the playback of the audio content include instructions for: playing back the portion of the audio content through the second speaker according to the timestamp while taking into account the acoustic time of flight (ToF), such that the sound of the portion of the audio content produced by the first speaker of the electronic device and the sound of the portion of the audio content produced by the second speaker of the headphones are synchronized when perceived by the user.

17. The non-transitory computer-readable storage medium of claim 15, wherein the storage medium has additional instructions for: The target sound level of the audio content is determined based on the representation of the audio content; and Determine the sound level of the audio content played back by the electronic device at the microphone of the headphones. The instructions for using the playback data to synchronize playback include instructions for: playing back the audio content through the second speaker based on the sound level at a level that meets the target sound level.

18. The non-transitory computer-readable storage device according to claim 15, The instructions for using the playback data to synchronize the playback of the audio content include instructions for driving the second speaker using an audio signal containing the audio content. The memory wherein the memory has additional instructions for attenuating the signal level of the audio signal at the headphones based on changes in the sound level of the audio content played back by the electronic device at the microphone of the headphones as the headphones move toward the electronic device.

19. The non-transitory computer-readable storage of claim 15, wherein the electronic device is a first electronic device, wherein the storage has additional instructions for: reducing the sound output level of the second speaker based on determining that the headphones are moving toward a second electronic device that is playing back the audio content through a third speaker.

20. The non-transitory computer-readable storage device of claim 15, further comprising instructions for: Determine the position of the electronic device relative to the headset; and The audio content is rendered in space to generate a virtual sound source including the audio content at the location via the second speaker.

21. The non-transitory computer-readable storage memory of claim 15, wherein the storage memory has additional instructions for: determining the sound level of the audio content played back by the electronic device at the microphone of the headphones. The instructions for determining that the headphones are moving away from the electronic device include instructions for detecting that the sound level of the sound is decreasing at a specific rate.

Images