Orchestration of Acoustic Direct Sequence Spread Spectrum Signals for Estimation of Acoustic Scene Metrics

JP2023552196A5Active Publication Date: 2024-12-09DOLBY LABORATORIES LICENSING CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023533890
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-09-07
Filing Date
2021-12-02
Publication Date
2024-12-09
Estimated Expiration
2041-12-02

AI Technical Summary

Technical Problem

Existing audio systems struggle with efficient estimation of acoustic scene metrics, such as audibility and positioning of audio devices, requiring manual calibration and failing to adapt to changes in the environment, which hinders user adoption and system responsiveness.

Method used

The use of direct sequence spread spectrum (DSSS) signals injected into audio playback signals allows audio devices to generate and detect these signals, enabling estimation of acoustic scene metrics like audibility, angle, and noise levels, and adapt to environmental changes without manual calibration.

Benefits of technology

This method enables dynamic and efficient estimation of acoustic scene metrics, improving audio device coordination and user experience by enhancing spatial audio reproduction and voice interaction, while reducing the need for manual calibration and adapting to environmental changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Some methods may include receiving a first content stream including a first group of audio signals, rendering the first group of audio signals to generate a first group of audio playback signals, generating a first group of direct sequence spread spectrum (DSSS) signals, generating a first group of modified audio playback signals by inserting the first group of DSSS signals into the first group of audio playback signals, and generating a first audio device playback sound by playing the first modified audio playback signal through a loudspeaker system. The method(s) may include receiving a group of microphone signals corresponding to at least the first audio device playback sound and second through Nth audio device playback sounds corresponding to second through Nth modified audio playback signals (including the second through Nth DSSS signals) played by second through Nth audio devices, extracting the second through Nth DSSS signals from the microphone signals, and estimating at least one acoustic scene metric based at least in part on the second through Nth DSSS signals.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 121,085, filed December 3, 2020, U.S. Provisional Patent Application No. 63 / 260,953, filed September 7, 2021, U.S. Provisional Patent Application No. 63 / 120,887, filed December 3, 2020, and U.S. Provisional Patent Application No. 63 / 201,561, filed May 4, 2021, the contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD The present disclosure relates to audio processing systems and methods. [Background technology]

[0003] Audio devices and systems are widely used, and while existing systems and methods for estimating acoustic scene metrics (e.g., the audibility of audio devices) are known, improved systems and methods are desired.

[0004] Notation and Nomenclature Throughout this disclosure, including the claims, "speaker" and "loudspeaker" are used interchangeably to refer to any acoustically emitting transducer (or set of transducers) driven by a single speaker feed. A typical headphone set includes two speakers. A speaker may be implemented to include multiple transducers (e.g., a woofer and a tweeter) driven by a single common speaker feed or multiple speaker feeds. In some examples, the speaker feed(s) may receive different processing, possibly in different circuit branches connected to different transducers.

[0005] Throughout this disclosure, including the claims, the expression "performing" an operation on a signal or data (e.g., filtering, scaling, transforming, or applying a gain to a signal or data) is used broadly to mean performing the operation directly on the signal or data, or performing the operation on a processed version of the signal or data (e.g., a pre-filtered or pre-processed version of the signal before undergoing the operation).

[0006] Throughout this disclosure, including the claims, the term "system" is used broadly to mean a device, system, or subsystem. For example, a subsystem that implements a decoder may be referred to as a decoder system, and a system that includes such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, where M of the inputs are generated by the subsystem and the other XM inputs are received from external sources) may also be referred to as a decoder system.

[0007] Throughout this disclosure, including the claims, the term "processor" is used broadly to mean a system or device that is programmable or otherwise configurable (e.g., by software or firmware) to perform operations on data (e.g., audio, or video or other image data). Examples of processors include field programmable gate arrays (or other configurable integrated circuits or chipsets), digital signal processors programmed and / or otherwise configured to perform pipelined processing on audio or other sound data, programmable general-purpose processors or computers, and programmable microprocessor chips or chipsets.

[0008] Throughout this disclosure, including the claims, the terms "couple" or "coupled" are used to mean either a direct connection or an indirect connection. Thus, when a first device is connected to a second device, the connection may be by way of a direct connection or by way of an indirect connection through other devices and connections.

[0009] As used herein, a "smart device" is an electronic device configured to communicate with one or more other devices (or networks), generally via various wireless protocols such as Bluetooth, Zigbee, near-field communications, WiFi, Light Fidelity (LiFi), 3G, 4G, 5G, etc., and capable of operating interactively and / or autonomously to some degree. Some representative types of smart devices are smartphones, smart cars, smart thermostats, smart doorbells, smart locks, smart refrigerators, phablets and tablets, smart watches, smart bands, smart keychains, and smart audio devices. The term "smart device" may also refer to devices that exhibit some attributes of ubiquitous computing, such as artificial intelligence.

[0010] The term "smart audio device" is used herein to refer to a smart device that is a single-purpose audio device or a multi-purpose audio device (e.g., an audio device that implements at least some aspects of virtual assistant functionality). A single-purpose audio device is a device (e.g., a television (TV)) that includes or is connected to at least one microphone (and optionally also includes or is connected to at least one speaker and / or at least one camera) and is generally or primarily designed to achieve a single purpose. For example, while a TV can typically play (or is considered to be able to play) audio from program material, modern TVs in most cases run some kind of operating system on which multiple applications, including applications for watching television, run locally. In this sense, single-purpose audio devices having speaker(s) and microphone(s) are often configured to run local applications and / or services for direct use of the speaker(s) and microphone(s). Some single-purpose audio devices are configured to be grouped to achieve audio playback over a zone, i.e., a user-defined area.

[0011] One common type of multipurpose audio device is an audio device that implements at least some aspects of virtual assistant functionality, but other aspects of the virtual assistant functionality may be implemented by one or more other devices, such as one or more servers with which the multipurpose audio device is configured to communicate. Such multipurpose audio devices may be referred to herein as "virtual assistants." A virtual assistant is a device (e.g., a smart speaker or voice assistant integrated device) that includes or is connected to at least one microphone (and, optionally, also includes or is connected to at least one speaker and / or at least one camera). In some examples, a virtual assistant may be cloud-enabled in some sense or otherwise provide the ability to utilize multiple devices (different from the virtual assistant) for applications not fully implemented within or on the virtual assistant itself. In other words, at least some aspects of the virtual assistant functionality, e.g., voice recognition functionality, may be implemented (at least in part) by one or more servers or other devices with which the virtual assistant may communicate via a network such as the Internet. Multiple virtual assistants may cooperate, for example, in very discrete and conditionally defined ways. For example, two or more virtual assistants may collaborate in the sense that one of them (e.g., the one that is most certain it heard the wake word) will respond to the wake word. In some aspects, multiple connected virtual assistants may form a kind of flock managed by a single main application, which may be (or may implement) a virtual assistant.

[0012] As used herein, the term "wake word" is used broadly to mean any sound (e.g., a word uttered by a human being, or any other sound). The smart audio device is configured to wake up in response to detecting ("hearing") a sound (using at least one microphone included in or connected to the smart audio device, or at least one other microphone). "Awake" in this context refers to the device entering a state where it waits for a sound command (i.e., listens). In some examples, what may be referred to herein as a "wake word" may include multiple words, e.g., a phrase.

[0013] As used herein, the term "wake word detector" refers to a device (or software including instructions for configuring a device) configured to continuously search for a match between real-time sound (e.g., speech) features and a trained model. Typically, a wake word event is triggered whenever the wake word detector determines that the probability of detecting the wake word exceeds a predefined threshold. For example, the threshold may be a predetermined threshold tuned to provide a good compromise between false acceptance and false rejection rates. After a wake word event, the device enters a state where it listens for commands (sometimes referred to as an "awakened" or "attentiveness" state), in which it may pass received commands to a larger, more computationally intensive recognizer.

[0014] As used herein, the terms "program stream" and "content stream" refer to a collection of one or more audio signals, and possibly video signals, at least some of which are intended to be listened to together. Examples include music selections, movie soundtracks, movies, television programs, audio portions of television programs, podcasts, live voice calls, synthesized voice responses from smart assistants, etc. In some instances, a content stream may contain multiple versions of at least some of the audio signals, e.g., the same dialogue in multiple languages. In such cases, only one version or portion of the audio data (e.g., a version corresponding to one language) is intended to be played at a time. Summary of the Invention [Means for solving the problem]

[0015] At least some aspects of the present disclosure may be implemented via one or more audio processing methods. In some examples, the method(s) may be implemented at least in part by a control system and / or instructions (e.g., software) stored on one or more non-transitory media. Some methods include causing a first audio device in an audio environment to generate, by a control system, first direct sequence spread spectrum (DSSS) signals. According to some aspects, the control system may be or include an orchestration device control system. Some such methods include causing the control system to insert the first DSSS signals into first audio playback signals corresponding to a first content stream to generate first modified audio playback signals for the first audio device. Some such methods include causing the control system to cause the first audio device to play the first modified audio playback signals to generate first audio device playback sound.

[0016] Some such methods include causing a second audio device in the audio environment to generate a second set of DSSS signals by the control system. Some such methods include causing the control system to insert the second set of DSSS signals into a second content stream to generate a second set of modified audio playback signals for the second audio device. Some such methods include causing the control system to cause the second audio device to play the second set of modified audio playback signals to generate second audio device playback. Some methods may include causing each of multiple audio devices in the audio environment to simultaneously play the modified audio playback signals.

[0017] Some such methods include causing at least one microphone in the audio environment to detect at least the first audio device and the second audio device and generate microphone signals corresponding to at least the first audio device and the second audio device. Some such methods include causing the control system to extract the first DSSS signals and the second DSSS signals from the microphone signals. Some such methods include causing the control system to estimate at least one acoustic scene metric based at least in part on the first DSSS signals and the second DSSS signals. Some methods may include controlling one or more aspects of audio device playback based at least in part on the at least one acoustic scene metric.

[0018] In some examples, the at least one acoustic scene metric may include one or more of time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, position of audio devices, noise of an audio environment, or signal-to-noise ratio. According to some examples, having the at least one acoustic scene metric estimated may include estimating the at least one acoustic scene metric. Alternatively, or additionally, having the at least one acoustic scene metric estimated may include having another device estimate the at least one acoustic scene metric.

[0019] In some examples, a first content stream component of sound reproduced by the first audio device may cause perceptual masking of a first DSSS signal component of sound reproduced by the first audio device. In some examples, a second content stream component of sound reproduced by the second audio device may cause perceptual masking of a second DSSS signal component of sound reproduced by the second audio device.

[0020] Some methods may include causing three or more audio devices in the audio environment to generate three or more direct sequence spread spectrum (DSSS) signals, by a control system. Some such methods may include causing the control system to insert the three or more DSSS signals into three or more content streams to generate three or more sets of modified audio playback signals for the three or more audio devices. Some such methods may include causing the control system to cause the three or more audio devices to play corresponding instances of the three or more sets of modified audio playback signals to generate three or more instances of audio device playback sound.

[0021] Some such methods may include, by a control system, causing third through Nth audio devices in the audio environment to generate third through Nth direct sequence spread spectrum (DSSS) signals. Some such methods may include, by the control system, causing the third through Nth DSSS signals to be inserted into third through Nth content streams to generate third through Nth modified audio playback signals for the third through Nth audio devices. Some such methods may include, by the control system, causing the third through Nth audio devices to play corresponding instances of the third through Nth modified audio playback signals to generate third through Nth instances of audio device playback sounds.

[0022] Some methods may include causing, by the control system, at least one microphone of each of the first through Nth audio devices to detect first through Nth instances of audio device reproduced sound and generating microphone signals corresponding to the first through Nth instances of audio device reproduced sound. In some examples, the first through Nth instances of audio device reproduced sound may include the first audio device reproduced sound, the second audio device reproduced sound, and at least a third instance of audio device reproduced sound (in some examples, the third through Nth instances).

[0023] Some such methods may include causing the control system to extract the first through Nth DSSS signals from the set of microphone signals, and in some examples, the at least one acoustic scene metric may be estimated based at least in part on the first through Nth DSSS signals.

[0024] Some methods may include determining one or more DSSS parameters for a plurality of audio devices in the audio environment. In some examples, the one or more DSSS parameters may be usable to generate a set of DSSS signals. Some such methods may include providing the one or more DSSS parameters to each audio device of the plurality of audio devices.

[0025] In some examples, determining the one or more DSSS parameters may include scheduling, for each audio device of the plurality of audio devices, a time slot for playing back a set of modified audio playback signals, In some such examples, a first time slot for a first audio device may be different from a second time slot for a second audio device.

[0026] According to some examples, determining the one or more DSSS parameters may include determining, for each audio device of the plurality of audio devices, a frequency band for reproducing the modified audio playback signals, In some such examples, a first frequency band for a first audio device may be different from a second frequency band for a second audio device.

[0027] In some examples, determining the one or more DSSS parameters may include determining a spreading code for each audio device of the plurality of audio devices. According to some such examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device.

[0028] Some methods may include determining at least one spreading code length based at least in part on corresponding audio device audibility. In some examples, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of a plurality of audio devices in the audio environment.

[0029] According to some examples, determining the one or more DSSS parameters may include determining a current playback objective. Some such methods may include applying an acoustic model based at least in part on the interaudibility of each of multiple audio devices in the audio environment to determine estimated performance of DSSS signals in the audio environment. Some such methods may include determining a perceptual impact of DSSS signals in the audio environment by applying a perceptual model based on human sound perception. Some such methods may include determining the one or more DSSS parameters based at least in part on one or more of the current playback objective, the estimated performance, and the perceptual impact.

[0030] In some examples, determining the one or more DSSS parameters may include detecting a DSSS parameter change trigger. Some such methods may include determining one or more new DSSS parameters corresponding to the DSSS parameter change trigger. Some such methods may include providing the one or more new DSSS parameters to one or more audio devices of the audio environment.

[0031] According to some examples, detecting the DSSS parameter change trigger may include detecting one or more of a new audio device in the audio environment, a change in the position of an audio device, a change in the orientation of an audio device, a change in the settings of an audio device, a change in the position of a person in the audio environment, a change in the type of audio content being played in the audio environment, a change in background noise in the audio environment, a change in audio environment configuration including, but not limited to, a change in door or window configuration in the audio environment, a clock skew between two or more audio devices in the audio environment, a clock bias between two or more audio devices in the audio environment, a change in interaudibility between two or more audio devices in the audio environment, or a change in playback intent.

[0032] Some methods may include processing received microphone signals to generate preprocessed microphone signals. In some such examples, DSSS signals may be extracted from the preprocessed microphone signals. Processing the received microphone signals may include, for example, one or more of applying beamforming, bandpass filtering, or echo cancellation.

[0033] According to some examples, extracting at least the first and second DSSS signals from the microphone signals may include applying a matched filter to the microphone signals or pre-processed versions of the microphone signals to generate delayed waveforms. In some examples, the delayed waveforms may include a first delayed waveform based on at least the first DSSS signals and a second delayed waveform based on the second DSSS signals. Some methods may include applying a low-pass filter to the delayed waveforms. According to some examples, applying the matched filter may be part of a demodulation process. In some examples, an output of the demodulation process may be a demodulated coherent baseband signal.

[0034] Some methods may include estimating a bulk delay and providing the bulk delay estimate to the demodulation process. Some methods may include performing baseband processing on the demodulated coherent baseband signal. In some examples, the baseband processing may output at least one estimated acoustic scene metric.

[0035] According to some examples, the baseband processing may include generating a non-coherently integrated delayed waveform based on demodulated coherent baseband signals received during a non-coherent integration period. In some examples, generating the non-coherently integrated delayed waveform may include squaring the demodulated coherent baseband signals received during the non-coherent integration period to generate squared demodulated baseband signals. Some such examples may include integrating the squared demodulated baseband signals. In some examples, the baseband processing may include applying one or more of a leading edge estimation process, a steered response power estimation process, or a signal-to-noise estimation process to the non-coherently integrated delayed waveform.

[0036] Some methods may include estimating bulk delay. Some such examples may include providing bulk delay estimates to baseband processing.

[0037] Some methods may include estimating at least a first noise power level at a first audio device position and a second noise power level at a second audio device position. In some examples, estimating the first noise power level may be based on the first delayed waveform and estimating the second noise power level may be based on the second delayed waveform. Some such examples may include generating a distribution noise estimate for the audio environment based at least in part on the estimated first noise power level and the estimated second noise power level.

[0038] Some methods may include performing an asynchronous two-way ranging process for canceling unknown clock biases between two asynchronous audio devices. In some examples, the asynchronous two-way ranging process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include performing the asynchronous two-way ranging process between each of a plurality of pairs of audio devices in the audio environment.

[0039] Some methods may include performing a clock bias estimation process to determine an estimated clock bias between two asynchronous audio devices. In some examples, the clock bias estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock bias.

[0040] Some methods may include performing the clock bias estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock biases. Some such examples may include compensating each estimated clock bias of the plurality of estimated clock biases.

[0041] Some methods may include performing a clock skew estimation process to determine an estimated clock skew between two asynchronous audio devices. In some examples, the clock skew estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock skew. Some methods may include performing the clock skew estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock skews. Some such examples may include compensating for each estimated clock skew of the plurality of estimated clock skews.

[0042] Some methods may include detecting a DSSS signal transmitted by an audio device. In some examples, the DSSS signal may correspond to a first spreading code. Some such examples may include providing a second spreading code to the audio device. In some examples, the first spreading code may be or include a first pseudo-random number sequence reserved for newly activated audio devices.

[0043] In some examples, at least a portion of the first group of audio playback signals, at least a portion of the second group of audio playback signals, or at least a portion of each of the first group of audio playback signals and the second group of audio playback signals corresponds to silence.

[0044] At least some aspects of the present disclosure may be implemented via an apparatus. For example, one or more devices may be capable of, at least in part, performing the methods disclosed herein. In some aspects, the apparatus may be or include an audio processing system having an interface system and a control system. The control system may include one or more general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof.

[0045] According to some aspects, the device may also include a loudspeaker system including at least one loudspeaker. In some aspects, the device may also include a microphone system including at least one microphone.

[0046] In some aspects, the control system may be configured to receive a first content stream. The first content stream may include a first group of audio signals. In some such examples, the control system may be configured to render the first group of audio signals to generate a first group of audio playback signals. In some such examples, the control system may be configured to generate a first group of direct sequence spread spectrum (DSSS) signals. In some such examples, the control system may be configured to insert the first group of DSSS signals into the first group of audio playback signals to generate a first group of modified audio playback signals. In some examples, inserting the first group of DSSS signals into the first group of audio playback signals may include mixing the first group of DSSS signals with the first group of audio playback signals. In some such examples, the control system may be configured to cause the loudspeaker system to play the first group of modified audio playback signals to generate a first audio device playback sound.

[0047] According to some examples, the control system may include a DSSS signal generator configured to generate a group of DSSS signals. In some examples, the control system may include a DSSS signal modulator configured to modulate the group of DSSS signals generated by the DSSS signal generator to generate a first group of DSSS signals. In some examples, the control system may include a DSSS signal injector configured to insert the first group of DSSS signals into the first group of audio playback signals to generate the first group of modified audio playback signals.

[0048] In some examples, the control system may be configured to receive from the microphone system microphone signals corresponding to at least the first audio device playback sound and the second audio device playback sound. In some examples, the second audio device playback sound may correspond to second modified audio playback signals played by a second audio device. In some examples, the second modified audio playback signals may include second DSSS signals. In some examples, the control system may be configured to extract at least the second DSSS signals from the microphone signals.

[0049] In some aspects, the control system may be configured to receive, from the microphone system, microphone signals corresponding to at least the first audio device playback sound and second through Nth audio device playback sound microphone signals. In some examples, the second through Nth audio device playback sounds may correspond to second through Nth modified audio playback signals played by the second through Nth audio devices. In some examples, the second through Nth modified audio playback signals may include second through Nth DSSS signals. In some aspects, the control system may be configured to extract at least the second through Nth DSSS signals from the microphone signals.

[0050] In some examples, the control system may be configured to estimate at least one acoustic scene metric based at least in part on the second through Nth DSSS signals. In some examples, the at least one acoustic scene metric may include one or more of time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, position of audio devices, noise of an audio environment, or signal-to-noise ratio. In some aspects, the control system may be configured to control one or more aspects of audio device playback based at least in part on the at least one acoustic scene metric and / or at least one audio device characteristic.

[0051] In some examples, the control system may be configured to determine one or more DSSS parameters for each audio device of a plurality of audio devices in the audio environment. In some examples, the one or more DSSS parameters may be usable to generate DSSS signals. In some such aspects, the control system may be configured to provide the one or more DSSS parameters to each audio device of the plurality of audio devices.

[0052] In some examples, determining the one or more DSSS parameters may include scheduling, for each audio device of the plurality of audio devices, a time slot for playing back a set of modified audio playback signals, In some such examples, a first time slot for a first audio device may be different from a second time slot for a second audio device.

[0053] According to some examples, determining the one or more DSSS parameters may include determining, for each audio device of the plurality of audio devices, a frequency band for reproducing the modified audio playback signals, where a first frequency band for a first audio device may be different from a second frequency band for a second audio device.

[0054] In some aspects, determining the one or more DSSS parameters may include determining a spreading code for each audio device of the plurality of audio devices. In some examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device. In some examples, the control system may be configured to determine at least one spreading code length based at least in part on the audibility of the corresponding audio device. According to some aspects, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of the plurality of audio devices in the audio environment.

[0055] In some aspects, determining the one or more DSSS parameters may include determining a current playback objective. In some such examples, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of a plurality of audio devices in the audio environment to determine an estimated performance of DSSS signals in the audio environment. In some such examples, determining the one or more DSSS parameters may include determining a perceptual impact of DSSS signals in the audio environment by applying a perceptual model based on human sound perception. In some such examples, determining the one or more DSSS parameters may be based at least in part on one or more of the current playback objective, the estimated performance, or the perceptual impact. In some examples, determining the one or more DSSS parameters may be based at least in part on the current playback objective, the estimated performance, and the perceptual impact.

[0056] According to some aspects, determining the one or more DSSS parameters may include detecting a DSSS parameter change trigger. In some such aspects, the control system may be configured to determine one or more new DSSS parameters corresponding to the DSSS parameter change trigger. In some such aspects, the control system may be configured to provide the one or more new DSSS parameters to one or more audio devices in the audio environment.

[0057] In some aspects, detecting the DSSS parameter change trigger may include detecting one or more of a new audio device in the audio environment, a change in the position of an audio device, a change in the orientation of an audio device, a change in the settings of an audio device, a change in the position of a person in the audio environment, a change in the type of audio content that may be played in the audio environment, a change in background noise in the audio environment, a change in audio environment configuration including, but not limited to, a change in door or window configuration in the audio environment, a clock skew between two or more audio devices in the audio environment, a clock bias between two or more audio devices in the audio environment, a change in interaudibility between two or more audio devices in the audio environment, or a change in playback intent.

[0058] In some aspects, the control system may be configured to process the received microphone signals to generate preprocessed microphone signals. In some such examples, the control system may be configured to extract DSSS signals from the preprocessed microphone signals. In some aspects, processing the received microphone signals may include one or more of applying beamforming, bandpass filtering, or echo cancellation.

[0059] According to some examples, extracting at least the second through Nth DSSS signals from the group of microphone signals may include applying a matched filter to the group of microphone signals or pre-processed versions of the group of microphone signals to generate second through Nth delayed waveforms. In some such examples, the second through Nth delayed waveforms may correspond to each of the second through Nth DSSS signals. In some examples, the control system may be configured to apply a low pass filter to each of the second through Nth delayed waveforms.

[0060] In some aspects, the control system may be configured to implement a demodulator. In some such aspects, applying the matched filter may be part of a demodulation process performed by the demodulator. In some such examples, the output of the demodulation process may be a demodulated coherent baseband signal.

[0061] In some examples, the control system may be configured to estimate a bulk delay and provide the bulk delay estimate to the demodulator. In some aspects, the control system may be configured to implement a baseband processor configured for baseband processing of the demodulated coherent baseband signal. In some such examples, the baseband processor may be configured to output at least one estimated acoustic scene metric.

[0062] According to some examples, the baseband processing may include generating a non-coherently integrated delayed waveform based on demodulated coherent baseband signals received during a non-coherent integration period. In some examples, generating the non-coherently integrated delayed waveform may include squaring the demodulated coherent baseband signals received during the non-coherent integration period to generate squared demodulated baseband signals and integrating the squared demodulated baseband signals. According to some examples, the baseband processing may include applying one or more of a leading edge estimation process, a steered response power estimation process, or a signal-to-noise estimation process to the non-coherently integrated delayed waveform. In some examples, the control system may be configured to estimate a bulk delay and provide the bulk delay estimates to the baseband processor.

[0063] In some aspects, the control system may be configured to estimate second through Nth noise power levels at the locations of the second through Nth audio devices based on the second through Nth delayed waveforms. In some such examples, the control system may be configured to generate a distributed noise estimate for the audio environment based at least in part on the second through Nth noise power levels.

[0064] In some examples, the control system may be configured to perform an asynchronous two-way ranging process to cancel out unknown clock biases between two asynchronous audio devices. According to some examples, the asynchronous two-way ranging process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. In some examples, the control system may be further configured to perform the asynchronous two-way ranging process between each of a plurality of pairs of audio devices in the audio environment.

[0065] In some aspects, the control system may be configured to perform a clock bias estimation process to determine an estimated clock bias between two asynchronous audio devices. In some examples, the clock bias estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. In some aspects, the control system may be configured to compensate for the estimated clock bias.

[0066] In some examples, the control system may be configured to perform the clock bias estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock biases, and in some aspects, the control system may be configured to compensate for each estimated clock bias of the plurality of estimated clock biases.

[0067] In some aspects, the control system may be configured to perform a clock skew estimation process to determine an estimated clock skew between two asynchronous audio devices. In some aspects, the clock skew estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. In some such examples, the control system may be configured to compensate for the estimated clock skew.

[0068] In some examples, the control system may be configured to perform the clock skew estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock skews, and in some such examples, the control system may be configured to compensate for each estimated clock skew of the plurality of estimated clock skews.

[0069] In some aspects, the control system may be configured to provide a DSSS signal transmitted by an audio device. In some such examples, the DSSS signal may correspond to a first spreading code. In some such examples, the first spreading code may be or include a first pseudo-random number sequence reserved for newly activated audio devices. In some aspects, the control system may be configured to provide a second spreading code to the audio device for subsequent transmission.

[0070] In some examples, the control system may be configured to simultaneously play modified audio playback signals to each of a plurality of audio devices within the audio environment.

[0071] Some additional aspects of the present disclosure may be implemented via one or more methods. In some examples, the method(s) may be implemented at least in part by a control system and / or instructions (e.g., software) stored on one or more non-transitory media. Some methods may include receiving, by a control system, a first content stream. The first content stream may include a first group of audio signals. Some such methods include rendering, by the control system, the first group of audio signals to generate a first group of audio playback signals. Some such methods include generating, by the control system, a first group of direct sequence spread spectrum (DSSS) signals. Some such methods include generating, by the control system, a first group of modified audio playback signals by inserting the first group of DSSS signals into the first group of audio playback signals. Some such methods include generating, by the control system, a first audio device playback sound by causing a loudspeaker system to play the first group of modified audio playback signals.

[0072] Some methods may include receiving, by the control system, from a microphone system, microphone signals corresponding to at least the first audio device playback sound and the second audio device playback sound. In some examples, the second audio device playback sound may correspond to second modified audio playback signals played by a second audio device. In some examples, the second modified audio playback signals may include second DSSS signals. Some methods may include extracting, by the control system, at least the second DSSS signals from the microphone signals.

[0073] Some methods may include receiving, by the control system, from the microphone system, microphone signals corresponding to at least the first audio device playback sound and the second through Nth audio device playback sounds. In some examples, the second through Nth audio device playback sounds may correspond to second through Nth modified audio playback signals played by the second through Nth audio devices. In some examples, the second through Nth modified audio playback signals may include second through Nth DSSS signals. Some methods may include extracting, by the control system, at least the second through Nth DSSS signals from the microphone signals.

[0074] Some methods may include estimating, by the control system, at least one acoustic scene metric based at least in part on the second through Nth DSSS signals, in some examples, the at least one acoustic scene metric includes one or more of time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, location of audio devices, noise of an audio environment, or signal-to-noise ratio.

[0075] Some methods may include controlling, by the control system, one or more aspects of audio device playback based at least in part on at least one acoustic scene metric, at least one audio device characteristic, or both the at least one acoustic scene metric and the at least one audio device characteristic.

[0076] In some examples, the first content stream component of the first audio device playback may cause perceptual masking of the first DSSS signal component of the first audio device playback.

[0077] Some methods may include determining, by the control system, one or more DSSS parameters for each audio device of a plurality of audio devices in the audio environment. In some examples, the one or more DSSS parameters may be usable for generating DSSS signals. Some methods may include providing, by the control system, the one or more DSSS parameters to each audio device of the plurality of audio devices.

[0078] In some examples, determining the one or more DSSS parameters may include scheduling, for each audio device of the plurality of audio devices, a time slot for playing back a group of modified audio playback signals. In some examples, a first time slot for a first audio device may be different from a second time slot for a second audio device. According to some examples, determining the one or more DSSS parameters may include determining, for each audio device of the plurality of audio devices, a frequency band for playing back a group of modified audio playback signals. In some examples, a first frequency band for a first audio device may be different from a second frequency band for a second audio device.

[0079] According to some examples, determining the one or more DSSS parameters may include determining a spreading code for each audio device of the plurality of audio devices. In some examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device. Some examples may include determining at least one spreading code length based at least in part on corresponding audio device audibility. In some examples, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of the plurality of audio devices in the audio environment.

[0080] In some examples, at least some of the first set of audio signals may correspond to silence.

[0081] Some or all of the operations, functions, and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices as described herein. Memory devices include, but are not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, etc. As such, some innovative aspects of the subject matter described in this disclosure may be implemented via one or more non-transitory media having software stored thereon.

[0082] The details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Note that the relative dimensions of the following figures may not be drawn to scale. [Brief explanation of the drawings]

[0083] Like reference numbers and designations in the various drawings indicate like elements.

[0084] [Figure 1A] FIG. 1A shows an example of an audio environment. [Figure 1B] FIG. 1B is a block diagram illustrating example components of an apparatus capable of implementing various aspects of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating example audio device elements in accordance with some disclosed aspects. [Figure 3] FIG. 3 is a block diagram illustrating example audio device components according to another disclosed aspect. [Figure 4] FIG. 4 is a block diagram illustrating example audio device components according to another disclosed embodiment. [Figure 5]FIG. 5 is a graph showing an example of the level of a content stream component of audio device playback sound and the level of a DSSS signal component of audio device playback sound over a certain frequency range. [Figure 6] FIG. 6 is a graph showing an example of the power of two DSSS signals with different bandwidths but located at the same center frequency. [Figure 7] FIG. 7 illustrates elements of an orchestration module according to one example. [Figure 8] FIG. 8 shows another example of an audio environment. [Figure 9] FIG. 9 shows an example of the main lobe of the acoustic DSSS signals generated by the audio devices 100B and 100C of FIG. [Figure 10] FIG. 10 is a graph showing an example of a time domain multiple access (TDMA) system. [Figure 11] FIG. 11 is a graph showing an example of a frequency domain multiple access (FDMA) scheme. [Figure 12] FIG. 12 is a graph showing another example of an orchestration method. [Figure 13] FIG. 13 is a graph showing another example of an orchestration method. [Figure 14] FIG. 14 shows elements of another example audio environment. [Figure 15] FIG. 15 is a flow diagram outlining another example of the disclosed method for orchestrating an audio device. [Figure 16] FIG. 16 shows another example of an audio environment. [Figure 17] FIG. 17 is a block diagram illustrating an example of a DSSS signal demodulator element, a baseband processor element, and a DSSS signal generator element according to some disclosed aspects. [Figure 18] FIG. 18 shows elements of another example DSSS signal demodulator. [Figure 19]FIG. 19 is a block diagram illustrating an example of a baseband processor element in accordance with some disclosed aspects. [Figure 20] FIG. 20 shows an example of a delayed waveform. [Figure 21] FIG. 21 shows an example of a block according to another aspect. [Figure 22] FIG. 22 shows an example of a block according to yet another aspect. [Figure 23] FIG. 23 is a block diagram illustrating example audio device elements in accordance with some disclosed aspects. [Figure 24] FIG. 24 shows a block diagram of another example embodiment. [Figure 25] FIG. 25 shows another example of an audio environment. [Figure 26] FIG. 26 is a timing diagram according to an example. [Figure 27] FIG. 27 is a timing diagram illustrating clock terms relevant in estimating time of flight between two asynchronous audio devices according to one example. [Figure 28] FIG. 28 is a graph showing an example of detecting the relative clock skew between two audio devices using one acoustic DSSS signal. [Figure 29] FIG. 29 is a graph showing an example of detecting relative clock skew between two audio devices by measuring one acoustic DSSS signal multiple times. [Figure 30] FIG. 30 is a graph illustrating an example of acoustic DSSS spreading codes reserved for device discovery. [Figure 31] FIG. 31 shows another example of an audio environment. [Figure 32A] FIG. 32A illustrates example delayed waveforms generated by audio device 100C of FIG. 31 based on acoustic DSSS signals received from audio devices 100A and 100B. [Figure 32B] FIG. 32B illustrates an example of delayed waveforms generated by audio device 100B of FIG. 31 based on the acoustic DSSS signals received from audio devices 100A and 100C. [Figure 33] FIG. 33 is a flow diagram outlining another example of the disclosed method. [Figure 34] FIG. 34 is a flow diagram outlining another example of the disclosed method. [Figure 35] FIG. 35 is a flow diagram illustrating an example of how multiple audio devices may inter-coordinate a measurement session, according to some aspects. [Figure 36A] FIG. 36A is a flow diagram illustrating an example of how multiple audio devices may inter-coordinate a measurement session, according to some aspects. [Figure 36B] FIG. 36B is a flow diagram illustrating an example of how multiple audio devices may inter-coordinate a measurement session, according to some aspects. DETAILED DESCRIPTION OF THE INVENTION

[0085] Achieving compelling spatial reproduction of media and entertainment content requires evaluating and considering the physical layout and relative capabilities of available speakers. Similarly, providing high-quality audio interactions (both with virtual assistants and distant speakers) requires users to both hear and hear the dialogue being played through loudspeakers. As more cooperating devices are added to the audio environment, overall usefulness for users is expected to increase as devices come within a more common and convenient audio range. Increasing the number of speakers can enhance the spatiality of a media presentation, thereby making it more immersive.

[0086] With sufficient coordination and cooperation between devices, such opportunities and experiences may be possible. Acoustic information about each audio device is a key component of such coordination and cooperation. Such acoustic information includes the audibility of each loudspeaker from various positions in the audio environment and the amount of noise in the audio environment.

[0087] Some conventional methods for mapping and calibrating a population of smart audio devices require a dedicated calibration procedure, whereby known stimuli are played from the audio devices (often one audio device playing at a time) while one or more microphones record. While this process can be appealing to some user groups due to creative sound design, adding, removing, or simply rearranging devices requires repeatedly re-running the process, creating a barrier to widespread adoption. Forcing such procedures on users can disrupt the normal operation of the device and frustrate some users. More rudimentary approaches also prevail, relying on manual user intervention via a software application ("app") and / or a guided process where the user indicates the physical location of the audio devices within the audio environment. These approaches pose additional barriers to user adoption and may provide relatively less information to the system than a dedicated calibration procedure.

[0088] Calibration and mapping algorithms generally require basic acoustic information for each audio device in the audio environment. Many such methods have been proposed, using measurements of various basic acoustic measurements or characteristics. Examples of acoustic characteristics (also referred to herein as "acoustic scene metrics") derived from microphone signals for use in such algorithms include:

[0089] Estimated physical distance between devices (acoustic ranging) Estimated angle between devices (direction of arrival (DoA)) Estimates of the impulse response between devices (e.g., with a swept sine wave stimulus or other measurement signal) Estimated background noise

[0090] However, existing calibration and mapping algorithms are generally not implemented in a way that allows them to respond to changes in the acoustic scene of the audio environment, such as movement of people within the audio environment or relocation of audio devices within the audio environment.

[0091] This disclosure describes techniques involving a direct sequence spread spectrum (DSSS) signal being injected into content being rendered by an audio device. Such methods can enable an audio device to generate observations after receiving signals transmitted by other audio devices in the audio environment. In some aspects, each participating audio device in the audio environment can be configured to generate a DSSS signal, generate a modified audio playback signal by injecting the DSSS signal into a rendered loudspeaker feed signal, and generate a first audio device playback sound by causing a loudspeaker system to play the modified audio playback signal. In some aspects, each participating audio device in the audio environment can be configured to do the foregoing while detecting audio device playback sounds from other orchestrated audio devices in the audio environment and processing the audio device playback sounds to extract the DSSS signal.

[0092] DSSS signals have traditionally been deployed in the context of telecommunications. When used in a telecommunications context, DSSS signals are used to spread transmitted data over a wider frequency range before the transmitted data is sent over a channel to a receiver. In contrast, most or all of the disclosed aspects do not involve using DSSS signals to modify or transmit data. Instead, these disclosed aspects involve transmitting DSSS signals between audio devices in an audio environment. What happens to the transmitted DSSS signal between transmission and reception is itself the transmitted information. This is one major difference between how DSSS signals are used in a telecommunications context and how DSSS signals are used in the disclosed aspects.

[0093] Furthermore, the disclosed aspects relate to the transmission and reception of acoustic DSSS signals, rather than the transmission and reception of electromagnetic DSSS signals. In many disclosed aspects, acoustic DSSS signals are inserted into the content stream to be rendered and played back, such that these acoustic DSSS signals are included in the played back audio. According to some such aspects, acoustic DSSS signals are inaudible to humans, so that a person in the audio environment does not perceive the acoustic DSSS signals, but detects only the played back audio content.

[0094] Another difference between the use of acoustic DSSS signals disclosed herein and how DSSS signals are used in telecommunications contexts relates to what is referred to herein as the "near / far problem." In some examples, the acoustic DSSS signals disclosed herein may be transmitted and received by multiple audio devices in an audio environment. The acoustic DSSS signals may overlap in time and frequency. Some disclosed aspects depend on how DSSS spreading codes are generated to separate the acoustic DSSS signals. In some examples, audio devices may be so close to each other that signal levels may violate the separation of the acoustic DSSS signals, making it difficult to separate the signals. This is one manifestation of the near / far problem, and some solutions are disclosed herein.

[0095] Some methods may include receiving a first content stream including a first group of audio signals, rendering the first group of audio signals to generate a first group of audio playback signals, generating a first group of direct sequence spread spectrum (DSSS) signals, generating a first group of modified audio playback signals by inserting the first group of DSSS signals into the first group of audio playback signals, and generating a first audio device playback sound by playing the first modified audio playback signal through a loudspeaker system. The method(s) may include receiving a group of microphone signals corresponding to at least the first audio device playback sound and second through Nth audio device playback sounds corresponding to second through Nth modified audio playback signals (including the second through Nth DSSS signals) played by second through Nth audio devices, extracting the second through Nth DSSS signals from the microphone signals, and estimating at least one acoustic scene metric based at least in part on the second through Nth DSSS signals.

[0096] The acoustic scene metric(s) may be or may include the audibility of the audio devices, the impulse responses of the audio devices, the angles between the audio devices, the positions of the audio devices, and / or the noise of the audio environment. Some disclosed methods may include controlling one or more aspects of audio device playback based at least in part on the acoustic scene metric(s).

[0097] Some disclosed methods may include orchestrating a plurality of audio devices to perform a method involving DSSS signals. Some such methods may include causing a first audio device in an audio environment, via a control system, to generate first DSSS signals, causing the control system to insert the first DSSS signals into first audio playback signals corresponding to a first content stream to generate first modified audio playback signals for the first audio device, and causing the control system to play the first modified audio playback signals to generate first audio device playback sound.

[0098] Some such methods may include causing a second audio device in the audio environment to generate a second set of DSSS signals, by the control system; causing the control system to insert the second set of DSSS signals into a second content stream to generate a second set of modified audio playback signals for the second audio device; causing the control system to cause the second audio device to play the second set of modified audio playback signals to generate second audio device playback sounds; and causing the control system to cause the second audio device to play the second set of modified audio playback signals to generate second audio device playback sounds.

[0099] Some such aspects may include causing at least one microphone in the audio environment to detect at least the first audio device and the second audio device and generate microphone signals corresponding to at least the first audio device and the second audio device. Some such methods may include causing the control system to extract at least the first DSSS signals and the second DSSS signals from the microphone signals, and causing the control system to estimate at least one acoustic scene metric based at least in part on the first DSSS signals and the second DSSS signals.

[0100] Figure 1A illustrates an example of an audio environment. As with other figures provided herein, the types and number of elements shown in Figure 1A are provided by way of example only. Other implementations may include more, fewer, and / or different types and numbers of elements.

[0101] According to this example, audio environment 130 is a living space in a home. In the example shown in FIG. 1A, audio devices 100A, 100B, 100C, and 100D are disposed within audio environment 130. In this example, audio devices 100A-100D each include a corresponding one of loudspeaker systems 110A, 110B, 110C, and 110D. According to this example, loudspeaker system 110B of audio device 100B includes at least left loudspeaker 110B1 and right loudspeaker 110B2. In this example, audio devices 100A-100D include loudspeakers of different sizes and capabilities. At the time depicted in FIG. 1A, audio devices 100A-100D are generating corresponding instances of audio device playback sounds 120A, 120B1, 120B2, 120C, and 120D.

[0102] In this example, audio devices 100A-100D each include a corresponding one of microphone systems 111A, 111B, 111C, and 111D. Each of microphone systems 111A-111D includes one or more microphones. In some examples, audio environment 130 may include at least one audio device lacking a loudspeaker system or at least one audio device lacking a microphone system.

[0103] In some examples, at least one acoustic event may be occurring in audio environment 130. For example, one such acoustic event may be caused by a speaker issuing a voice command in some cases. In other cases, the acoustic event may be caused at least in part by a variable element in audio environment 130, such as a door or window. For example, when a door opens, sounds from outside audio environment 130 may be perceived more clearly inside audio environment 130. Furthermore, a change in the angle of the door may change some of the echo paths within audio environment 130.

[0104] FIG. 1B is a block diagram illustrating example components of a device capable of implementing various aspects of the present disclosure. As with other figures provided herein, the types and number of elements shown in FIG. 1B are provided merely as examples. Other aspects may include more, fewer, and / or different types and numbers of elements. According to some examples, device 150 may be configured to perform at least some of the methods disclosed herein. In some aspects, device 150 may be or include one or more components of an audio system. For example, device 150 may be an audio device, such as a smart audio device, in some aspects. In other examples, device 150 may be a mobile device (such as a mobile phone), a laptop computer, a tablet device, a television, or other type of device.

[0105] In the example shown in FIG. 1A, audio devices 100A-100D are instances of apparatus 150. According to some examples, audio environment 100 of FIG. 1A may include an orchestration device, such as what may be referred to herein as a smart home hub. The smart home hub (or other orchestration device) may be an instance of apparatus 150. In some aspects, one or more of audio devices 100A-100D may function as an orchestration device.

[0106] According to some alternative aspects, apparatus 150 may be or include a server. In some such examples, apparatus 150 may be or include an encoder. Thus, in some cases, apparatus 150 may be a device configured for use in an audio environment, such as a home audio environment, while in other cases, apparatus 150 may be a device configured for use in the "cloud," e.g., a server.

[0107] In this example, device 150 includes interface system 155 and control system 160. Interface system 155, in some aspects, may include a wired or wireless interface configured to communicate with one or more other devices in an audio environment. The audio environment, in some examples, may be a home audio environment. In other examples, the audio environment may be another type of environment, such as an office environment, an automobile environment, a train environment, a road or sidewalk environment, a park environment, etc. Interface system 155, in some aspects, may be configured to exchange control information and associated data with audio devices in the audio environment. The control information and associated data, in some examples, may relate to one or more software applications that device 150 is executing.

[0108] The interface system 155 may, in some aspects, be configured to receive or provide a content stream. The content stream may include audio data. The audio data may include, but is not limited to, audio signals. In some examples, the audio data may include spatial data, such as channel data and / or spatial metadata. The metadata may be provided, for example, by what may be referred to herein as an “encoder.” In some examples, the content stream may include video data and audio data corresponding to the video data.

[0109] The interface system 155 may include one or more network interfaces and / or one or more external device interfaces (such as one or more Universal Serial Bus (USB) interfaces). According to some aspects, the interface system 155 may include one or more wireless interfaces, such as WiFi or Bluetooth. TM It may include those configured for communication.

[0110] Interface system 155, in some examples, may include one or more devices for implementing a user interface, such as one or more microphones, one or more speakers, a display system, a touch sensor system, and / or a gesture sensor system. In some examples, interface system 155 may include one or more interfaces between control system 160 and a memory system, such as optional memory system 165 shown in FIG. 1B . However, control system 160 may include a memory system in some cases. Interface system 155, in some aspects, may be configured to receive input from one or more microphones in the environment.

[0111] In some aspects, control system 160 may be configured, at least in part, to perform the methods disclosed herein. Control system 160 may include, for example, a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or discrete hardware components.

[0112] In some embodiments, control system 160 may reside on multiple devices. For example, in some embodiments, a portion of control system 160 may reside on a device within one of the environments depicted herein, and another portion of control system 160 may reside on a device external to the environment, such as a server, a mobile device (e.g., a smartphone, or a tablet computer). In other examples, a portion of control system 160 may reside on a device within one of the environments depicted herein, and another portion of control system 160 may reside on one or more other devices in the environment. For example, the functionality of the control system may be distributed among multiple smart audio devices in the environment, or may be shared by an orchestration device (such as what may be referred to herein as a smart home hub) and one or more other devices in the environment. In other examples, a portion of control system 160 may reside on a device implementing a cloud-based service, such as a server, and another portion of control system 160 may reside on another device implementing the cloud-based service, such as another server or memory device. Interface system 155 may also reside on multiple devices in some examples.

[0113] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices as described herein. Such memory devices include, but are not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, etc. One or more non-transitory media may reside, for example, in optional memory system 615 and / or control system 160 illustrated in FIG. 1B. Accordingly, various innovative aspects of the subject matter described in this disclosure may be implemented in one or more non-transitory media having software stored thereon. The software may include, for example, instructions for controlling at least one device to perform some or all of the methods disclosed herein. The software may be executable by one or more components of a control system, such as, for example, control system 160 of FIG. 1B.

[0114] In some examples, device 150 may include optional microphone system 111 shown in FIG. 1B . Optional microphone system 111 may include one or more microphones. According to some examples, optional microphone system 111 may include an array of microphones. The microphone array may, in some cases, be configured to perform receive-side beamforming, for example, according to instructions from control system 160. In some examples, the microphone array may be configured to determine direction of arrival (DOA) and / or time of arrival (TOA) information, for example, according to instructions from control system 160. Alternatively, or additionally, control system 160 may be configured to determine direction of arrival (DOA) and / or time of arrival (TOA) information, for example, according to microphone signals received from microphone system 111.

[0115] In some aspects, one or more of the microphones may be part of or associated with another device, such as a speaker of a speaker system, a smart audio device, etc. In some examples, device 150 may not include microphone system 111. However, in some such aspects, device 150 may nevertheless be configured to receive microphone data of one or more microphones in the audio environment via interface system 160. In some such aspects, cloud-based aspects of device 150 may be configured to receive microphone data, or data corresponding to microphone data, from one or more microphones in the audio environment via interface system 160.

[0116] According to some aspects, device 150 may include optional loudspeaker system 110, shown in FIG. 1B. Optional loudspeaker system 110 may include one or more loudspeakers, also referred to herein as "speakers" or more generally as "audio reproduction transducers." In some examples (e.g., cloud-based aspects), device 150 may not include loudspeaker system 110.

[0117] In some embodiments, device 150 may include optional sensor system 180, shown in FIG. 1B . Optional sensor system 180 may include one or more touch sensors, gesture sensors, motion detectors, etc. According to some embodiments, optional sensor system 180 may include one or more cameras. In some embodiments, the camera may be a free-standing camera. In some examples, one or more cameras of optional sensor system 180 may reside in a smart audio device. The smart audio device may be a single-purpose audio device or a virtual assistant. In some such examples, one or more cameras of optional sensor system 180 may reside in a television, a mobile phone, or a smart speaker. In some examples, device 150 may not include sensor system 180. However, in some such embodiments, device 150 may nevertheless be configured to receive sensor data of one or more sensors in the audio environment via interface system 160.

[0118] In some embodiments, device 150 may include optional display system 185, shown in FIG. 1B . Optional display system 185 may include one or more displays, such as one or more light-emitting diode (LED) displays. In some examples, optional display system 185 may include one or more organic light-emitting diode (OLED) displays. In some examples, optional display system 185 may include one or more displays of a smart audio device. In other examples, optional display system 185 may include a television display, a laptop display, a mobile device display, or other types of displays. In some examples in which device 150 includes display system 185, sensor system 180 may include a touch sensor system and / or a gesture sensor system proximate one or more displays of display system 185. According to some such embodiments, control system 160 may be configured to control display system 185 to present one or more graphical user interfaces (GUIs).

[0119] According to some such examples, device 150 may be or include a smart audio device. In some such aspects, device 150 may be or include a wake word detector. For example, device 150 may be or include a virtual assistant.

[0120] FIG. 2 is a block diagram illustrating example audio device elements according to some disclosed aspects. As with other figures provided herein, the types and number of elements shown in FIG. 2 are provided merely as examples. Other aspects may include more, fewer, and / or different types and numbers of elements. In this example, audio device 100A of FIG. 2 is an instance of apparatus 150 described above with reference to FIG. 1B. In this example, audio device 100A is one of multiple audio devices in an audio environment and, in some cases, may be an example of audio device 100A shown in FIG. 1A. According to this aspect, audio device 100A is one of multiple orchestrated audio devices in the audio environment. In this example, the audio environment includes at least two other orchestrated audio devices, audio device 100B and audio device 100C.

[0121] According to this aspect, audio device 100A includes the following elements:

[0122] 110A: An instance of loudspeaker system 110 of FIG. 1B, including one or more loudspeakers. 111A: An instance of microphone system 111 of FIG. 1B, including one or more microphones. 120A, B, C: A group of audio device playback sounds corresponding to rendered content played on audio devices 100A to 100C in the same acoustic space. 201A: A group of audio playback signals output by the rendering module 210A. 202A: Modified audio playback signals output by DSSS signal injector 211A. 203A: DSSS signals output by DSSS signal generator 212A. 204A: DSSS signal replicas corresponding to DSSS signals generated by other audio devices in the audio environment (in this example, at least audio devices 100B and 100C). In some examples, DSSS signal replicas 204A are received from an external source (e.g., via WiFi or Bluetooth) such as an orchestration device (which may be other audio devices in the audio environment, other local devices such as a smart home hub, etc.). TM The information may be received via a wireless communication protocol such as a radio wave.

[0123] 205A: DSSS information belonging to and / or used by one or more audio devices in the audio environment. DSSS information 205A may include parameters used by control system 160 of audio device 100A to generate DSSS signals, modulate DSSS signals, demodulate DSSS signals, etc. DSSS information 205A may include one or more DSSS spreading code parameters and one or more DSSS carrier parameters. DSSS spreading code parameters may include, for example, DSSS spreading code length information, chipping rate information (or chip period information), etc. One chip period is the time it takes for one chip (bit) of the spreading code to be reproduced. The reciprocal of the chip period is the chipping rate. Bits of a DSSS spreading code are sometimes referred to as "chips" to indicate that they do not contain data (like regular bits). In some examples, DSSS spreading code parameters may include a pseudo-random number sequence. DSSS information 205A may, in some examples, indicate which audio device is generating the acoustic DSSS signals. In some examples, the DSSS information 205A may be received (eg, via wireless communication) from an external source such as an orchestration device.

[0124] 206A: A set of microphone signals received at microphone(s) 111A.

[0125] 208A: Demodulated coherent baseband signal group.

[0126] 210A: A rendering module configured to render audio signals of content streams, such as audio data of music, movies, and TV shows, and generate audio playback signals.

[0127] 211A: A DSSS signal injector configured to insert the DSSS signals 230A modulated by the DSSS signal modulator 220A into the audio playback signals generated by the rendering module 210A to generate modified audio playback signals. The insertion process may, for example, be a mixing process in which the DSSS signals 230A modulated by the DSSS signal modulator 220A are mixed with the audio playback signals generated by the rendering module 210A to generate modified audio playback signals.

[0128] 212A: A DSSS signal generator configured to generate DSSS signal constellation 203A and provide DSSS signal constellation 203A to DSSS signal modulator 220A and DSSS signal demodulator 214A. In this example, DSSS signal generator 212A includes a DSSS spreading code generator and a DSSS carrier generator. In this example, DSSS signal generator 212A provides DSSS signal replica constellation 204A to DSSS signal demodulator 214A.

[0129] 214A: A DSSS signal demodulator configured to demodulate the microphone signals 206A received by the microphone(s) 111A. In this example, the DSSS signal demodulator 214A outputs demodulated coherent baseband signals 208A. Demodulation of the microphone signals 206A may be performed using standard correlation techniques, including, for example, an integrate-and-dump matched filtering correlator bank. Some detailed examples are provided below. To improve the performance of these demodulation techniques, in some aspects, the microphone signals 206A may be filtered before demodulation to remove unwanted content / phenomena. According to some aspects, the demodulated coherent baseband signals 208A may be filtered before being provided to the baseband processor 218A. The signal-to-noise ratio (SNR) generally improves with longer integration times (i.e., longer spreading codes are used).

[0130] 218A: A baseband processor configured for baseband processing of the demodulated coherent baseband signals 208A. In some examples, the baseband processor 218A may be configured to implement techniques such as non-coherent averaging to improve the SNR by reducing the variance of the squared waveform to generate the delayed waveform. Some detailed examples are provided below. In this example, the baseband processor 218A is configured to output one or more estimated acoustic scene metrics 225A.

[0131] 220A: A DSSS signal modulator configured to modulate the DSSS signal group 203A generated by the DSSS signal generator to generate the DSSS signal group 230A.

[0132] 225A: One or more DSSS-derived observations, also referred to herein as acoustic scene metrics. The acoustic scene metric(s) 225A may include or be data corresponding to time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, location of audio devices, noise and / or signal-to-noise ratio of the audio environment.

[0133] 233A: Acoustic scene metric processing module, configured to receive and apply acoustic scene metrics 225A. In this example, acoustic scene metric processing module 233A is configured to generate information 235A (and / or commands) based at least in part on at least one acoustic scene metric 225A and / or at least one audio device characteristic. The audio device characteristic(s) may correspond to audio device 100A or to another audio device in the audio environment, depending on the particular implementation. The audio device characteristic(s) may be stored in a memory of or accessible to control system 160, for example.

[0134] 235A: Information for controlling one or more aspects of audio processing and / or audio device playback. Information 235A may include, for example, information (and / or commands) for controlling a rendering process, an audio environment mapping process (such as an audio device auto-positioning process), an audio device calibration process, a noise suppression process, and / or an echo attenuation process.

[0135] Acoustic Scene Metrics Examples As mentioned above, in some aspects, the baseband processor 218A (or another module of the control system 160) may be configured to determine one or more acoustic scene metrics 225A. The following are some examples of acoustic scene metrics 225A:

[0136] Distance measurement The DSSS signal received by the audio device from another device contains information about the distance between the two devices in the form of the signal's time of flight (ToF). Thus, according to some examples, the control system may be configured to extract delay information from the demodulated DSSS signal and convert the delay information into a pseudorange measurement, for example, as follows:

[0137]

number

[0138] In the above equation, τ represents delay information (also referred to herein as ToF), ρ represents a pseudo-range measurement, and c represents the speed of sound. It is called "pseudo-range" because the range between devices is estimated according to timing estimates, rather than the range itself being measured directly. In a distributed, asynchronous system of audio devices, each audio device operates on its own clock, so biases exist in the raw delay measurements. With a sufficient set of delay measurements, these biases can be eliminated, and sometimes even estimated. Detailed examples of extracting delay information, generating and using pseudo-range measurements, and determining and resolving clock biases are provided below.

[0139] DoA Similar to ranging, using multiple microphones available on the listening device, the control system can be configured to estimate direction of arrival (DoA) by processing demodulated acoustic DSSS signals. In some such aspects, the resulting DoA information can be used as input to a DoA-based audio device automatic location method.

[0140] Audibility The signal strength of the demodulated acoustic DSSS signal is proportional to the audibility of the audio device being heard in the band over which the audio device is transmitting the acoustic DSSS signals. In some aspects, the control system can be configured to obtain a banded estimate of the entire frequency range by making multiple observations across a range of frequency bands. If the digital signal level of the transmitting audio device is known, the control system can, in some examples, be configured to estimate the absolute acoustic gain of the transmitting audio device.

[0141] FIG. 3 is a block diagram illustrating example audio device elements according to another disclosed aspect. As with other figures provided herein, the types and number of elements shown in FIG. 3 are provided merely as examples. Other aspects may include more, fewer, and / or different types and numbers of elements. In this example, audio device 100A of FIG. 3 is an instance of apparatus 150 described above with reference to FIGS. 1B and 2. However, according to this aspect, audio device 100A is configured to organize multiple audio devices in an audio environment, including at least audio devices 100B, 100C, and 100D.

[0142] The embodiment shown in Figure 3 includes all of the elements of Figure 2, plus some additional elements. Elements common to Figures 2 and 3 will not be described again here, except to the extent that their functionality may differ in the embodiment of Figure 3. According to this embodiment, audio device 100A includes the following elements and functionality:

[0143] 120A, B, C, D: A group of audio device playback sounds corresponding to rendered content being played on audio devices 100A to 100D in the same acoustic space.

[0144] 204A, B, C, D: DSSS signal replicas corresponding to DSSS signals generated by other audio devices in the audio environment (in this example, at least audio devices 100B, 100C, and 100D). In this example, DSSS signal replicas 204A-204D are provided by orchestration module 213A, which provides DSSS information 204B-204D to audio devices 100B-100D, for example, via wireless communication.

[0145] 205A, B, C, D: These elements correspond to DSSS information belonging to and / or used by each of audio devices 100A-100D. DSSS information 205A may include parameters (e.g., one or more DSSS spreading code parameters, one or more DSSS carrier parameters, etc.) used by control system 160 of audio device 100A to generate, modulate, demodulate, etc. the DSSS signals. DSSS information 205B, 205C, and 205D may include parameters (e.g., one or more DSSS spreading code parameters and one or more DSSS carrier parameters) used by audio devices 100B, 100C, and 100D, respectively, to generate, modulate, demodulate, etc. the DSSS signals. DSSS information 205A-205D may, in some examples, indicate which audio devices are generating the acoustic DSSS signals.

[0146] 213A: Orchestration Module. In this example, orchestration module 213A generates DSSS information 205A-205D and provides DSSS information 205A to DSSS signal generator 212A, provides DSSS information 205A-205D to DSSS signal demodulators, and provides DSSS information 205B-205D to audio devices 100B-100D, e.g., via wireless communication. In some examples, orchestration module 213A generates DSSS information 205A-205D based at least in part on information 235A-235D and / or acoustic scene metrics 225A-225D.

[0147] 214A: A DSSS signal demodulator configured to demodulate microphone signals 206A received by at least microphone(s) 111A. In this example, DSSS signal demodulator 214A outputs demodulated coherent baseband signals 208A. In some alternative aspects, DSSS signal demodulator 214A can receive and demodulate microphone signals 206B-206D from audio devices 100B-100D and output demodulated coherent baseband signals 208B-208D.

[0148] 218A: A baseband processor configured for baseband processing of at least the demodulated coherent baseband signals 208A, and in some examples, the demodulated coherent baseband signals 208B-208D received from the audio devices 100B-100D. In this example, the baseband processor 218A is configured to output one or more estimated acoustic scene metrics 225A-225D. In some aspects, the baseband processor 218A is configured to determine the acoustic scene metrics 225B-225D based on the demodulated coherent baseband signals 208B-208D received from the audio devices 100B-100D. However, in some cases, the baseband processor 218A (or the acoustic scene metric processing module 233A) may receive the acoustic scene metrics 225B-225D from the audio devices 100B-100D.

[0149] 233A: Acoustic scene metric processing module configured to receive and apply set of acoustic scene metrics 225A-225D. In this example, acoustic scene metric processing module 233A is configured to generate information 235A-235D based at least in part on set of acoustic scene metrics 225A-225D and / or at least one audio device characteristic. The audio device characteristic(s) may correspond to one or more of audio device 100A and / or audio devices 100B-100D.

[0150] FIG. 4 is a block diagram illustrating example audio device elements according to another disclosed embodiment. As with other figures provided herein, the types and number of elements shown in FIG. 4 are provided merely as examples. Other embodiments may include more, fewer, and / or different types and numbers of elements. In this example, audio device 100A of FIG. 4 is an instance of apparatus 150 described above with reference to FIGS. 1B, 2, and 3. The example of FIG. 4 includes all of the elements of FIG. 3 plus additional elements. Elements common to FIGS. 2 and 3 will not be described again here, except to the extent that their functionality may differ in the embodiment of FIG. 4.

[0151] According to this aspect, control system 160 is configured to process received microphone signals 206A to generate preprocessed microphone signals 207A. In some aspects, processing the received microphone signals may include applying a bandpass filter and / or echo cancellation. In this example, control system 160 (more specifically, DSSS signal demodulator 214A) is configured to extract DSSS signals from preprocessed microphone signals 207A.

[0152] According to this example, microphone system 111A includes an array of microphones, which in some cases may be or include one or more directional microphones. In this aspect, processing the received microphone signals includes receive-side beamforming, in this example via beamformer 215A. In this example, preprocessed microphone signals 207A output by beamformer 215A are or include spatial microphone signals.

[0153] In this manner, DSSS signal demodulator 214A processes spatial microphone signals, thereby improving performance for audio systems in which audio devices are spatially distributed throughout the audio environment. Receiver beamforming is one way to avoid the "near-far problem" described above. For example, control system 160 may be configured to use beamforming to receive audio device playback from more distant and / or quieter audio devices, compensating for closer and / or louder audio devices.

[0154] Receiver beamforming may include, for example, delaying and multiplying signals from each microphone in a microphone array by different coefficients. In some examples, the beamformer 215A may apply a Dolph-Chebyshev weighting pattern. However, in other embodiments, the beamformer 215A may apply a different weighting pattern. According to some such examples, a main lobe may be generated along with nulls and side lobes. In addition to controlling the main lobe width (beamwidth) and side lobe levels, in some examples, the location of the nulls may also be controlled.

[0155] Inaudible signal According to some aspects, the DSSS signal component of the audio device playback may be inaudible to a person in the audio environment, and in some such aspects, the content stream component of the audio device playback may cause perceptual masking of the DSSS signal component of the audio device playback.

[0156] 5 is a graph showing an example of the level of a content stream component of an audio device and the level of a DSSS signal component of an audio device over a range of frequencies, where curve 501 corresponds to the level of the content stream component and curve 530 corresponds to the level of the DSSS signal component.

[0157] A DSSS signal typically includes data, a carrier signal, and a spreading code. Without the need to transmit data over a channel, the modulated signal s(t) can be expressed as:

[0158]

number

[0159] where A represents the amplitude of the DSSS signal, C(t) represents the spreading code, and Sin() represents a sine wave carrier at a carrier frequency of f Hz. Curve 530 in Figure 5 corresponds to an example of s(t) in the above equation.

[0160] One potential advantage of some of the disclosed aspects involving acoustic DSSS signals is that spreading the signal reduces the amplitude of the DSSS signal components for a given amount of energy in the acoustic DSSS signal, thereby reducing the perceptibility of the DSSS signal components in the sound reproduced by the audio device.

[0161] This allows the DSSS signal component of the audio device playback (e.g., as represented by curve 530 in FIG. 5 ) to be placed at a level sufficiently lower than the level of the content stream component of the audio device playback (e.g., as represented by curve 501 in FIG. 5 ) so that the DSSS signal component is not perceived by the listener. Some disclosed aspects exploit the masking properties of the human auditory system to maximize the signal-to-noise ratio (SNR) of the derived DSSS signal observations and / or optimize parameters of the DSSS signal to reduce the probability of perception of the DSSS signal component. Some disclosed examples include applying weights to the level of the content stream component and / or applying weights to the level of the DSSS signal component. Some such examples apply a noise compensation method in which the acoustic DSSS signal component is treated as a signal and the content stream component is treated as noise. Some such examples include applying one or more weights according to (e.g., proportional to) a playback / listening target metric.

[0162] DSSS spreading code As noted elsewhere in this specification, in some examples, the DSSS information 205 provided by the orchestration device (e.g., provided by the orchestration module 213A described above with reference to FIG. 3) may include one or more DSSS spreading code parameters.

[0163] The spreading codes used to spread the carrier to generate the DSSS signal(s) are extremely important. A set of DSSS spreading codes is preferably selected so that the corresponding DSSS signals have the following properties:

[0164] 1. Sharp main lobe in the autocorrelation waveform. 2. Low sidelobes at non-zero delays of the autocorrelation waveform. 3. When multiple devices access the medium simultaneously (e.g., when simultaneously playing modified audio playback signals that include DSSS signal components), there is low cross-correlation between any two spreading codes in the set of spreading codes used. 4. DSSS signals are unbiased (have zero DC component).

[0165] A family of spreading codes (e.g., Gold codes commonly used in GPS contexts) typically exhibits these four characteristics. When multiple audio devices are simultaneously playing modified audio playback signals containing DSSS signal components, and each audio device uses a different spreading code (with good cross-correlation properties, e.g., low cross-correlation), the receiving audio device should be able to simultaneously receive and process all of the acoustic DSSS signals by using a code domain multiple access (CDMA) scheme. Using a CDMA scheme, multiple audio devices can simultaneously transmit acoustic DSSS signals, possibly using a single frequency band. The spreading codes can be generated on-the-fly and / or pre-generated and stored in memory in a data structure, such as a look-up table.

[0166] To implement DSSS, in some examples, binary phase shift keying (BPSK) modulation may be utilized. Additionally, DSSS spreading codes may be interplexed orthogonally to one another to implement a quadrature phase shift keying (QPSK) system, in some examples, as follows:

[0167]

number

[0168] In the above formula, A I and A Q represent the amplitudes of the in-phase and quadrature signals, respectively, and C I and C Qwhere f represents the code sequences of the in-phase and quadrature signals, respectively, and f represents the center frequency (8200) of the DSSS signal. The above are example coefficients that parameterize the DSSS carrier and DSSS spreading code, according to some examples. These parameters are an example of the DSSS information 205 mentioned above. As mentioned above, the DSSS information 205 may be provided by an orchestration device, such as orchestration module 213A, and may be used, for example, by signal generator block 212 to generate the DSSS signal.

[0169] 6 is a graph illustrating an example of the power of two DSSS signals with different bandwidths but located at the same center frequency. In these examples, FIG. 6 shows the spectra of two DSSS signals 630A and 630B centered at the same center frequency 605. In some examples, DSSS signal 630A may be generated by one audio device in the audio environment (e.g., audio device 100A), and DSSS signal 630B may be generated by another audio device in the audio environment (e.g., audio device 100B).

[0170] According to this example, DSSS signal 630B is chipped at a higher rate (i.e., more bits per second are used in the spread signal) than DSSS signal 630A, resulting in a larger bandwidth 610B for DSSS signal 630B than bandwidth 610A for DSSS signal 630A. For a given amount of energy in each DSSS signal, the larger the bandwidth of DSSS signal 630B, the lower the amplitude and perceptibility of DSSS signal 630B relative to DSSS signal 630A. A higher bandwidth DSSS signal also results in higher delay resolution of the baseband data product, leading to higher resolution estimates of acoustic scene metrics based on the DSSS signal (e.g., time-of-flight estimates, time-of-arrival (ToA) estimates, range estimates, direction-of-arrival (DoA) estimates, etc.). However, a higher bandwidth DSSS signal also results in a wider noise bandwidth at the receiver, reducing the SNR of the extracted acoustic scene metrics. Furthermore, if the bandwidth of the DSSS signal is too large, coherence and fading problems associated with DSSS signals can occur.

[0171] The length of the spreading code used to generate the DSSS signal limits the amount of cross-correlation cancellation. For example, a 10-bit Gold code has only -26 dB of cancellation from adjacent codes. This can result in an example of the near-far problem mentioned above, where a relatively low-amplitude signal is obscured by the cross-correlated noise of another, louder signal. Part of the novelty of the systems and methods described in this disclosure includes an orchestration scheme designed to mitigate or avoid such problems.

[0172] Orchestration Method FIG. 7 illustrates elements of an orchestration module according to one example. As with other figures provided herein, the types and number of elements illustrated in FIG. 7 are provided merely as examples. Other embodiments may include more, fewer, and / or different types and numbers of elements. According to some examples, orchestration module 213 may be implemented by an instance of device 150 described above with reference to FIG. 1B. In some such examples, orchestration module 213 may be implemented by an instance of control system 160. In some examples, orchestration module 213 may be an instance of the orchestration module described above with reference to FIG. 3. In some such examples,

[0173] According to this aspect, the orchestration module 213 includes a perceptual model application module 710 , an acoustic model application module 711 , and an optimization module 712 .

[0174] In this example, the perceptual model application module 710 is configured to apply a model of the human auditory system to make one or more perceptual impact estimates 702 of the perceptual impact of the acoustic DSSS signals on a listener in an acoustic space based at least in part on the a priori information 701. The acoustic space may be, for example, an audio environment in which audio devices orchestrated by the orchestration module 213 are located, a room in such an audio environment, etc. The estimate(s) 702 may change over time. In some examples, the perceptual impact estimate(s) 702 may be estimates of a listener's ability to perceive the acoustic DSSS signals based, for example, on the type and level of audio content (if any) currently being played in the acoustic space. The perceptual model application module 710 may be configured to apply one or more models of auditory masking, such as masking as a function of frequency and loudness, spatial auditory masking, etc. The perceptual model application module 710 may be configured to apply one or more models of human loudness perception, such as, for example, human loudness perception as a function of frequency.

[0175] According to some examples, the a priori information 701 may be or include information related to the acoustic space, information related to the propagation of acoustic DSSS signals in the acoustic space, and / or information related to listeners known to use the acoustic space. For example, the a priori information 701 may include information regarding the number of audio devices (e.g., of orchestrated audio devices) in the acoustic space, the locations of the audio devices, the capabilities of the loudspeaker systems and / or microphone systems of the audio devices, information regarding the impulse response of the audio environment, information regarding one or more doors and / or windows of the audio environment, information regarding audio content currently being played in the acoustic space, etc. In some examples, the a priori information 701 may include information regarding the hearing capabilities of one or more listeners.

[0176] In this aspect, the acoustic model application module 711 is configured to perform one or more acoustic DSSS signal performance estimates 703 of acoustic DSSS signals in the acoustic space based at least in part on the a priori information 701. For example, the acoustic model application module 711 may be configured to estimate how well each audio device's microphone system can detect acoustic DSSS signals from other audio devices in the acoustic space, which may be referred to herein as an aspect of the "interaudibility" of the audio devices. Such interaudibility may, in some cases, be an acoustic scene metric previously estimated by the baseband processor based at least in part on previously received acoustic DSSS signals. In some such aspects, the interaudibility estimate may be part of the a priori information 701, and in some such aspects, the orchestration module 213 may not include the acoustic model application module 711. However, in some aspects, the interaudibility estimate may be performed independently by the acoustic model application module 711.

[0177] In this example, optimization module 712 is configured to determine DSSS parameters 705 for all audio devices orchestrated by orchestration module 213 based at least in part on perceptual impact estimate(s) 702 and acoustic DSSS signal performance estimates 703, and current playback / listening intent information 704. Current playback / listening intent information 704 may indicate, for example, the relative need for new acoustic scene metrics based on the acoustic DSSS signals.

[0178] For example, if one or more audio devices are newly powered on in the acoustic space, there may be a strong need for new sets of acoustic scene metrics related to audio device auto-location, audio device inter-audibility, etc. The new sets of acoustic scene metrics may be based, at least in part, on acoustic DSSS signals. Similarly, if existing audio devices are moved within the acoustic space, there may be a strong need for new sets of acoustic scene metrics. Similarly, if new noise sources are present in or near the acoustic space, there may be a strong need to determine new sets of acoustic scene metrics.

[0179] If the current playback / listening intent information 704 indicates a high need for determining new acoustic scene metrics, the optimization module 712 may be configured to determine the DSSS parameters 705 by placing a relatively higher weight on the acoustic DSSS signal performance estimate(s) 703 than on the perceptual impact estimate(s) 702. For example, the optimization module 712 may be configured to determine the DSSS parameters 705 by placing an emphasis on the system's ability to generate high SNR observations of the acoustic DSSS signals and a de-emphasis on the impact / perceptibility of the acoustic DSSS signals to a user. In some such examples, the DSSS parameters 705 may correspond to an audible acoustic DSSS signal.

[0180] However, if no recent changes have been detected in or near the acoustic space and at least an initial estimate of one or more acoustic scene metrics has been made, the need for a new set of acoustic scene metrics may be less.If no recent changes have been detected in or near the acoustic space and at least an initial estimate of one or more acoustic scene metrics has been made, and audio content is currently being played in the acoustic space, the relative importance of immediately estimating one or more new acoustic scene metrics may be even less.

[0181] If the current playback / listening intent information 704 indicates a low level of need for determining new acoustic scene metrics, the optimization module 712 may be configured to determine the DSSS parameters 705 by placing relatively less weight on the acoustic DSSS signal performance estimate(s) 703 than on the perceptual impact estimate(s) 702. In such an example, the optimization module 712 may be configured to determine the DSSS parameters 705 by de-emphasizing the system's ability to generate high SNR observations of the acoustic DSSS signals and by emphasizing the impact / perceptibility of the acoustic DSSS signals to a user. In some such examples, the DSSS parameters 705 may correspond to sub-audible acoustic DSSS signals.

[0182] As described later in this document (e.g., in other examples of audio device orchestration), the parameters of the acoustic DSSS signals provide a wealth of variety in how the orchestration device can modify the acoustic DSSS signals to enhance the performance of the audio system.

[0183] FIG. 8 shows another example of an audio environment. In FIG. 8, audio devices 100B and 100C are distances 810 and 811, respectively, from device 100A. In this particular situation, distance 811 is greater than distance 810. Assuming that audio devices 100B and 100C are producing audio device playback sounds at approximately the same level, this means that audio device 100A will receive the acoustic DSSS signals from audio device 100C at a lower level than the acoustic DSSS signals from audio device 100B due to the additional acoustic loss caused by distance 811. In some embodiments, audio devices 100B and 100C can be organized to enhance the ability of audio device 100A to extract the acoustic DSSS signals and determine acoustic scene metrics based on the acoustic DSSS signals.

[0184] 9 shows an example of the main lobes of the acoustic DSSS signals generated by audio devices 100B and 100C of FIG. 8. In this example, these acoustic DSSS signals have the same bandwidth and are located at the same frequency, but have different amplitudes. Here, the main lobe of acoustic DSSS signal 230B is generated by audio device 100B, and the main lobe of acoustic DSSS signal 230C is generated by audio device 100C. According to this example, the peak power of acoustic DSSS signal 230B is 905B, and the peak power of acoustic DSSS signal 230C is 905C. Here, acoustic DSSS signal 230B and acoustic DSSS signal 230C have the same center frequency 901.

[0185] In this example, an orchestration device (which in some examples includes an instance of orchestration module 213 of FIG. 7 and in some cases may be audio device 100A of FIG. 8 ) enhances the ability of audio device 100A to extract the acoustic DSSS signals by equalizing the digital levels of the acoustic DSSS signals generated by audio devices 100B and 100C so that the peak power of acoustic DSSS signal 230C is greater than the peak power of acoustic DSSS signal 230B by a factor that offsets the difference in acoustic loss due to the difference in distances 810 and 811. Thus, according to this example, audio device 100A receives acoustic DSSS signals 230B from audio device 100C at approximately the same level as the acoustic DSSS signals it receives from audio device 100B, due to the additional acoustic loss due to the longer distance 811.

[0186] The surface area around a point sound source increases with the square of the distance from the sound source. This means that the same sound energy from the sound source is distributed over a larger area, and the energy intensity decreases with the square of the distance from the sound source, according to the inverse square law. If distance 810 is set to b and distance 811 is set to c, the sound energy received by audio device 100A from audio device 100B will be 1 / b 2, and the sound energy received by audio device 100A from audio device 100C is proportional to 1 / c 2 The difference in sound energy is proportional to 1 / (c 2 -b 2 ) . Thus, in some aspects, the orchestration device may calculate the energy generated by the audio device 100C as (c 2 -b 2 ) times. This is one example of how DSSS parameters can be modified to improve performance.

[0187] In some aspects, the optimization process may be more complex and may consider more factors than the inverse square law. In some examples, equalization may be performed via a full-band gain applied to the DSSS signal or via an equalization (EQ) curve that allows for equalization of a non-flat (frequency-dependent) response of microphone system 110A.

[0188] FIG. 10 is a graph illustrating an example of a time domain multiple access (TDMA) scheme. One way to avoid the near-far problem is to organize multiple audio devices transmitting and receiving acoustic DSSS signals so that each audio device is scheduled to play its acoustic DSSS signal in a different time slot. This is known as a TDMA scheme. In the example shown in FIG. 10, an orchestration device causes audio devices 1, 2, and 3 to emit acoustic DSSS signals according to the TDMA scheme. In this example, audio devices 1, 2, and 3 emit acoustic DSSS signals in the same frequency band. According to this example, the orchestration device causes audio device 3 to emit acoustic DSSS signals from time t0 to time t1. Then, the orchestration device causes audio device 2 to emit acoustic DSSS signals from time t1 to time t2. Then, the orchestration device causes audio device 1 to emit acoustic DSSS signals from time t2 to time t3.

[0189] Therefore, in this example, no two DSSS signals are transmitted or received simultaneously. Therefore, the remaining DSSS signal parameters, such as amplitude, bandwidth, and length (as long as each DSSS signal stays within its assigned time slot), are not relevant for multiple access. However, these DSSS signal parameters are relevant to the quality of the observations extracted from the DSSS signals.

[0190] FIG. 11 is a graph illustrating an example of a frequency domain multiple access (FDMA) scheme. In some aspects (e.g., due to limited bandwidth of DSSS signals), an orchestration device may be configured to cause an audio device to simultaneously receive acoustic DSSS signals from two other audio devices in an audio environment. In some such examples, the audio devices transmitting the acoustic DSSS signals may reproduce the signals in different frequency bands, resulting in significantly different received power levels. This is an FDMA scheme. In the example of the FDMA scheme shown in FIG. 11, the main lobes of DSSS signals 230B and 230C are transmitted simultaneously from different audio devices, but have different center frequencies (f1, f2) and different frequency bands (b1, b2). In this example, the frequency bands b1 and b2 of the main lobes do not overlap. Such an FDMA scheme is advantageous when the acoustic losses associated with the paths of the acoustic DSSS signals are significantly different.

[0191] In some embodiments, the orchestration device may be configured to vary the FDMA, TDMA, or CDMA scheme to mitigate the near-far problem. In some examples, the length of the DSSS spreading code may be changed depending on the relative audibility of devices in the room. As described above with reference to FIG. 6, for the same amount of energy in an acoustic DSSS signal, increasing the bandwidth of the spreading code results in the acoustic DSSS signal having a relatively lower maximum power and therefore a lower audibility. Alternatively, or additionally, in some embodiments, DSSS signals may be arranged orthogonal to one another. Such an implementation allows the system to simultaneously have DSSS signals with different spreading code lengths. Alternatively, or additionally, in some embodiments, the energy of each DSSS signal may be modified to reduce the effects of the near-far problem (e.g., to increase the level of acoustic DSSS signals generated by relatively quieter and / or more distant transmitting audio devices) and / or to obtain an optimal signal-to-noise ratio for a given operational objective.

[0192] Figure 12 is a graph illustrating another example of an orchestration method. The elements of Figure 12 are as follows:

[0193] 1210, 1211 and 1212: Non-overlapping frequency bands. 230Ai, Bi, and Ci: Multiple acoustic DSSS signals time-division multiplexed within frequency band 1210. Although audio devices 1, 2, and 3 may appear to use different portions of frequency band 1210, in this example, the main lobes of acoustic DSSS signals 230Ai, Bi, and Ci span most or all of frequency band 1210. 230D and E: Multiple acoustic DSSS signals code division multiplexed within frequency band 1211. Although audio devices 4 and 5 may appear to use different portions of frequency band 1211, in this example the main lobes of acoustic DSSS signals 230D and 230E extend across most or all of frequency band 1211. 230Aii, Bii, and Cii: Multiple acoustic DSSS signals code division multiplexed within frequency band 1212. Although audio devices 1, 2, and 3 may appear to use different portions of frequency band 1210, in this example, the main lobes of acoustic DSSS signals 230Aii, Bii, and Cii extend across most or all of frequency band 1212.

[0194] 12 shows an example of how TDMA, FDMA, and CDMA may be used together in certain aspects of the present invention. In frequency band 1 (1210), TDMA is used to organize acoustic DSSS signals 230Ai, Bi, and Ci transmitted by audio devices 1-3, respectively. Frequency band 1210 is a single frequency band within which acoustic DSSS signals 230Ai, Bi, and Ci cannot simultaneously reside without overlapping.

[0195] In frequency band 2 (1211), CDMA is used to organize acoustic DSSS signals 230D and 230E from audio devices 4 and 5, respectively. In this particular example, acoustic DSSS signal 230D is generated using a longer DSSS spreading code than the DSSS spreading code used to generate acoustic DSSS signal 230E. A shorter DSSS spreading code period for audio device 5 is useful when audio device 5 is louder than audio device 4, as seen by the receiving audio device, because a shorter DSSS spreading code period results in a wider bandwidth and lower peak frequencies in the resulting DSSS signal. The signal-to-noise ratio (SNR) may also be improved by the relatively longer DSSS spreading code period for acoustic DSSS signal 230D.

[0196] In frequency band 3 (1212), CDMA is used to organize the acoustic DSSS signals 230Aii, Bii, and Cii transmitted by audio devices 1-3, respectively. These acoustic DSSS signals are alternate codes transmitted by audio devices 1-3, which are simultaneously transmitting TDMA-organized acoustic DSSS signals to the same audio devices in frequency band 1210. This is a form of FDMA, where longer spreading codes are placed in one frequency band (1212) and transmitted simultaneously (no TDMA), while shorter spreading codes are placed in another frequency band (1210) where TDMA is used.

[0197] FIG. 13 is a graph illustrating another example of an orchestration method. In this embodiment, audio device 4 transmits orthogonal acoustic DSSS signals 230Di and 230Dii, and audio device 5 transmits orthogonal acoustic DSSS signals 230Ei and 230Eii. In this example, all acoustic DSSS signals are transmitted simultaneously within a single frequency band 1310. In this case, orthogonal acoustic DSSS signals 230Di and 230Ei are longer than the in-phase signals 230Dii and 230Eii transmitted by the two audio devices. As a result, each audio device has a higher SNR set of observations derived from acoustic DSSS signals 230Di and 230Ei, as well as a faster, noisier set of observations derived from acoustic DSSS signals 230Dii and 230Eii, albeit at a lower update rate. This is an example of a CDMA-based orchestration method in which two audio devices are transmitting acoustic DSSS signals designed for the acoustic space they share. In some examples, the orchestration method may be based at least in part on the current listening objective.

[0198] FIG. 14 illustrates elements of an audio environment according to another example. In this example, audio environment 1401 is a multi-room dwelling that includes acoustic spaces 130A, 130B, and 130C. According to this example, doors 1400A and 1400B can alter the coupling of each acoustic space. For example, when door 1400A is open, acoustic spaces 130A and 130C are acoustically coupled to at least some extent, while when door 1400A is closed, acoustic spaces 130A and 130C are not acoustically coupled to any significant degree. In some aspects, an orchestration device may be configured to detect the opening of a door (or the movement of another acoustic obstruction) in response to the detection, or lack thereof, of audio device playback sound in an adjacent acoustic space.

[0199] In some examples, the orchestration device can orchestrate all of the audio devices 100A-100E in all of the acoustic spaces 130A, 130B, and 130C. However, due to a significant level of acoustic separation between the acoustic spaces 130A, 130B, and 130C when the doors 1400A and 1400B are closed, in some examples, the orchestration device can treat the acoustic spaces 130A, 130B, and 130C as independent when the doors 1400A and 1400B are closed. In some examples, the orchestration device can treat the acoustic spaces 130A, 130B, and 130C as independent even when the doors 1400A and 1400B are open. However, in some cases, the orchestration device may manage audio devices located near doors 1400A and / or 1400B such that when acoustic spaces are joined by opening the doors, the audio devices near the open door are treated as corresponding audio devices for the rooms on either side of the door. For example, if the orchestration device determines that door 1400A is open, the orchestration device may be configured to consider audio device 100C to be an audio device for acoustic space 130A and also to be an audio device for acoustic space 130C.

[0200] FIG. 15 is a flow diagram outlining another example of the disclosed audio device orchestration method. The blocks of method 1500, as with other methods described herein, are not necessarily performed in the order shown. Also, such methods may include more or fewer blocks than those shown and / or described. Method 1500 may be performed by a system including an orchestration device and an audio device to be orchestrated. The system may include instances of apparatus 150 shown in FIG. 1B and described above, one of which is configured as the orchestration device. The orchestration device may, in some examples, include an instance of orchestration module 213 disclosed herein.

[0201] According to this example, block 1505 includes steady-state operation of all participating audio devices. In this context, "steady-state" operation means operating according to a set of parameters most recently received from the orchestration device. According to this aspect, the set of parameters includes one or more DSSS spreading code parameters and one or more DSSS carrier parameters.

[0202] In this example, block 1505 also includes one or more devices waiting for a trigger condition. The trigger condition may be, for example, an acoustic change in the audio environment in which the orchestrated audio device is located. The acoustic change may be or include a noise from a noise source, a change corresponding to the opening or closing of a door or window (e.g., an increase or decrease in audibility of sound played from one or more loudspeakers in an adjacent room), detected movement of an audio device within the audio environment, detected movement of a person within the audio environment, detected speech (e.g., of a wake word) by a person within the audio environment, the start of playback of audio content (e.g., the start of a movie, television program, music content, etc.), a change in audio content playback (e.g., a volume change equal to or greater than a threshold change in decibels), etc. In some examples, the acoustic change is detected, for example, via acoustic DSSS signals (e.g., one or more acoustic scene metrics 225A estimated by baseband processor 218 of an audio device within the audio environment) as disclosed herein.

[0203] In some examples, the trigger condition may be an indication that a new audio device has been powered on in the audio environment. In some such examples, the new audio device may be configured to generate one or more characteristic sounds, which may or may not be audible to humans. According to some examples, the new audio device may be configured to reproduce an acoustic DSSS signal in accordance with a DSSS spreading code of a type that is reserved for the new device. Some examples of reserved DSSS spreading codes are described below.

[0204] In this example, in block 1510, it is determined whether a trigger condition is detected. If so, the process proceeds to block 1515. If not, the process returns to block 1505. In some aspects, block 1505 may include block 1510.

[0205] According to this example, block 1515 includes determining, by the orchestration device, one or more updated acoustic DSSS parameters for one or more (possibly all) of the orchestrated audio devices and providing the updated acoustic DSSS parameter(s) to the orchestrated audio device(s). In some examples, block 1515 may include providing, by the orchestration device, DSSS information 205, described elsewhere herein. Determining the updated acoustic DSSS parameter(s) may include using existing knowledge and estimates of the acoustic space, such as:

[0206] Device location · Device range; · Device orientation and relative angle of incidence; Relative clock bias and skew between devices; · Relative audibility of the device; · Indoor noise estimate; · Number of microphones and loudspeakers on each device; ·Directivity of the loudspeakers of each device; - Microphone directionality for each device; · The type of content being rendered in the acoustic space; the position of one or more listeners in the acoustic space; and / or · Knowledge of acoustic space, including specularity and occlusion.

[0207] Such factors may, in some instances, be combined with certain operating objectives to determine a new operating point. Note that many of the parameters used as prior knowledge in determining updated DSSS parameters may themselves be derived from acoustic DSSS parameters. Thus, it can be readily seen that, in some instances, an orchestrated acoustic DSSS system can iteratively improve its performance as the system acquires more information, more accurate information, etc.

[0208] In this example, block 1520 includes reconfiguring, by one or more orchestrated audio devices, one or more parameters used to generate the acoustic DSSS signals according to the updated acoustic DSSS parameter(s) received from the orchestration device. According to this aspect, after block 1520 is completed, the process returns to block 1505. Although an end is not shown in the flow diagram of FIG. 15, method 1500 can end in various ways, such as, for example, when an audio device is powered down.

[0209] FIG. 16 shows another example of an audio environment. Audio environment 130 shown in FIG. 16 is similar to that shown in FIG. 8, but also shows the angular separation of audio devices 100B and 100C from the perspective of audio device 100A (relative to audio device 100A). In FIG. 16, audio devices 100B and 100C are spaced apart from device 100A by distances 810 and 811, respectively. In this particular situation, distance 811 is greater than distance 810. Assuming that audio devices 100B and 100C are producing audio device playback sounds at approximately the same level, this means that audio device 100A will receive the acoustic DSSS signals from audio device 100C at a lower level than the acoustic DSSS signals from audio device 100B due to the additional acoustic loss caused by distance 811.

[0210] In this example, we focus on the alignment of devices 100B and 100C to optimize device 100A's ability to hear both devices 100B and 100C. While there are other factors to consider, as outlined above, this example focuses on the diversity in angles of arrival caused by the angular separation of audio devices 100B and 100C relative to audio device 100A. Due to the difference in distances 810 and 811, the alignment results in longer code lengths for audio devices 100B and 100C, which mitigates the near-far problem by reducing cross-channel correlation. However, when the receive beamformer (215) is implemented by audio device 100A, the angular separation between audio devices 100B and 100C places microphone signals corresponding to sounds from audio devices 100B and 100C in different lobes, providing further separation of the two received signals, thereby mitigating the near-far problem somewhat. This additional separation therefore allows the orchestration device to reduce the acoustic DSSS spreading code length and obtain observations at a faster rate.

[0211] This does not just apply to acoustic DSSS spreading code lengths: if spatial microphone feeds are used by audio device 100A (and / or audio devices 100B and 100C) instead of omnidirectional microphone feeds, acoustic DSSS parameters that can be changed to mitigate the near-far problem (e.g., even when using FDMA or TDMA) may no longer be necessary.

[0212] Orchestration according to spatial measures (in this case, angular diversity) relies on estimates of these characteristics already being available. In one example, DSSS parameters may be optimized 206 for an omnidirectional microphone feed, and then acoustic DSSS parameters may be optimized for a spatial microphone feed after DoA estimates are available. This is one implementation of the trigger condition described above with reference to FIG. 15.

[0213] FIG. 17 is a block diagram illustrating examples of DSSS signal demodulator elements, baseband processor elements, and DSSS signal generator elements according to some disclosed aspects. As with other figures provided herein, the types and number of elements shown in FIG. 17 are provided merely as examples. Other aspects may include more, fewer, and / or different types and numbers of elements. In other examples, other methods, such as frequency-domain correlation, may be implemented. In this example, the DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212 are implemented by instances of the control system 160 described above with reference to FIG. 1B.

[0214] According to some aspects, there is one instance of DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212 for each acoustic DSSS signal transmitted (played back) from each audio device from which the acoustic DSSS signals are received. In other words, in the aspect shown in Figure 16, audio device 100A implements one instance of DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212 corresponding to the acoustic DSSS signals received from audio device 100B, and one instance of DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212 corresponding to the acoustic DSSS signals received from audio device 100C.

[0215] For purposes of explanation, the following description of Figure 17 continues to use this example of audio device 100A of Figure 16 as the local device implementing instances of DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212. More specifically, the following description of Figure 17 assumes that microphone signals 206 received by DSSS signal demodulator 214 include reproduced sound that includes acoustic DSSS signals produced by a loudspeaker of audio device 100B, and that the instances of DSSS signal demodulator 214, baseband processor 218, and DSSS signal generator 212 shown in Figure 17 correspond to the acoustic DSSS signals reproduced by the loudspeaker of audio device 100B.

[0216] According to this aspect, DSSS signal generator 212 includes an acoustic DSSS carrier module 1715. Acoustic DSSS carrier module 1715 is configured to provide DSSS signal demodulator 214 with a DSSS carrier replica 1705 of the DSSS carrier used by audio device 100B to generate its acoustic DSSS signals. In some alternative aspects, acoustic DSSS carrier module 1715 may be configured to provide DSSS signal demodulator 214 with one or more DSSS carrier parameters used by audio device 100B to generate its acoustic DSSS signals.

[0217] In this aspect, DSSS signal generator 212 also includes an acoustic DSSS spreading code module 1720. Acoustic DSSS spreading code module 1720 is configured to provide DSSS signal demodulator 214 with a DSSS spreading code 1706 used by audio device 100B to generate its acoustic DSSS signals. DSSS spreading code 1706 corresponds to spreading code C(t) in the equations disclosed herein. DSSS spreading code 1706 may be, for example, a pseudorandom number (PRN) sequence.

[0218] According to this aspect, DSSS signal demodulator 214 includes bandpass filter 1703 configured to generate bandpass filtered microphone signals 1704 from received microphone signals 206. In some examples, the passband of bandpass filter 1703 may be centered on the center frequency of the acoustic DSSS signal from audio device 100B being processed by DSSS signal demodulator 214. Bandpass filter 1703 may, for example, pass the main lobe of the acoustic DSSS signal. In some examples, the passband of bandpass filter 1703 may equal the frequency band for transmission of the acoustic DSSS signal from audio device 100B.

[0219] In this example, the DSSS signal demodulator 214 includes a multiplication block 1711A configured to convolve bandpass filtered microphone signals 1704 with DSSS carrier replicas 1705 to generate baseband signals 1700. According to this aspect, the DSSS signal demodulator 214 also includes a multiplication block 1711B configured to apply a DSSS spreading code 1706 to the baseband signals 1700 to generate unspread baseband signals 1701.

[0220] According to this example, DSSS signal demodulator 214 includes accumulator 1710A, and baseband processor 218 includes accumulator 1710B. Accumulators 1710A and 1710B may also be referred to herein as summing elements. Accumulator 1710A operates during what may be referred to herein as a “coherent time” that corresponds to the code length of each acoustic DSSS signal (in this example, the code length of the acoustic DSSS signal currently being played by audio device 100B). In this example, accumulator 1710A performs an “integrate and dump” process. In other words, after summing unspread baseband signals 1701 over the coherent time, accumulator 1710A outputs (“dumps”) demodulated coherent baseband signal 208 to baseband processor 218. In some aspects, demodulated coherent baseband signal 208 may be a single number.

[0221] In this example, the baseband processor 218 includes a square law module 1712. The square law module, in this example, is configured to square the magnitude of the demodulated coherent baseband signal 208 and output a power signal 1722 to the accumulator 1710B. After the magnitude and squaring processing, the power signal can be considered a non-coherent signal. In this example, the accumulator 1710B operates over a "non-coherent time." The non-coherent time, in some examples, may be based on an input from an orchestration device. The non-coherent time, in some examples, may be based on a desired SNR. According to this example, the accumulator 1710B outputs a delayed waveform 400 with multiple delays (also referred to herein as "tau," or instances of tau (τ)).

[0222] Steps 1704 to 208 in Figure 17 can be represented as follows:

number

[0223] In the above equation, Y(tau) represents the coherent demodulator output (208), d[n] represents the bandpass filtered signal (1704 or A in FIG. 17), CA represents the local copy of the spreading code used to modulate the DSSS signal by the distant device in the room (in this example, audio device 100B), and the final term is the carrier signal. In some examples, all of these signal parameters are orchestrated among the audio devices in the audio environment (e.g., may be determined and provided by an orchestration device).

[0224] From Y(tau)(208) in Figure 17<Y(tau)> The signal chain up to (400) is a non-coherent integration, where the coherent demodulator output is squared and averaged. The number of averages (the number of times the non-coherent accumulator 1710B is run) is a parameter that may be determined and provided by an orchestration device in some examples, for example, based on a determination that a sufficient SNR has been achieved. In some examples, an audio device implementing the baseband processor 218 may determine the number of averages, for example, based on a determination that a sufficient SNR has been achieved.

[0225] Incoherent integration can be expressed mathematically as:

number

[0226] The above formula involves simply averaging the squared coherent delayed waveform over a period defined by N, where N represents the number of blocks used for non-coherent integration.

[0227] 18 illustrates elements of a DSSS signal demodulator according to another example. According to this example, DSSS signal demodulator 214 is configured to generate delay estimates, DoA estimates, and audibility estimates. In this example, DSSS signal demodulator 214 is configured to perform coherent demodulation, followed by non-coherent integration on the fully delayed waveform. As with the example described above with reference to FIG. 17, this example assumes that DSSS signal demodulator 214 is implemented by audio device 100A and configured to demodulate acoustic DSSS signals played by audio device 100B.

[0228] In this example, the DSSS signal demodulator 214 includes a bandpass filter 1703 configured to remove unwanted energy from some of the audio content being rendered for the listener's experience and from other audio signals, such as acoustic DSSS signals placed in other frequency bands to avoid near-far problems.

[0229] Matched filter 1811 is configured to calculate delayed waveform 1802 by correlating bandpass filtered signal 1704 with a local replica of the acoustic DSSS signal of interest. In this example, the local replica is an instance of DSSS signal replicas 204 corresponding to the DSSS signals generated by audio device 100B. Matched filter output 1802 is then lowpass filtered by lowpass filter 712 to generate coherently demodulated complex delayed waveform 208. In some alternative aspects, lowpass filter 712 may be located after the squaring operation in baseband processor 218 to generate a non-coherent averaged delayed waveform, such as the example described above with reference to FIG. 17 .

[0230] In this example, channel selector 1813 is configured to control bandpass filter 1703 (e.g., the passband of bandpass filter 1703) and matched filter 1811 according to DSSS information 205. As described above, DSSS information 205 may include parameters used by control system 160 to demodulate DSSS signals, etc. DSSS information 205 may, in some examples, indicate which audio devices are generating the acoustic DSSS signals. In some examples, DSSS information 205 may be received (e.g., via wireless communication) from an external source, such as an orchestration device.

[0231] 19 is a block diagram illustrating an example of baseband processor elements according to some disclosed aspects. As with other figures provided herein, the types and number of elements shown in FIG. 19 are provided by way of example only. Other aspects may include more, fewer, and / or different types and numbers of elements. In this example, baseband processor 218 is implemented by an instance of control system 160 described above with reference to FIG. 1B.

[0232] In this particular embodiment, no coherent techniques are applied. Thus, the first operation performed is to take the power of the complex delayed waveform 208 via square law module 1712 to generate a non-coherent delayed waveform 1922. The non-coherent delayed waveform 1922 is integrated by accumulator 1710B over a period of time (specified in this example by DSSS information 205 received from the orchestration device, but may be determined locally in some examples) to generate a non-coherent averaged delayed waveform 400. According to this example, the delayed waveform 400 is then processed in several ways, as follows:

[0233] 1. The leading edge estimator 1912 is configured to perform a delay estimate 1902, which is an estimated time delay of the received signal. In some examples, the delay estimate 1902 may be based at least in part on an estimate of the position of the leading edge of the delayed waveform 400. According to some such examples, the delay estimate 1902 may be determined according to the number of time samples in the signal portion (e.g., the positive portion) of the delayed waveform from the time sample corresponding to the position of the leading edge of the delayed waveform 400 up to and including the time sample less than one chip period (which is inversely proportional to the signal bandwidth). In the latter case, this delay can be used to compensate for the width of the autocorrelation of the DSSS code. As the chipping rate increases, the width of the autocorrelation peak narrows and is minimized when the chipping rate is equal to the sampling rate. This condition (chipping rate equals sampling rate) results in a delayed waveform 400 that most closely approximates the true impulse response of the audio environment for a given DSSS code. As the chipping rate increases, spectral overlap (aliasing) may occur subsequent to the DSSS signal modulator 220A. In some examples, if the chipping rate is equal to the sampling rate, the DSSS signal modulator 220A may be bypassed or omitted. A chipping rate approaching that of the sampling rate (e.g., a chipping rate that is 80% of the sampling rate, a chipping rate that is 90% of the sampling rate, etc.) may provide a delayed waveform 400 that is a sufficient approximation of the actual impulse response for some purposes. In some such examples, the delay estimate 1902 may be based in part on information about the DSSS signal characteristics. In some examples, the leading edge estimator 1912 may be configured to estimate the position of the leading edge of the delayed waveform 400 according to the first instance of a value greater than a threshold during a certain time window. Some examples are described below with reference to FIG. 20. In other examples, the leading edge estimator 1912 may be configured to estimate the position of the leading edge of the delayed waveform 400 according to the position of a maximum value (e.g., a local maximum within a certain time window), which is an example of "peak picking."Note that many other techniques can be used to estimate the delay (e.g., peak picking).

[0234] 2. In this example, the baseband processor 218 is configured to perform DoA estimation 1903 by windowing the delayed waveform 400 (windowing block 1913) before using the delay-and-sum DoA estimator 1914. The delay-and-sum DoA estimator 1914 may perform DoA estimation based at least in part on determining the steered response power (SRP) of the delayed waveform 400. Accordingly, the delay-and-sum DoA estimator 1914 may also be referred to herein as an SRP module or a delay-and-sum beamformer. Windowing helps isolate a time interval around the leading edge so that the resulting DoA estimate is based more on signal than noise. In some examples, the window size may range from tens of milliseconds to hundreds of milliseconds, e.g., 10 milliseconds to 200 milliseconds. In some examples, the window size may be selected based on knowledge of typical room decay times or knowledge of the decay times of the audio environment in question. In some examples, the window size may be adaptively updated over time. For example, some aspects may include determining a window size such that at least some portion of the window will be occupied by a signal portion of the delayed waveform 400. Some such aspects may include estimating noise power according to time samples occurring before the leading edge. Some such aspects may include selecting a window size such that at least a threshold percentage of the window will be occupied by at least a portion of the delayed waveform corresponding to a threshold signal level, e.g., at least 6 dB greater than the estimated noise power, at least 8 dB greater than the estimated noise power, at least 10 dB greater than the estimated noise power, etc.

[0235] 3. According to this example, the baseband processor 218 is configured to perform the audibility estimate 1904 by estimating signal-to-noise power using an SNR estimation block 1915. In this example, the SNR estimation block 1915 is configured to extract a signal power estimate 402 and a noise power estimate 401 from a delayed waveform 400. According to some such examples, the SNR estimation block 1915 may be configured to determine a signal portion and a noise portion of the delayed waveform 400, as described below with reference to FIG. 20 . In some such examples, the SNR estimation block 1915 may be configured to determine the signal power estimate 402 and the noise power estimate 401 by averaging the signal portion and the noise portion over selected time windows. In some such examples, the SNR estimation block 1915 may be configured to perform the SNR estimation according to the ratio of the signal power estimate 402 to the noise power estimate 401. In some examples, the baseband processor 218 may be configured to perform the audibility estimate 1904 according to the SNR estimate. Given a given amount of noise power, the SNR is proportional to the audibility of an audio device. Thus, in some aspects, the SNR may be used directly as a proxy (e.g., proportional value) for an estimate of the audibility of the actual audio device. Some aspects involving calibrated microphone feeds may involve measuring absolute audibility (e.g., in dBSPL) and converting the SNR to an absolute audibility estimate. In some such aspects, the method for determining the absolute audibility estimate takes into account acoustic losses due to distance between audio devices and variability in noise in the room. In other aspects, other techniques for estimating signal power, noise power, and / or relative audibility from delayed waveforms are used.

[0236] An example of a delayed waveform is shown in Figure 20. In this example, delayed waveform 400 is output by an instance of baseband processor 218. According to this example, the vertical axis represents power and the horizontal axis represents pseudorange in meters. As discussed above, baseband processor 218 is configured to extract delay information, sometimes referred to herein as τ, from the demodulated acoustic DSSS signal. The value of τ can be converted to a pseudorange measurement, sometimes referred to as ρ, as follows:

[0237]

number

[0238] In the above equation, c represents the speed of sound. In Figure 20, the delayed waveform 400 includes a noise portion 2001 (sometimes called the noise floor) and a signal portion 2002. Negative values ​​in the pseudorange measurements (and the corresponding delayed waveform) can be identified as noise. Since negative ranges (distances) have no physical meaning, the power corresponding to negative pseudoranges is assumed to be noise.

[0239] In this example, signal portion 2002 of waveform 400 includes a leading edge 2003 and a trailing edge. Leading edge 2003 is a prominent feature of delayed waveform 400 when the power of signal portion 2002 is relatively strong. In some examples, leading edge estimator 1912 of FIG. 19 may be configured to estimate the position of leading edge 2003 according to the first instance of a power value greater than a threshold during a time window. In some examples, the time window may begin when τ (or ρ) is zero. In some examples, the window size may range from tens of milliseconds to hundreds of milliseconds, e.g., from 10 milliseconds to 200 milliseconds. According to some aspects, the threshold may be a preselected value, e.g., −5 dB, −4 dB, −3 dB, −2 dB, etc. In some alternative examples, the threshold may be based on the power of at least a portion of delayed waveform 400, e.g., the average power of the noise portion.

[0240] However, as discussed above, in other examples, the leading edge estimator 1912 may be configured to estimate the position of the leading edge 2003 according to the position of a maximum value (e.g., a local maximum within a time window). In some examples, the time window may be selected as discussed above.

[0241] 19, in some examples, may be configured to determine an average noise value corresponding to at least a portion of noise portion 2001 and an average or peak signal value corresponding to at least a portion of signal portion 2002. SNR estimation block 1915 of FIG. 19, in some such examples, may be configured to estimate the SNR by dividing the average signal value by the average noise value.

[0242] FIG. 21 shows an example of another embodiment of the block. This embodiment includes a correlator bank implementation of the DSSS signal demodulator 214. In this context, the term "correlator bank" means that multiple instances of the acoustic DSSS signals are correlated at different delays. According to this embodiment, a bulk delay estimator 2110 is used to coarsely align the DSSS correlator bank (214) so ​​that only a subset of all delays need be calculated by the baseband processor 218. In this embodiment, the DSSS correlator bank (214) generates a windowed demodulated coherent baseband signal 208, and the baseband processor 218 generates a windowed non-coherent averaged delayed waveform 400.

[0243] In this embodiment, the bulk delay estimator 2110 utilizes a signal being rendered by a distant device as a reference to estimate the bulk delay. In one such example, the bulk delay estimator 2110 is configured to implement a cross-correlator that correlates a reference signal (2102) being played by another audio device (the "distant device") in the audio environment with the received microphone signals 206 to estimate the bulk delay 2103. In general, the estimated bulk delay 2103 will be different for each audio device receiving the acoustic DSSS signals.

[0244] Some alternatives include estimating the bulk delay 2103 according to the filter taps of an acoustic echo canceller canceling the reference reproduction of the distant device, where the filter exhibits peaks corresponding to the direct signals from the other device, thus providing a rough match.

[0245] The bulk delay estimator 2110 can increase efficiency by limiting subsequent "downstream" calculations, for example by windowing the pseudoranges to a range of x to y meters, e.g., 1 to 4 meters, 0 to 4 meters, 1 to 5 meters, -1 to 4 meters, etc., rather than the range shown in Figure 20.

[0246] 22 shows an example of yet another aspect of the block. This example includes a "matched filter" version of the DSSS signal demodulator 214, which in some cases may be configured as described above with reference to FIG. 18. This example also includes an instance of a bulk delay estimator 2110, which in this aspect provides bulk delay estimates 2103 to the baseband processor 218.

[0247] According to this example, a window is steered (centered) by the external bulk delay estimate 2103 about the signal component of the delayed waveform 2204 extracted using windowing block 1913. An additional windowing block 2213 is centered using the bulk delay estimate 2103 and an offset 2206 to window the delayed waveform 400 in the noise-only region of the delayed waveform. For example, the offset windowed delayed waveform 2205 may correspond to the noise portion 2001 of FIG. 20.

[0248] 19. However, in this example, the baseband processor 218 controls the windowing block 1913 based on the bulk delay estimate 2103. According to this aspect, the windowing block 1913 provides a windowed delayed waveform 2204 to the leading edge estimator 1912, the delay-and-sum beamformer 1914, and the SNR estimation block 1915. Furthermore, in this example, the baseband processor 218 controls the windowing block 2213 based on the bulk delay estimate 2103.

[0249] In some aspects, the delay estimate 1902 estimated using the leading edge estimator 1912 may, in some examples, be used to window subsequent acoustic DSSS observations. In some such aspects, the delay estimate 1902 may replace the bulk delay 2103 in FIGS. 21 and 22.

[0250] FIG. 23 is a block diagram illustrating example audio device elements according to some disclosed aspects. As with other figures provided herein, the types and number of elements shown in FIG. 23 are provided merely as examples. Other aspects may include more, fewer, and / or different types and numbers of elements. In this example, audio device 100A of FIG. 23 is an instance of apparatus 150 described above with reference to FIGS. 1B and 2-4. The aspect shown in FIG. 23 includes all of the elements of FIG. 4, except that beamformer 215A of FIG. 4 is replaced in FIG. 23 by a more generalized pre-processing module 221A. Elements common to FIGS. 4 and 23 will not be described again here, except to the extent that their functionality may differ in the aspect of FIG. 23.

[0251] According to this aspect, preprocessing module 221A is configured to preprocess received microphone signals 206A to generate preprocessed microphone signals 207A. In some aspects, preprocessing the received microphone signals may include applying a bandpass filter and / or noise cancellation. According to some examples, microphone system 111A may include an array of microphones, and in some cases, the microphone array may be or include one or more directional microphones. In some such examples, preprocessing the received microphone signals may include receive-side beamforming via preprocessing module 221A.

[0252] Typically, each audio device has its own internal clock, which often functions independently of the clocks implemented by other audio devices in the audio environment. Clock offset or bias refers to clocks that are off by a specific amount of time (for example, the clock on audio device A and the clock on audio device B). Clocks typically run at slightly different speeds, which is known as clock skew. Clock skew causes the clock bias to change over time. This change in clock bias changes the estimated range or distance between devices, a phenomenon known as "range walk."

[0253] For systems where clock skew is limited by network synchronization and / or where clock skew is estimated (possibly by techniques listed in this disclosure), it may be advantageous to limit the coherent integration time of the receiving device to mitigate SNR loss due to range walk during the integration period. In some examples, this can be combined with range walk compensation techniques, such as when the skew is not large on the coherent integration time scale but is large on the non-coherent integration time scale.

[0254] Figure 24 illustrates blocks of another example embodiment. As with other figures provided herein, the types and number of elements shown in Figure 23 are provided by way of example only. Other embodiments may include more, fewer, and / or different types and numbers of elements. For example, in some embodiments, the baseband processor 218 may include additional elements, such as those described above with reference to Figures 19 and 22.

[0255] In this embodiment, one method of monitoring one of the types of trigger conditions (to trigger an update of acoustic DSSS parameters) mentioned above with reference to FIG. 15 is implemented as a block configured to detect a change in the relative clock skew of any two audio devices in an audio environment. Several detailed examples of calculating the relative clock skew of two audio devices are provided below. In some examples, the enhanced coefficients of the DSSS signal demodulator 214 and the baseband processor 218 may be based at least in part on the relative clock skew. Furthermore, a change in clock skew greater than a threshold amount may, in some examples, result in a change in the global operating configuration (e.g., CDMA, FDMA, TDMA allocation) of all participating audio devices and, in some cases, may be a trigger condition that triggers the flow from block 1510 to block 1515 of FIG. 15.

[0256] 24, DSSS signal generator 212A receives signal skew parameters 2402 and provides DSSS signal replicas 204 corresponding to DSSS signals generated by other audio devices in the audio environment to DSSS signal demodulator 214. In some examples, DSSS signal generator 212A may receive DSSS signal replicas 204 and signal skew parameters 2402 from an orchestration device.

[0257] 24 shows DSSS signal demodulator 214 receiving DSSS signal replicas 204, as well as microphone signals 206 and coherent integration time information 2401. According to this example, square law module 1712 of baseband processor 218 is configured to receive demodulated coherent baseband signals 208 from DSSS signal demodulator 214, generate non-coherent delay waveform 1922, and provide non-coherent delay waveform 1922 to delay walk compensator 2410. According to this example, delay walk compensator 2410 is configured to compensate for the delay walk between the receiving audio device and the audio device currently being processed by baseband processor 218 for the acoustic DSSS signal. In this example, delay walk compensator 2410 is configured to compensate for the delay walk according to received delay rate estimate 2403 and output non-coherent compensated power delay waveform 2405. The term "delay walk" refers to the effect of a non-zero delay rate term, e.g., how much a delayed waveform shifts over a period of time. This is caused by mismatches in the physical clock frequencies of the transmitting and receiving devices. In this example, delay rate estimate 2403 is the rate of change of the estimated delay over time. According to some examples, delay rate estimate 2403 may be determined according to stored instances of delay estimates determined over a period of time (e.g., hours, days, weeks, etc.). If the estimated delay rate is large, when the delayed waveforms are integrated (averaged) non-coherently, the shift in the instantaneous delayed waveforms (e.g., the shift in the demodulated coherent baseband signals 208 in FIG. 24) will blur the final non-coherent averaged signal (e.g., signal 400 in FIG. 24). Considering a -3 dB shift in peak power response due to delay rate induced error (as an example corresponding to a "significant" delay rate effect), a delay rate exceeding the delay rate limit, represented as delay_rate_lim in the following equation, will induce an error worse than -3 dB, where T_code represents the time length of the entire spreading code sequence.

[0258]

number

[0259] According to some examples, the delay walk compensator 2410 can shift the signal (1922) before averaging it using the delay rate estimate 2403. In some such examples, this shift is equal to the amount of delay walk that occurs over the non-coherent integration period, but the shift is applied in the opposite direction to cancel out the delay walk.

[0260] In some alternative aspects, the coherent processing occurring in the DSSS signal demodulator 214 may be altered according to clock bias and / or clock skew information. According to one such example, a clock bias estimate may be used in the DSSS signal generator 212 to shift the phase of the replica signal code (1720) so that the delay in the delayed waveform is due solely to the physical distance between the audio devices. In some examples, clock skew estimates may be used in the DSSS signal generator 212 to shift the replica signal carrier (1715) frequency so that the resulting coherent waveform (208) is free of residual frequency components (i.e., no sinusoids remain). This condition may occur when the replica signal generates a carrier corresponding to the physical signal transmitted by the audio device currently being evaluated / listened to. These carrier frequencies are slightly different due to different clock frequencies.

[0261] Figure 25 shows another example of an audio environment. According to this example, the elements in Figure 25 are as follows:

[0262] 100i,j,k: Distributed audio devices receiving multiple orchestrations; 2500: signal transmitted from audio device i (100i) and received at audio device j (100j); 2501: signal transmitted from audio device i (100i) and received at audio device i (100i); 2502: Signal transmitted from audio device j (100j) and received at audio device i (100i); 2503: Signal transmitted from audio device j (100j) and received at audio device j (100j); 2510: the actual distance between audio device i (100i) and audio device j (100j), and 2511(i,j): The distance between the loudspeaker and microphone of the audio device.

[0263] Some examples of asynchronous two-way ranging will now be described with reference to FIG. 25. In this example, the audio devices are asynchronous and there is a bias between their clocks. In this particular embodiment, two-way ranging is used so that all unknown clock terms are canceled out. This particular example is performed on a pair of audio devices and will be described with reference to audio devices 100i and 100j. A set of ranges between all audio devices in the acoustic space can be obtained by repeating this for all audio device pairs (e.g., audio device pairs 100i-100k and audio device pairs 100j-100k).

[0264] Figure 26 is a timing diagram according to an example. The timing diagram in Figure 26 is used as a reference as part of the process of explaining the asynchronous two-way ranging method. The symbols, acronyms used in this paper and their meanings are as follows:

[0265] [Table 1] [Table 2]

[0266] Additionally, the acronym "DW" refers to a delayed waveform. A hat above a symbol indicates an estimated value. A tilde above a symbol indicates a measured value. An audio device's "clock epoch" is the time when the audio device control system sends a playback signal to the loudspeaker(s). An audio device's "playback epoch" is the time when the loudspeaker(s) actually play the sound corresponding to the playback signal. The terms "latency" and "delay" are used synonymously. For example, "playback latency" is the delay between the time when the audio device control system sends a playback signal to the loudspeaker(s) and the time when the loudspeaker(s) actually play the sound corresponding to the playback signal. Similarly, "recording latency" is the delay between the time when a microphone receives a signal and the time when that signal is received by the control system.

[0267] Figure 26 shows the timing diagram for estimating the playback and recording latency of audio device i. Assuming that the playback and recording input / output (I / O) streams are synchronized, the full-duplex audio thread synchronizes with the audio device clock. When outputting a signal synchronized with JPEG2023552196000011.jpg73, playback latency JPEG2023552196000012.jpg74 JPEG2023552196000013.jpg74 i The signal is not reproduced through the speaker until

[0268]

number

[0269] Then there is the acoustic delay caused by the distance between the speaker and microphone on the audio device. JPEG2023552196000015.jpg74 and then the signal reaches the microphone of the same audio device. The received signal experiences recording latency before entering the audio thread of the audio device. JPEG2023552196000016.jpg74 is further delayed.

[0270]

number

[0271] The DW generated by the audio device is delayed JPEG2023552196000018.jpg72 ii , where ~ indicates the measured value. JPEG2023552196000019.jpg72 ii represents the measured pseudorange between audio device i and itself. The code phase difference between the local replica generated by the audio thread and the signal in the microphone feed is the code delay of the DW peak.

number

[0272] 27 is a timing diagram illustrating clock terms relevant to estimating time of flight between two asynchronous audio devices according to one example, where both audio devices are playing acoustic DSSS signals and one audio device is generating DW by processing the other audio device's acoustic DSSS signal. As a result, the delay measurement JPEG2023552196000021.jpg74 and The resulting image,JPEG2023552196000022.jpg74, corresponds to the ToF between the audio devices.,Figure 27 shows the transmission from device i and,reception at device j, and vice versa.

[0273] In this example, the symbols and acronyms in Figure 27 have the following meanings and contexts:

[0274] JPEG2023552196000023.jpg720 are synchronized with the audio threads on devices i and j, respectively. The actual acoustic delay between the two devices is the same, i.e. JPEG2023552196000024.jpg727, with the respective acoustic paths indicated by green and blue arrows. The code phase of the transmitted signal is determined by the speaker of device i at the time of transmission (ToT). JPEG2023552196000025.jpg715). After ToF, this signal arrives at the receiver (device j), but is delayed by the recording latency on device j. Therefore, the phase of the transmitted signal in the microphone buffer of device j's audio thread is ( JPEG2023552196000026.jpg724). The code phase of the local replica generated by the audio thread running on device j is JPEG2023552196000027.jpg has a phase of 722.

[0275] The code phase difference between the local replica and the received signal determines where the DW peak occurs, so the measured delay can be expressed as:

number

[0276] Doing a similar analysis to find the measured delay when device j is transmitting and i is receiving, we get:

number

[0277] (5) and (6), the relative clock bias term

number

number

[0278] Now, substituting (4) into (8) and rearranging, we get the following:

number

number

[0279] Therefore, (9) can be used to obtain unbiased pseudorange estimates if we have: Mutual delay measurement: JPEG2023552196000034.jpg718 Playback and Recording Latency Measurements: JPEG2023552196000035.jpg718 Estimates of the acoustic delay that make up the playback and recording latency for a particular device: JPEG2023552196000036.jpg718

[0280] In some instances, δ aIn some cases, it is not possible to estimate or eliminate δ in (9). a can be omitted, leaving the bias in the estimated pseudorange:

number

[0281] Or, δ based on the type of audio device a Use an approximation of δ a It is also possible to rely on prior measurement of

[0282] Clock bias estimate Instead of summing any two mutual pseudorange estimates, we take their difference, resulting in:

number

number

number

[0283] Equation (14) allows solving for the relative clock bias (e.g., control system) Δ tij can be solved: 1. The difference between playback and recording latency is known (i.e., measured in advance and substituted into (14)), or 2. The difference between playback and recording latency is equal on both devices (terms in (14) cancel out), or 3. The difference between playback latency and recording latency is zero (terms cancel in (13)).

[0284] Clock Skew Estimation Depending on the signal used to generate the DW, it may also be processed to obtain an estimate of the frequency difference (skew) between the clocks of two audio devices. The DSSS signal used in this experiment is a carrier signal located at f0 Hz, spread with a pseudorandom sequence (sometimes referred to herein as a PRN sequence, PRN code, spreading code, or simply code). Receiving this signal involves both despreading and shifting it back to baseband. However, if the two clocks have different frequencies, after coherent integration (matched filtering with a local replica), there will be a residual frequency equal to the difference between the two clock frequencies. Therefore, instead of averaging the squared results of the coherent integration to generate the DW, some implementations involve performing spectral analysis to determine the frequency of the residual carrier and then inferring the clock frequency difference from the residual carrier frequency. This method allows the control system to obtain an estimate after one coherent integration period. However, unless the DSSS parameters are modified to optimize for such measurements, the estimate is likely to be quite noisy even after only one coherent integration period. Such DSSS parameter changes may include making the spreading code (and coherent integration period) much longer in time (e.g., in the range of hundreds of milliseconds to several seconds), which can be done by using longer codes (more chips) and / or by reducing the chipping rate (bandwidth).

[0285] Another approach takes advantage of the fact that clock frequency differences cause the relative code phase (and clock bias) to walk (i.e., change over time). In some such embodiments, the control system can track how τ changes over time (this is the rate at which the code phase walks).

[0286] There are trade-offs between the two approaches, which can be summarized as follows: Carrier-based techniques require spectral analysis to be performed on each coherent integration result, which entails non-negligible complexity. Code-walk-based techniques require the control system to only maintain a history of measured pseudoranges and process this amount of data, which is significantly smaller. If the clock frequency difference were large enough to be detected on the scale of the coherent integration period, there would likely be an SNR loss in the DW, and the period would need to be shortened, resulting in an inability to resolve the clock rate difference. While carrier-based techniques produce estimates after just one coherent integration period, code-walk-based techniques require a sufficient number of DW and pseudorange estimates to reliably estimate the code walk in the phase noise of the DW. This makes code-walk-based techniques significantly slower. However, coherent carrier-based techniques, which are inherently noisy, may require temporal smoothing, which can result in comparable observation times.

[0287] According to some aspects, a delay rate estimator (e.g., as described above with reference to FIG. 24) may be used to estimate the clock skew. The delay rate is proportional to the clock skew.

[0288] Figure 28 is a graph showing an example of detecting the relative clock skew between two audio devices using a single audio DSSS signal. In this example, the horizontal axis represents frequency and the vertical axis represents power. Figure 28 shows the spectrum of the main lobe of the received modulated audio DSSS signal 2807 and the frequency of the demodulated audio DSSS signal 2808. Note that the demodulated audio DSSS signal 2808 is not at zero Hz, which indicates the relative clock skew between the devices.

[0289] Figure 29 is a graph showing an example of detecting relative clock skew between two audio devices by measuring a single acoustic DSSS signal multiple times. In this example, the horizontal axis represents delay time, and the vertical axis represents power. Figure 29 shows an example of delayed waveforms generated from acoustic DSSS signals of blocks of received audio (t=1 and t=2). A shift in the position of the delayed waveform peaks (which itself indicates bulk delay) indicates clock skew between the devices. In some examples, time 2 may be several hours or even days after time 1. Using such a relatively large time interval is advantageous when the clock skew is relatively small.

[0290] Clock Discipline In some aspects, a control system is configured to utilize clock bias and delay estimation to actually drive (discipline) the local clock using a closed-loop approach. A frequency-locked loop, a delay-locked loop, a phase-locked loop, or a combination thereof can be used to implement the signal processing chain to achieve clock discipline.

[0291] In the alternative, instead of actually adjusting the local clock, the DSSS signal parameters may be adjusted to compensate for the clock bias.

[0292] Because the accuracy of clock bias and delay estimation techniques is highly SNR-dependent, they may be best suited to observations in which the optimization module 712 determines the DSSS parameters 705 by placing a relatively higher weight on the acoustic DSSS signal performance estimate(s) 703 than on the perceptual impact estimate(s) 702 (see FIG. 7 ). For example, the optimization module 712 may be configured to determine the DSSS parameters 705 by placing an emphasis on the system's ability to generate high SNR observations of the acoustic DSSS signals and a de-emphasis on the impact / perceptibility of the acoustic DSSS signals to the user. In some such examples, the DSSS parameters 705 may correspond to audible acoustic DSSS signals.

[0293] However, in some alternatives, coarse techniques (such as DW delay tracking methods) may be implemented in a continuous sub-audible and low SNR manner.

[0294] Device Discoverability 30 is a graph illustrating an example of reserved acoustic DSSS spreading codes for device discovery. In this example, the reserved spreading codes are used, for example, when a new audio device is powered on and is in the process of being configured for use in an audio environment. During runtime operation, different ("normal") acoustic DSSS spreading codes are used. The reserved spreading codes may or may not use the same frequency band as the normal acoustic DSSS spreading codes.

[0295] The elements of Figure 30 are: 3001: Several reserved acoustic DSSS spreading codes, also known as pseudorandom sequences. 3002: Multiple pseudo-random number sequences assigned (by the orchestration device). 3003: Device 1 already has a code assigned. 3006: Device 2 is sending a reservation code (3001). 3004: Device 2 is discovered and the orchestration device assigns a code to device 2. 3007: Device 2 is sending its assigned code. 3008: Device 3 begins transmitting reservation codes after it is powered on for the first time. 3005: Device 3 is discovered and the orchestration device assigns a code to device 3. 3009: Device 3 is transmitting its assigned code.

[0296] In this example, when a new audio device is introduced into the audio environment system, the new audio device begins playing acoustic DSSS signals generated using a reserved spreading code sequence, allowing other devices in the room to identify that a new audio device has been introduced into the acoustic space and initiating an integration sequence. After the new audio device is discovered and integrated into the system of orchestrated audio devices, the new audio device begins playing acoustic DSSS signals using a spreading code (assigned by the orchestrating device in this example).

[0297] According to this example, devices 2 and 3 move from their discovery code channel (frequency band) to a frequency band assigned by the orchestration system. Upon integration, the amplitude, bandwidth, and center frequency of all devices playing acoustic DSSS signals may be changed to optimize the sound for the new system configuration. In some examples, the orchestration device may recalculate the acoustic DSSS parameters of all devices in the acoustic space, so the newly discovered audio device may cause the DSSS parameters of all audio devices to change.

[0298] Noise Estimation In this example, a set of acoustic DSSS-based observations produced by multiple audio devices is used to estimate the noise in the acoustic space.

[0299] Another example of an audio environment is shown in Figure 31. In Figure 31, an acoustic space 130 is shown having multiple distributed orchestrated audio devices 100A, 100B, and 100C participating in DSSS operation. In this example, there is also a noise source 8500 generating noise 8501. The elements of Figure 31 are as follows:

[0300] 130: Acoustic space; 100(A,B,C): Multiple distributed, orchestrated audio devices; 110: multiple loudspeakers; 111: Multiple microphones; 8010: Distance between 100A and 100B; 8011: Distance between 100A and 100C; 8012: Distance between 100B and 100C; 8500: noise source; 8501:noise; 8510: Distance between 8500 and 100A; 8511: the distance between 8500 and 100B; and 8512: Distance between 8500 and 100C.

[0301] Figure 32A shows example delayed waveforms generated by audio device 100C of Figure 31 based on the acoustic DSSS signals received from audio devices 100A and 100B. The delayed waveform corresponding to the acoustic DSSS signals received from audio device 100A is labeled 400Ca, and the delayed waveform corresponding to the acoustic DSSS signals received from audio device 100B is labeled 400Cb.

[0302] Fig. 32B shows an example of delayed waveforms generated by audio device 100B of Fig. 31 based on the acoustic DSSS signals received from audio devices 100A and 100C. The delayed waveform corresponding to the acoustic DSSS signals received from audio device 100A is denoted as 400Ba, and the delayed waveform corresponding to the acoustic DSSS signals received from audio device 100C is denoted as 400Bc.

[0303] The elements in Figures 32A and 32B are as follows: 400Ca: Delayed waveforms generated by device 100C in response to the set of acoustic DSSS signals received from 100A; 400Cb: delayed waveform generated by device 100C in response to the set of acoustic DSSS signals received from 100B; 400Ba: delayed waveforms generated by device 100B in response to the group of acoustic DSSS signals received from 100A; 400Bc: delayed waveforms generated by device 100B in response to the set of acoustic DSSS signals received from 100C; 401C, 401B: noise floor area of ​​delayed waveform group; 8552Ca: Signal power of the delayed waveform generated by 100C in response to the acoustic DSSS signals received from 100A; 8552Cb: Signal power of the delayed waveform generated by 100C in response to the acoustic DSSS signals received from 100B; 8552Ba: Signal power of the delayed waveform generated by 100B in response to the acoustic DSSS signals received from 100A; 8552Bc: Signal power of the delayed waveform generated by 100B in response to the acoustic DSSS signals received from 100C; 8551Ca: Noise power of the delayed waveform generated by 100C in response to the acoustic DSSS signals received from 100A; 8551Cb: Noise power of the delayed waveform generated by 100C in response to the acoustic DSSS signals received from 100B; 8551Ba: Noise power of the delayed waveforms generated by 100B in response to the acoustic DSSS signals received from 100A; and 8551Bc: Noise power of the delayed waveform generated by 100B in response to the acoustic DSSS signals received from 100C.

[0304] 31 , in this example, distance 8511 between audio device 100B and noise source 8500 is less than distance 8512 between audio device 100C and noise source 8500, which is also less than distance 8510 between audio device 100A and noise source 8500. In this particular scenario, the relative proximity of audio device 100B and noise source 8500 causes noise power 8551Ba and 8551Bc in signals 400Ba and 400Bc to be greater than noise power 8551Ca and 8551Cb in signals 400Ca and 400Cb. Furthermore, there is relatively more noise in signal 400Bc than in signal 400Ba. This suggests that noise source 8500 is located closer to the path between audio devices 100B and 100C than to the path between audio devices 100B and 100A. In some aspects, one or more of the audio devices may include directional microphones or may be configured for receive-side beamforming. Such capabilities may provide additional information regarding the DoA of the sound from a noise source, and therefore additional information regarding the location of the noise source.

[0305] Thus, using the known or calculated locations of the audio devices, the known or calculated distances between the audio devices, the measured locations of the noise sources, and the relative noise levels of the delayed waveforms generated by each audio device, in some examples, the control system may be configured to generate a distributional noise estimate for the audio environment 130. Such a distributional noise estimate may be a set of, or may be based on, estimates of noise measured by microphones on audio devices at different locations within the acoustic space. For example, one audio device may be located near a kitchen bench, another near a lounge chair, and another near a door. These devices may each be more sensitive to noise in their immediate vicinity or at different locations within the acoustic space, and collectively generate estimates of the noise distribution throughout the room. Some such aspects may include applying, by the control system, an assumed attenuation function based on the distance between the audio devices and the noise source. Some such examples may include the control system comparing the calculated noise level at each audio device against the measured noise floor of the delayed waveforms and / or against the difference between the measured noise floors of the delayed waveforms (e.g., the difference in level or power between 8551Ca and 8551Cb).

[0306] 33 is a flow diagram outlining another example of the disclosed method. The blocks of method 3300, as with other methods described herein, are not necessarily performed in the order shown. Also, such methods may include more or fewer blocks than shown and / or described. Method 3300 may be performed by an apparatus or system such as apparatus 150 shown in FIG. 1B and described above.

[0307] In this example, block 3305 includes receiving, by the control system, a first content stream including a first group of audio signals. The content stream and the first group of audio signals may vary depending on the particular aspect. In some examples, the content stream may correspond to a television program, a movie, music, a podcast, etc.

[0308] According to this example, block 3310 includes rendering, by the control system, the first group of audio signals to generate first group of audio playback signals, which may be or may include loudspeaker feed signals for a loudspeaker system of an audio device.

[0309] In this example, block 3315 includes generating, by the control system, a first group of direct sequence spread spectrum (DSSS) signals. According to this example, the first group of DSSS signals corresponds to signals referred to herein as acoustic DSSS signals. In some examples, the first group of DSSS signals may be generated by one or more DSSS signal generator modules, such as DSSS signal generator 212A and DSSS signal modulator 220A described above with reference to FIG. 2.

[0310] According to this example, block 3320 includes generating, by the control system, first modified audio playback signals by inserting the first DSSS signal group into the first audio playback signals. In some examples, block 3320 may be performed by DSSS signal injector 211A, described above with reference to FIG. 2.

[0311] In this example, block 3325 includes causing the control system to play the first group of modified audio playback signals to a loudspeaker system to generate a first audio device playback sound. In some examples, block 3320 may include control system 160 of FIG. 2 controlling loudspeaker system 110A to play the first group of modified audio playback signals to generate a first audio device playback sound.

[0312] In some aspects, the method 3300 may include receiving, by the control system, from the microphone system, microphone signals corresponding to at least the first audio device playback sound and the second audio device playback sound. The second audio device playback sound may correspond to second modified audio playback signals played by the second audio device. In some examples, the second modified audio playback signals may include second DSSS signals generated by the second audio device. In some such examples, the method 3300 may include extracting, by the control system, at least the second DSSS signals from the microphone signals.

[0313] According to some aspects, method 3300 may include receiving, by the control system, from the microphone system, microphone signals corresponding to at least the first audio device playback sound and the second through Nth audio device playback sounds. The second through Nth audio device playback sounds may correspond to second through Nth modified audio playback signals played by the second through Nth audio devices. In some examples, the second through Nth modified audio playback signals may include second through Nth DSSS signals. In some such examples, method 3300 may include extracting, by the control system, at least the second through Nth DSSS signals from the microphone signals.

[0314] In some aspects, method 3300 may include estimating, by the control system, at least one acoustic scene metric based at least in part on the second through Nth DSSS signals. In some examples, the acoustic scene metric(s) may be or may include time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, position of audio devices, noise and / or signal-to-noise ratio of an audio environment. According to some examples, method 3300 may include controlling, by the control system, one or more aspects of audio device playback based at least in part on at least one acoustic scene metric and / or at least one audio device characteristic.

[0315] According to some examples, a first content stream component of the first audio device playback may cause perceptual masking of a first DSSS signal component of the first audio device playback, and in some such examples, the first DSSS signal component may be inaudible to humans.

[0316] In some examples, method 3300 may include determining, by the control system, one or more DSSS parameters for each audio device of a plurality of audio devices in the audio environment. The one or more DSSS parameters may be usable for generating DSSS signals. Some such examples may include providing, by the control system, the one or more DSSS parameters to each audio device of the plurality of audio devices.

[0317] In some aspects, determining the one or more DSSS parameters may include scheduling, for each audio device of the plurality of audio devices, a time slot for playing back a set of modified audio playback signals, In some such examples, a first time slot for a first audio device may be different from a second time slot for a second audio device.

[0318] According to some examples, determining the one or more DSSS parameters may include determining, for each audio device of the plurality of audio devices, a frequency band for reproducing the modified audio playback signals, In some such examples, a first frequency band for a first audio device may be different from a second frequency band for a second audio device.

[0319] In some examples, determining the one or more DSSS parameters may include determining a DSSS spreading code for each audio device of the plurality of audio devices. In some such examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device. In some examples, determining the one or more DSSS parameters may include determining at least one spreading code length based at least in part on corresponding audio device audibility. According to some examples, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of the plurality of audio devices in the audio environment.

[0320] In some examples, determining the one or more DSSS parameters may include determining a current playback objective. Some such examples may include applying an acoustic model based at least in part on the interaudibility of each of multiple audio devices in the audio environment to determine estimated performance of DSSS signals in the audio environment. Some such examples may include determining a perceptual impact of DSSS signals in the audio environment by applying a perceptual model based on human sound perception. Some such examples may include determining the one or more DSSS parameters based at least in part on the current playback objective, the estimated performance, and / or the perceptual impact.

[0321] According to some examples, determining the one or more DSSS parameters may include detecting a DSSS parameter change trigger and determining one or more new DSSS parameters corresponding to the DSSS parameter change trigger. Some such examples may include providing the one or more new DSSS parameters to one or more audio devices of the audio environment.

[0322] In some examples, detecting the DSSS parameter change trigger may include detecting one or more of the following: a new audio device in the audio environment, a change in the position of an audio device, a change in the orientation of an audio device, a change in the settings of an audio device, a change in the position of a person in the audio environment, a change in the type of audio content being played in the audio environment, a change in background noise in the audio environment, a change in audio environment configuration including, but not limited to, a change in door or window configuration in the audio environment, a clock skew between two or more audio devices in the audio environment, a clock bias between two or more audio devices in the audio environment, a change in interaudibility between two or more audio devices in the audio environment, and / or a change in playback intent.

[0323] In some examples, method 3300 may include processing the received microphone signals to generate pre-processed microphone signals. Some such examples may include extracting DSSS signals from the pre-processed microphone signals. Processing the received microphone signals may include, for example, applying beamforming, bandpass filtering, and / or echo cancellation.

[0324] According to some aspects, extracting at least the second through Nth DSSS signals from the group of microphone signals may include applying a matched filter to the group of microphone signals or pre-processed versions of the group of microphone signals to generate second through Nth delayed waveforms. The second through Nth delayed waveforms may correspond to, for example, each of the second through Nth DSSS signals. Some such examples may include applying a low-pass filter to each of the second through Nth delayed waveforms.

[0325] In some examples, method 3300 may include implementing a demodulator via the control system. Some such examples may include applying the matched filter as part of a demodulation process performed by the demodulator. In some such examples, an output of the demodulation process may be a demodulated coherent baseband signal. Some examples may include estimating a bulk delay and providing the bulk delay estimate to the demodulator via the control system.

[0326] In some examples, method 3300 may include implementing, via the control system, a baseband processor configured for baseband processing of the demodulated coherent baseband signals. In some such examples, the baseband processor may be configured to output at least one estimated acoustic scene metric. In some examples, the baseband processing may include generating a non-coherently integrated delayed waveform based on demodulated coherent baseband signals received during a non-coherent integration period. In some such examples, generating the non-coherently integrated delayed waveform may include squaring the demodulated coherent baseband signals received during the non-coherent integration period to generate squared demodulated baseband signals and integrating the squared demodulated baseband signals. In some examples, the baseband processing may include applying one or more of a leading edge estimation process, a steered response power estimation process, or a signal-to-noise estimation process to the non-coherently integrated delayed waveform. Some examples may include estimating bulk delay and providing bulk delay estimates to the baseband processor via the control system.

[0327] According to some examples, method 3300 may include estimating, by the control system, second through Nth noise power levels at the locations of the second through Nth audio devices based on the second through Nth delayed waveforms. Some such examples may include generating a distributed noise estimate for the audio environment based at least in part on the second through Nth noise power levels.

[0328] In some examples, the method 3300 may include performing an asynchronous two-way ranging process for cancellation of unknown clock biases between two asynchronous audio devices. The asynchronous two-way ranging process may be based on, for example, DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include performing the asynchronous two-way ranging process between each of a plurality of pairs of audio devices in the audio environment.

[0329] According to some examples, the method 3300 may include performing a clock bias estimation process to determine an estimated clock bias between two asynchronous audio devices. The clock bias estimation process may be based on, for example, DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock bias. Some aspects may include performing the clock bias estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock biases. Some such aspects may include compensating for each estimated clock bias.

[0330] In some examples, the method 3300 may include performing a clock skew estimation process to determine an estimated clock skew between two asynchronous audio devices. The clock skew estimation process may be based on, for example, DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock skew. Some such examples may include performing the clock skew estimation process between each of a plurality of pairs of audio devices in the audio environment to generate a plurality of estimated clock skews. Some such examples may include compensating for each estimated clock skew.

[0331] According to some examples, the method 3300 may include detecting a DSSS signal transmitted by an audio device. In some examples, the DSSS signal may correspond to a first spreading code. Some such examples may include providing the audio device with a second spreading code for subsequent transmission. In some such examples, the first spreading code may include a first pseudo-random number sequence reserved for newly activated audio devices.

[0332] In some examples, the method 3300 may include causing each of a plurality of audio devices in the audio environment to simultaneously play modified audio playback signals.

[0333] In some examples, the acoustic DSSS signals may be played back during one or more time periods during which the audio playback signals are not audible, referred to herein as “silent intervals” or “silence.” In some such examples, at least a portion of the first audio signals may correspond to silence.

[0334] 34 is a flow diagram outlining another example of the disclosed method. The blocks of method 3400, as well as other methods described herein, are not necessarily performed in the order shown. Also, such methods may include more or fewer blocks than shown and / or described. Method 3400 may be performed by an apparatus or system such as apparatus 150 shown in FIG. 1B and described above.

[0335] In some examples, the blocks of method 3400 may be performed by one or more devices in an audio environment, for example, an orchestration device such as an audio system controller (such as what is referred to herein as a smart home hub), or by another component of an audio system, for example, a smart speaker, a television, a television control module, a laptop computer, a mobile device (such as a cell phone), etc. In some aspects, the audio environment may include one or more rooms in a home environment. In other examples, the audio environment may be another type of environment, such as an office environment, an automobile environment, a train environment, a road or sidewalk environment, a park environment, etc. However, in other aspects, at least some blocks of method 3400 may be performed by an apparatus implementing a cloud-based service, such as a server.

[0336] In this example, block 3405 includes causing a first audio device in the audio environment to generate, by a control system, a first set of direct sequence spread spectrum (DSSS) signals. According to this example, the first set of DSSS signals corresponds to signals referred to herein as acoustic DSSS signals. In some examples, the first set of DSSS signals may be generated by one or more DSSS signal generator modules, such as DSSS signal generator 212A and DSSS signal modulator 220A described above with reference to FIG. 2, in accordance with instructions received from an orchestration device. Thus, the control system may be an orchestration device control system. In some examples, the instructions may be received from an orchestration module of an audio device, e.g., from the orchestration module of the first audio device.

[0337] According to this example, block 3410 includes causing the control system to insert the first set of DSSS signals into first set of audio playback signals corresponding to a first content stream to generate first set of modified audio playback signals for the first audio device. In some examples, block 3410 may be performed by DSSS signal injector 211A, described above with reference to FIG. 2, in accordance with instructions received from an orchestration device or module.

[0338] In this example, block 3415 includes causing the control system to cause the first audio device to play the first modified set of audio playback signals to generate a first audio device playback sound. In some examples, block 3415 may include control system 160 of FIG. 2 controlling loudspeaker system 110A (in accordance with instructions received from an orchestration device or orchestration module) to play the first modified set of audio playback signals to generate a first audio device playback sound.

[0339] In some aspects, blocks 3405, 3410, and 3415 may include providing, by an orchestration device or module, DSSS information (such as DSSS information 205A described above with reference to FIG. 2) to the first audio device of the audio environment. As described above, the DSSS information may include parameters used by a control system of the first audio device to generate DSSS signals, modulate DSSS signals, demodulate DSSS signals, etc. The DSSS information may include one or more DSSS spreading code parameters and one or more DSSS carrier parameters, for example, as described elsewhere herein.

[0340] According to this example, block 3420 includes causing a second audio device in the audio environment to generate a second set of DSSS signals, by the control system. In this aspect, block 3425 includes causing the control system to insert the second set of DSSS signals into a second content stream to generate a second set of modified audio playback signals for the second audio device. In this example, block 3430 includes causing the control system to cause the second audio device to play the second set of modified audio playback signals to generate second audio device playback sound. Blocks 3420-3430 may be performed, for example, in accordance with blocks 3405-3415. In some examples, blocks 3420-3430 are performed in parallel with blocks 3405-3415.

[0341] In this example, block 3435 includes causing, by the control system, at least one microphone in the audio environment to detect at least the first audio device playback sound and the second audio device playback sound and generate microphone signals corresponding to at least the first audio device playback sound and the second audio device playback sound, where the at least one microphone may be a component of one or more audio devices in the audio environment, such as the first audio device, the second audio device, another audio device (e.g., the orchestration device), etc.

[0342] According to this example, block 3440 includes causing the control system to extract the first and second DSSS signals from the microphone signals. Block 3440 may be performed by one or more audio devices of the audio environment, for example, having at least one microphone as mentioned in block 3435.

[0343] In this example, block 3445 includes causing the control system to estimate at least one acoustic scene metric based at least in part on the first set of DSSS signals and the second set of DSSS signals, where the at least one acoustic scene metric may include, for example, one or more of time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, location of audio devices, noise of an audio environment, or signal-to-noise ratio.

[0344] In some examples, having the at least one acoustic scene metric estimated may include estimating the at least one acoustic scene metric or having another device estimate the at least one acoustic scene metric, i.e., the acoustic scene metric may be estimated by an orchestration device or another device in the audio environment.

[0345] In some aspects, method 3400 may include controlling one or more aspects of audio device playback based at least in part on the at least one acoustic scene metric. For example, some aspects may include controlling a noise compensation process based at least in part on the one or more acoustic scene metrics. Some examples may include controlling a rendering process and / or one or more audio device playback levels based at least in part on the one or more acoustic scene metrics.

[0346] According to some aspects, the DSSS signal component of the sound reproduced by the audio device may be inaudible to humans. In some examples, a first content stream component of the sound reproduced by the first audio device may cause perceptual masking of the first DSSS signal component of the sound reproduced by the first audio device. In some examples, a second content stream component of the sound reproduced by the second audio device may cause perceptual masking of the second DSSS signal component of the sound reproduced by the second audio device.

[0347] In some examples, method 3400 may include causing third through Nth audio devices of the audio environment to generate third through Nth direct sequence spread spectrum (DSSS) signals. Some such examples may include causing the control system to insert the third through Nth DSSS signals into third through Nth content streams to generate third through Nth modified audio playback signals for the third through Nth audio devices. Some such examples may include causing the control system to cause the third through Nth audio devices to play corresponding instances of the third through Nth modified audio playback signals to generate third through Nth instances of audio device playback sounds.

[0348] In some examples, the method 3400 may include causing each of a plurality of audio devices in the audio environment to simultaneously play modified audio playback signals.

[0349] Some such examples may include causing at least one microphone of each of the first through Nth audio devices to detect first through Nth instances of audio device reproduced sound and generating microphone signals corresponding to the first through Nth instances of audio device reproduced sound. In some such examples, the first through Nth instances of audio device reproduced sound may include the first audio device reproduced sound, the second audio device reproduced sound, and the third through Nth instances of audio device reproduced sound. Some such examples may include causing the control system to extract the first through Nth DSSS signals from the microphone signals, wherein the at least one acoustic scene metric is estimated based at least in part on the first through Nth DSSS signals.

[0350] In some examples, method 3400 may include determining one or more DSSS parameters for a plurality of audio devices in the audio environment. The one or more DSSS parameters may be usable for generating DSSS signals. Some such examples may include providing the one or more DSSS parameters to each audio device of the plurality of audio devices. In some examples, determining the one or more DSSS parameters may include scheduling, for each audio device of the plurality of audio devices, a time slot for playing modified audio playback signals. In some examples, a first time slot for a first audio device may be different from a second time slot for a second audio device.

[0351] According to some examples, determining the one or more DSSS parameters may include determining, for each audio device of the plurality of audio devices, a frequency band for reproducing the modified audio playback signals, where a first frequency band for a first audio device may be different from a second frequency band for a second audio device.

[0352] In some examples, determining the one or more DSSS parameters may include determining a spreading code for each audio device of the plurality of audio devices. In some examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device. In some examples, determining the one or more DSSS parameters may include determining at least one spreading code length based at least in part on a corresponding audio device audibility.

[0353] According to some examples, determining the one or more DSSS parameters may include applying an acoustic model based at least in part on the interaudibility of each of a plurality of audio devices in the audio environment.

[0354] In some examples, determining the one or more DSSS parameters may include determining a current playback objective. Some such examples may include applying an acoustic model based at least in part on the interaudibility of each of multiple audio devices in the audio environment to determine estimated performance of DSSS signals in the audio environment. Some such examples may include determining a perceptual impact of DSSS signals in the audio environment by applying a perceptual model based on human sound perception. Some such examples may include determining the one or more DSSS parameters based at least in part on the current playback objective, the estimated performance, and the perceptual impact.

[0355] According to some examples, determining the one or more DSSS parameters may include detecting a DSSS parameter change trigger. Some such examples may include determining one or more new DSSS parameters corresponding to the DSSS parameter change trigger. Some such examples may include providing the one or more new DSSS parameters to one or more audio devices of the audio environment.

[0356] In some examples, detecting the DSSS parameter change trigger may include detecting one or more of a new audio device in the audio environment, a change in the position of an audio device, a change in the orientation of an audio device, a change in the settings of an audio device, a change in the position of a person in the audio environment, a change in the type of audio content being played in the audio environment, a change in background noise in the audio environment, a change in audio environment configuration including, but not limited to, a change in door or window configuration in the audio environment, a clock skew between two or more audio devices in the audio environment, a clock bias between two or more audio devices in the audio environment, a change in interaudibility between two or more audio devices in the audio environment, and / or a change in playback intent.

[0357] According to some examples, method 3400 may include processing the received microphone signals to generate pre-processed microphone signals. In some such examples, DSSS signals may be extracted from the pre-processed microphone signals. In some such examples, processing the received microphone signals may include one or more of applying beamforming, bandpass filtering, or echo cancellation.

[0358] In some examples, extracting at least the first and second DSSS signals from the microphone signals may include applying a matched filter to the microphone signals or pre-processed versions of the microphone signals to generate delayed waveforms. In some such examples, the delayed waveforms may include at least a first delayed waveform based on the first DSSS signals and a second delayed waveform based on the second DSSS signals. Some examples may include applying a low-pass filter to the delayed waveforms.

[0359] According to some examples, applying the matched filter is part of a demodulation process. In some such examples, the demodulation process may be performed by demodulator 214A described above with reference to FIG. 2, demodulator 214 described above with reference to FIG. 17, or demodulator 214 described above with reference to FIG. 18. According to some such examples, an output of the demodulation process may be a demodulated coherent baseband signal. Some examples may include estimating a bulk delay and providing the bulk delay estimate to the demodulation process.

[0360] Some examples may include performing baseband processing on the demodulated coherent baseband signals, for example, by an instance of baseband processor 218 disclosed herein. In some examples, the baseband processing may output at least one estimated acoustic scene metric. In some examples, the baseband processing may include generating a non-coherently integrated delayed waveform based on demodulated coherent baseband signals received during a non-coherent integration period. According to some such examples, generating the non-coherently integrated delayed waveform may include squaring the demodulated coherent baseband signals received during the non-coherent integration period to generate squared demodulated baseband signals and integrating the squared demodulated baseband signals. According to some aspects, the baseband processing may include applying a leading edge estimation process, a steered response power estimation process, and / or a signal-to-noise estimation process to the non-coherently integrated delayed waveform. Some examples may include estimating a bulk delay and providing the bulk delay estimate to the baseband processing.

[0361] Some examples may include estimating at least a first noise power level at a first audio device position and a second noise power level at a second audio device position. In some such examples, estimating the first noise power level may be based on the first delayed waveform and estimating the second noise power level may be based on the second delayed waveform. Some such examples may include generating a distribution noise estimate for the audio environment based at least in part on the estimated first noise power level and the estimated second noise power level.

[0362] In some examples, the method 3400 may include performing an asynchronous two-way ranging process for cancellation of unknown clock bias between two asynchronous audio devices. In some examples, the asynchronous two-way ranging process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some examples may include performing the asynchronous two-way ranging process between each of a plurality of pairs of audio devices in the audio environment.

[0363] According to some examples, method 3400 may include performing a clock bias estimation process to determine an estimated clock bias between two asynchronous audio devices. In some examples, the clock bias estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock bias. Some aspects may include performing the clock bias estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock biases. Some such examples may include compensating for each estimated clock bias of the plurality of estimated clock biases.

[0364] In some examples, method 3400 may include performing a clock skew estimation process to determine an estimated clock skew between two asynchronous audio devices. The clock skew estimation process may be based on DSSS signals transmitted by each of the two asynchronous audio devices. Some such examples may include compensating for the estimated clock skew. Some examples may include performing the clock skew estimation process between each of a plurality of audio devices in the audio environment to generate a plurality of estimated clock skews. Some such examples may include compensating for each estimated clock skew of the plurality of estimated clock skews.

[0365] According to some examples, method 3400 may include detecting a DSSS signal transmitted by an audio device. In some examples, the DSSS signal may correspond to a first spreading code. Some such examples may include providing a second spreading code to the audio device. In some examples, the first spreading code may be or may include a first pseudo-random number sequence reserved for newly activated audio devices.

[0366] In some examples, the acoustic DSSS signals may be reproduced during one or more time intervals in which the audio playback signals are inaudible, and in some such examples, at least a portion of the first audio playback signals, at least a portion of the second audio playback signals, or at least a portion of each of the first audio playback signals and the second audio playback signals corresponds to silence.

[0367] 35, 36A, and 36B are flow diagrams illustrating example methods for multiple audio devices to inter-coordinate a measurement session, according to some aspects. The blocks illustrated in FIGS. 35-36B, as well as other methods described herein, are not necessarily performed in the order shown. For example, in some aspects, the operation of block 3501 of FIG. 35 may be performed before the operation of block 3500. Furthermore, such methods may include more or fewer blocks than those illustrated and / or described.

[0368] According to these examples, the smart audio device is the orchestration device (sometimes referred to herein as the "leader"), and only one device can be the orchestration device at a time. In other examples, the orchestration device may be what is referred to herein as a smart home hub. The orchestration device may be an instance of apparatus 150 described above with reference to FIG. 1B.

[0369] FIG. 35 illustrates blocks executed by all participating audio devices according to this example. In this example, block 3500 includes obtaining a list of all other participating audio devices. The list of block 3500 can be created, for example, by aggregating information from other audio devices via network packets: other audio devices can, for example, broadcast their willingness to participate in a measurement session. As audio devices are added and / or removed from the audio environment, the list of block 3500 can be updated. In some such examples, the list of block 3500 may be updated according to various heuristics to keep the list up-to-date with only the most important devices (e.g., audio devices currently in the primary living space 130 of FIG. 1A).

[0370] In the example shown in FIG. 35, link 3504 indicates passing the list from block 3500 to block 3501, the leadership negotiation process. This negotiation process in block 3501 can take different forms depending on the particular implementation. In the simplest embodiment, assuming all devices are able to implement the same scheme, an alphanumeric sort on the lowest or highest device ID code (or other unique device identifier) ​​can determine the leader without multiple rounds of communication between devices. In more complex implementations, devices can negotiate with each other to determine which device is best suited to be the leader. For example, it may be advantageous for a device that aggregates organized information to also become the leader for purposes of facilitating measurement sessions. The device with the longest uptime, the most computational power, and / or the device connected to the mains power may be a suitable candidate for leader. In general, coordinating such consensus among multiple devices is a difficult problem, but one with many existing satisfactory protocols and solutions (e.g., the Paxos protocol). It will be appreciated that many such protocols exist and may be suitable.

[0371] According to this example, all participating audio devices then execute block 3503, which means that in this example link 3506 is an unconditional link. Block 3503 is described below with reference to Figure 36B. If a device is the leader, it executes block 3502. In this example, link 3505 includes a leadership check. An example of a leadership process is described below with reference to Figure 36A. Outputs from this leadership process (including, but not limited to, messages to other audio devices) are shown by link 3507 in Figure 35.

[0372] FIG. 36A illustrates an example of a process performed by an orchestration device or leader. Block 3601 includes determining acoustic DSSS parameters for each participating audio device. In some examples, block 3601 may include determining one or more DSSS spreading code parameters and one or more DSSS carrier parameters. In some examples, block 3601 may include determining a spreading code for each participating audio device. According to some such examples, a first spreading code for a first audio device may be different from a second spreading code for a second audio device. In some examples, block 3601 may include determining a spreading code length based at least in part on the audibility of the corresponding audio device. According to some examples, block 3601 may be based at least in part on a current playback objective. In some examples, block 3601 may be based at least in part on whether a DSSS parameter change trigger is detected.

[0373] According to this example, after the orchestration device determines the acoustic DSSS parameters in block 3601, the process of FIG. 36A continues to block 3602. In this example, block 3602 includes sending the acoustic DSSS parameters determined in block 3601 to the other participating audio devices. In some examples, block 3602 may include sending the acoustic DSSS parameters to the other participating audio devices via wireless communication, e.g., a local WiFi network, Bluetooth, etc. In some examples, block 3602 may include sending a “start session” indication, for example, as described below with reference to FIG. 36B. In some examples, the participating audio devices update their acoustic DSSS parameters in block 502.

[0374] According to this example, after block 3602, the process of FIG. 36A continues to block 3603, where the orchestration device waits for the current measurement session to end. In this example, in block 3603, the orchestration device waits for confirmation that all other participating audio devices have ended their sessions. In other examples, block 503 may include waiting a predetermined period of time. In some examples, block 503 may include waiting for a DSSS parameter change trigger to be detected.

[0375] In this example, after block 3603, the process of FIG. 36A continues to block 3600, where the orchestration device is provided with information regarding the measurement session. Such information may influence the selection and timing of future measurement sessions. In some embodiments, block 3600 includes accepting measurements obtained during the measurement session from all other participating audio devices. The type of measurements received may depend on the particular aspect. According to some examples, the received measurements may be or include microphone signals. Alternatively, or additionally, in some examples, the received measurements may be or include audio data extracted from the microphone signals. In some aspects, the orchestration device may perform (or cause to be performed) one or more operations on the received measurements. For example, the orchestration device may estimate (or cause to be estimated) the audibility of the target audio device or the location of the target audio device based at least in part on the extracted audio data. Some aspects may include estimating a far-field audio environment impulse response and / or audio environment noise based at least in part on the extracted audio data.

[0376] In the example shown in Figure 36A, after block 3600 is executed, the process returns to block 3601. In some such examples, the process returns to block 3601 a predetermined period of time after block 3600 is executed. In some examples, the process may return to block 3601 in response to user input. In some examples, the process may return to block 3601 after a DSSS parameter change trigger is detected.

[0377] 36B illustrates an example of processing performed by participating audio devices other than the orchestration device. Here, block 3610 includes each of the other participating audio devices sending a transmission (e.g., a network packet) to the orchestration device indicating each device's willingness to participate in one or more measurement sessions. In some embodiments, block 3610 may also include transmitting results of one or more previous measurement sessions to the leader.

[0378] In this example, block 3615 follows block 3610. According to this example, block 3615 includes waiting for notification that a new measurement session is to begin, for example, as indicated via a "session start" packet.

[0379] According to this example, block 3620 includes applying DSSS parameters according to information provided from the orchestration device, e.g., in conjunction with the “start session” packet awaited at block 3615. In this example, block 3620 includes applying the DSSS parameters to generate modified audio playback signals to be played by the participating audio devices during the measurement session. According to this example, block 3620 includes detecting audio device playback sounds via the microphones of the audio devices and generating corresponding microphone signals during the measurement session. As link 3622 suggests, in some cases, block 3620 may be repeated until all measurement sessions directed by the orchestration device are completed (e.g., pursuant to a “stop” instruction (e.g., a stop packet) received from the orchestration device or after a predetermined duration). In some examples, block 3620 may be repeated for each of multiple target audio devices.

[0380] Finally, block 3625 includes providing information obtained during the measurement session to the orchestration device. In this example, after block 3625, the process of FIG. 36B returns to block 3610. In some such examples, the process returns to block 3610 a predetermined period of time after block 3625 is executed. In some examples, the process may return to block 3610 in response to user input.

[0381] Some aspects of the present disclosure include systems or devices configured (e.g., programmed) to perform one or more examples of the disclosed methods, and tangible computer-readable media (e.g., disks) storing code for implementing one or more examples of the disclosed methods, or steps thereof. For example, some disclosed systems may be or include a programmable general-purpose processor, digital signal processor, or microprocessor programmed and / or otherwise configured by software or firmware to perform any of various operations on data, including embodiments of the disclosed methods, or steps thereof. Such a general-purpose processor may be or include a computer system including an input device, memory, and a processing subsystem programmed (and / or otherwise configured) to perform one or more examples of the disclosed methods (or steps thereof) in response to data asserted thereon.

[0382] Some embodiments may be implemented as a configurable (e.g., programmable) digital signal processor (DSP) configured (e.g., programmed or otherwise configured) to perform necessary processing on audio signal(s), including performing one or more examples of the disclosed methods. Alternatively, embodiments of the disclosed systems (or elements thereof) may be implemented as a general-purpose processor (e.g., a personal computer (PC) or other computer system or microprocessor, which may include input devices and memory) programmed and / or otherwise configured by software or firmware to perform any of various operations, including one or more examples of the disclosed methods. Alternatively, elements of some embodiments of the inventive systems may be implemented as a general-purpose processor or DSP configured (e.g., programmed) to perform one or more examples of the disclosed methods, the system also including other elements (e.g., one or more loudspeakers and / or one or more microphones). A general-purpose processor configured to perform one or more examples of the disclosed methods may be coupled to an input device (e.g., a mouse and / or keyboard), memory, and a display device.

[0383] Another aspect of the present disclosure is a computer-readable medium (e.g., a disk or other tangible storage medium) storing code for performing (e.g., executable code) one or more examples of the disclosed methods or steps thereof.

[0384] While particular embodiments of and applications of the present disclosure have been described herein, it will be apparent to those skilled in the art that many variations on the embodiments and applications described herein are possible without departing from the scope of the disclosure as described and claimed herein. While particular forms of the present disclosure have been shown and described, it is to be understood that the disclosure is not limited to the specific embodiments described and illustrated or to the specific methods described.

Claims

1. causing a first audio device in the audio environment to generate a first set of direct sequence spread spectrum (DSSS) signals by a control system; causing the control system to insert the first set of DSSS signals into a first set of audio playback signals corresponding to a first content stream to generate a first set of modified audio playback signals for the first audio device; causing the first audio device to play the first set of modified audio playback signals by the control system to generate a first audio device playback sound; causing a second audio device in the audio environment to generate a second set of DSSS signals by the control system; causing the control system to insert the second set of DSSS signals into a second content stream to generate second set of modified audio playback signals for the second audio device; causing the second audio device to reproduce the second set of modified audio playback signals by the control system to generate second audio device playback sounds; causing at least one microphone in the audio environment to detect at least the first audio device and the second audio device and generate microphone signals corresponding to at least the first audio device and the second audio device; causing the control system to extract the first and second DSSS signals from the microphone signals; estimating, by the control system, at least one acoustic scene metric based at least in part on the first set of DSSS signals and the second set of DSSS signals; controlling one or more aspects of audio device playback based at least in part on the at least one acoustic scene metric; 13. An audio processing method comprising:

2. 2. The audio processing method of claim 1, wherein the at least one acoustic scene metric comprises one or more of: time of flight, time of arrival, range, audibility of audio devices, impulse response of audio devices, angle between audio devices, position of audio devices, noise or signal to noise ratio of the audio environment.

3. 3. The audio processing method of claim 1, wherein having the at least one acoustic scene metric estimated comprises estimating the at least one acoustic scene metric or having another device estimate the at least one acoustic scene metric.

4. 4. The audio processing method of claim 1, wherein a first content stream component of the first audio device reproduced sound causes perceptual masking of a first DSSS signal component of the first audio device reproduced sound.

5. 5. The audio processing method of claim 1, wherein a second content stream component of the second audio device reproduced sound causes perceptual masking of a second DSSS signal component of the second audio device reproduced sound.

6. 6. The audio processing method according to claim 1, wherein the control system is an orchestration device control system.

7. causing third through Nth audio devices in the audio environment to generate third through Nth direct sequence spread spectrum (DSSS) signals by a control system; causing the control system to insert the third through Nth DSSS signals into the third through Nth content streams to generate third through Nth modified audio playback signals for the third through Nth audio devices; causing the third through Nth audio devices to play corresponding instances of the third through Nth modified audio playback signals by the control system to generate third through Nth instances of audio device playback sounds; 7. The audio processing method of claim 1, further comprising:

8. causing at least one microphone of each of the first through Nth audio devices to detect first through Nth instances of audio device reproduced sounds by the control system and generating microphone signals corresponding to the first through Nth instances of audio device reproduced sounds, the first through Nth instances of audio device reproduced sounds including the first audio device reproduced sound, the second audio device reproduced sound, and the third through Nth instances of audio device reproduced sounds; causing the control system to extract the first through Nth DSSS signals from the set of microphone signals, and the at least one acoustic scene metric is estimated based at least in part on the first through Nth DSSS signals; The audio processing method of claim 7 , further comprising:

9. determining one or more DSSS parameters for a plurality of audio devices in the audio environment, the one or more DSSS parameters being usable to generate a set of DSSS signals; providing the one or more DSSS parameters to each audio device of the plurality of audio devices; 9. The audio processing method according to claim 1, further comprising:

10. 10. The audio processing method of claim 9, wherein determining the one or more DSSS parameters includes scheduling, for each audio device of the plurality of audio devices, a time slot for playing a group of modified audio playback signals, a first time slot for a first audio device being different from a second time slot for a second audio device.

11. 10. The audio processing method of claim 9, wherein determining the one or more DSSS parameters comprises determining, for each audio device of the plurality of audio devices, a frequency band for reproducing a set of modified audio reproduction signals.

12. 12. The audio processing method of claim 11, wherein the first frequency band for the first audio device is different from the second frequency band for the second audio device.

13. 12. The audio processing method of claim 9, wherein determining the one or more DSSS parameters comprises determining a spreading code for each audio device of the plurality of audio devices.

14. 14. The audio processing method of claim 13, wherein the first spreading code for the first audio device is different from the second spreading code for the second audio device.

15. 15. The audio processing method of claim 13 or 14, further comprising determining at least one spreading code length based at least in part on a corresponding audio device audibility.

16. 16. The audio processing method of claim 9, wherein determining the one or more DSSS parameters comprises applying an acoustic model based at least in part on the interaudibility of each of a plurality of audio devices in the audio environment, the interaudibility being a measure of the extent to which acoustic DSSS signals from other audio devices can be detected by a microphone system of each of a plurality of audio devices in the audio environment.

17. Determining the one or more DSSS parameters comprises: Determining the current rehabilitation objectives; applying an acoustic model based at least in part on interaudibility of each of a plurality of audio devices in the audio environment, the interaudibility being a measure of the degree to which acoustic DSSS signals from other audio devices can be detected by a microphone system of each of a plurality of audio devices in the audio environment; determining a perceptual impact of DSSS signals in said audio environment by applying a perceptual model based on human sound perception; determining the one or more DSSS parameters based at least in part on the current playback objective, the estimated performance, and the perceptual impact; 16. An audio processing method according to any one of claims 9 to 15, comprising:

18. Determining the one or more DSSS parameters comprises: Detecting a DSSS parameter change trigger; determining one or more new DSSS parameters corresponding to a DSSS parameter change trigger; providing the one or more new DSSS parameters to one or more audio devices of the audio environment; 18. An audio processing method according to any one of claims 9 to 17, comprising:

19. Apparatus configured to perform the method of any one of claims 1 to 18.

20. 19. A system configured to perform the method of any one of claims 1 to 18.

21. A computer program comprising instructions for controlling one or more devices to carry out a method according to any one of claims 1 to 18.