Methods, apparatus and systems for adaptive calibration process for listener centric acoustic mapping

WO2026187602A1PCT designated stage Publication Date: 2026-09-10DOLBY LABORATORIES LICENSING CORP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/US2026/017034
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2026-02-02
Filing Date
2026-02-27
Publication Date
2026-09-10

Smart Images

  • Figure US2026017034_10092026_PF_FP_ABST
    Figure US2026017034_10092026_PF_FP_ABST
Patent Text Reader

Abstract

Some methods involve obtaining, by a control system, sensor signals and microphone signals obtained during a data-gathering process of obtaining calibration data for calibration of an audio data rendering process, wherein each loudspeaker of the plurality of loudspeakers produces one or more audio calibration signals during the data-gathering process. Some methods involve estimating, by the control system, during the data-gathering process and based on at least one of the sensor signals or the microphone signals, one or more calibration data quality metrics. Some methods involve controlling one or more aspects of the data-gathering process based, at least in part, on the one or more calibration data quality metrics.
Need to check novelty before this filing date? Find Prior Art

Description

D25031W001 METHODS, APPARATUS AND SYSTEMS FOR ADAPTIVE CALIBRATION PROCESS FOR LISTENER CENTRIC ACOUSTIC MAPPINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the priority benefit of U. S. Provisional Application Nos.63 / 767,243 filed March 5, 2025, and 63 / 974,257 filed February 2, 2026, each of which is hereby incorporated by reference in their entireties.TECHNICAL FIELD

[0002] This disclosure pertains to audio processing systems and methods.BACKGROUND

[0003] Audio devices and systems are widely deployed. Although existing systems and methods for loudspeaker mapping and flexible rendering are known, improved systems and methods would be desirable.NOTATION AND NOMENCLATURE

[0004] Throughout this disclosure, including in the claims, the terms “speaker,” “loudspeaker” and “audio reproduction transducer” are used synonymously to denote any sound-emitting transducer (or set of transducers). A typical set of headphones includes two speakers. A speaker may be implemented to include multiple transducers (e.g., a woofer and a tweeter), which may be driven by a single, common speaker feed or by multiple speaker feeds. In some examples, the speaker feed(s) may undergo different processing in different circuitry branches coupled to the different transducers.

[0005] Throughout this disclosure, including in the claims, the expression performing an operation “on” a signal or data (e.g., filtering, scaling, transforming, or applying gain to, the signal or data) is used in a broad sense to denote performing the operation directly on the signal or data, or on a processed version of the signal or data (e.g., on a version of the signal that has undergone preliminary filtering or pre-processing prior to performance of the operation thereon).

[0006] Throughout this disclosure including in the claims, the expression “system” is used in a broad sense to denote a device, system, or subsystem. For example, a subsystem that implements a decoder may be referred to as a decoder system, and a system including such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, inD25031W001 which the subsystem generates M of the inputs and the other X - M inputs are received from an external source) may also be referred to as a decoder system.

[0007] Throughout this disclosure including in the claims, the term “processor” is used in a broad sense to denote a system or device programmable or otherwise configurable (e.g., with software or firmware) to perform operations on data (e.g., audio, or video or other image data). Examples of processors include a field-programmable gate array (or other configurable integrated circuit or chip set), a digital signal processor programmed and / or otherwise configured to perform pipelined processing on audio or other sound data, a programmable general purpose processor or computer, and a programmable microprocessor chip or chip set.

[0008] Throughout this disclosure including in the claims, the term “couples” or “coupled” is used to mean either a direct or indirect connection. Thus, if a first device couples to a second device, that connection may be through a direct connection, or through an indirect connection via other devices and connections.

[0009] As used herein, a “smart device” is an electronic device, generally configured for communication with one or more other devices (or networks) via various wireless protocols such as Bluetooth, Zigbee, near-field communication, Wi-Fi, light fidelity (Li-Fi), 3G, 4G, 5G, etc., that can operate to some extent interactively and / or autonomously. Several notable types of smart devices are smartphones, smart cars, smart thermostats, smart doorbells, smart locks, smart refrigerators, phablets and tablets, smartwatches, smart bands, smart key chains and smart audio devices. The term “smart device” may also refer to a device that exhibits some properties of ubiquitous computing, such as artificial intelligence.

[0010] Herein, we use the expression “smart audio device” to denote a smart device which is either a single-purpose audio device or a multi-purpose audio device (e.g., an audio device that implements at least some aspects of virtual assistant functionality). A singlepurpose audio device is a device (e.g., a television (TV)) including or coupled to at least one microphone (and optionally also including or coupled to at least one speaker and / or at least one camera), and which is designed largely or primarily to achieve a single purpose. For example, although a TV typically can play (and is thought of as being capable of playing) audio from program material, in most instances a modern TV runs some operating system on which applications run locally, including the application of watching television. In this sense, a single-purpose audio device having speaker(s) and microphone(s) is often configured to run a local application and / or service to use the speaker(s) and microphone(s) directly. Some singlepurpose audio devices may be configured to group together to achieve playing of audio over a zone or user configured area.D25031W001

[0011] One common type of multi-purpose audio device is an audio device that implements at least some aspects of virtual assistant functionality, although other aspects of virtual assistant functionality may be implemented by one or more other devices, such as one or more servers with which the multi-purpose audio device is configured for communication. Such a multi-purpose audio device may be referred to herein as a “virtual assistant.” A virtual assistant is a device (e.g., a smart speaker or voice assistant integrated device) including or coupled to at least one microphone (and optionally also including or coupled to at least one speaker and / or at least one camera). In some examples, a virtual assistant may provide an ability to utilize multiple devices (distinct from the virtual assistant) for applications that are in a sense cloud-enabled or otherwise not completely implemented in or on the virtual assistant itself. In other words, at least some aspects of virtual assistant functionality, e.g., speech recognition functionality, may be implemented (at least in part) by one or more servers or other devices with which a virtual assistant may communication via a network, such as the Internet. Virtual assistants may sometimes work together, e.g., in a discrete and conditionally defined way. For example, two or more virtual assistants may work together in the sense that one of them, e.g., the one which is most confident that it has heard a wakeword, responds to the wakeword. The connected virtual assistants may, in some implementations, form a sort of constellation, which may be managed by one main application which may be (or implement) a virtual assistant.

[0012] Herein, “wakeword” is used in a broad sense to denote any sound (e.g., a word uttered by a human, or some other sound), where a smart audio device is configured to awake in response to detection of (“hearing”) the sound (using at least one microphone included in or coupled to the smart audio device, or at least one other microphone). In this context, to “awake” denotes that the device enters a state in which it awaits (in other words, is listening for) a sound command. In some instances, what may be referred to herein as a “wakeword” may include more than one word, e.g., a phrase.

[0013] Herein, the expression “wakeword detector” denotes a device configured (or software that includes instructions for configuring a device) to search continuously for alignment between real-time sound (e.g., speech) features and a trained model. Typically, a wakeword event is triggered whenever it is determined by a wakeword detector that the probability that a wakeword has been detected exceeds a predefined threshold. For example, the threshold may be a predetermined threshold which is tuned to give a reasonable compromise between rates of false acceptance and false rejection. Following a wakeword event, a device might enter a state (which may be referred to as an “awakened” state or a stateD25031W001 of “attentiveness”) in which it listens for a command and passes on a received command to a larger, more computationally-intensive recognizer.

[0014] As used herein, the terms “program stream” and “content stream” refer to a collection of one or more audio signals, and in some instances video signals, at least portions of which are meant to be heard together. Examples include a selection of music, a movie soundtrack, a movie, a television program, the audio portion of a television program, a podcast, a live voice call, a synthesized voice response from a smart assistant, etc. In some instances, the content stream may include multiple versions of at least a portion of the audio signals, e.g., the same dialogue in more than one language. In such instances, only one version of the audio data or portion thereof (e.g., a version corresponding to a single language) is intended to be reproduced at one time.

[0015] As used herein, the term “includes” and its variants are to be read as open-ended terms that mean “includes, but is not limited to.” The term “or” is to be read as “and / or” unless the context clearly indicates otherwise. The term “based on” is to be read as “based at least in part on.” The term “one example implementation” and “an example implementation” are to be read as “at least one example implementation.” The term “another implementation” is to be read as “at least one other implementation.” The terms “determined,” “determines,” or “determining” are to be read as obtaining, receiving, computing, calculating, estimating, predicting, or deriving. In addition, in the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0016] Moreover, as used herein the following acronyms have the following meanings:• dB: Decibel• dBSPL: Sound pressure level in decibel units• DSSS: Direct sequence spread spectrum• DW: Delay waveform• IMU: Inertial Measurement Unit• LCAM: Listener-centric acoustic mapping• SPL: Sound pressure level• SNR: Signal-to-noise ratioSUMMARY

[0017] At least some aspects of the present disclosure may be implemented via one or more audio processing methods. In some instances, the method(s) may be implemented, atD25031W001 least in part, by a control system and / or via instructions (e.g., software) stored on one or more non-transitory media. Some methods may involve obtaining, by a control system, sensor signals and microphone signals during a data-gathering process of obtaining calibration data for calibration of an audio data rendering process. In some examples, each loudspeaker of the plurality of loudspeakers may produce one or more audio calibration signals during the data-gathering process. Some methods may involve estimating, by the control system, during the data-gathering process and based on at least one of the sensor signals or the microphone signals, one or more calibration data quality metrics. Some methods may involve controlling one or more aspects of the data-gathering process based, at least in part, on the one or more calibration data quality metrics.

[0018] In some examples, the sensor signals and microphone signals may be obtained by a handheld sensing device. According to some examples, the handheld sensing device may be pointed in a direction of each loudspeaker of a plurality of loudspeakers during the data-gathering process.

[0019] Controlling the one or more aspects of the data-gathering process may involve causing, by the control system, a user interface to provide a user prompt. The user prompt may be, or may include, a user instruction. The user instruction may be, or may include, an instruction to point the handheld sensing device more accurately at a loudspeaker, an instruction to reduce motion of the handheld sensing device, an instruction to wait for a quieter moment or an instruction to point the handheld sensing device at a different loudspeaker. In some examples, the user prompt may be, or may include, a data-gathering progress indication, a data quality indication, or both.

[0020] According to some examples, controlling the one or more aspects of the data-gathering process may involve altering a level of media played back by one or more loudspeakers of the plurality of loudspeakers. In some examples, the one or more audio calibration signals may be simultaneously emitted by two or more loudspeakers during at least a portion of the data-gathering process. Controlling the one or more aspects of the data-gathering process may involve causing only one loudspeaker of the plurality of loudspeakers to produce the one or more audio calibration signals during at least a portion of the data-gathering process.

[0021] In some examples, the one or more audio calibration signals may be, or include, direct sequence spread spectrum (DSSS) signals. In some such examples, the DSSS signals may use orthogonal spreading codes. According to some examples, each loudspeaker of the plurality of loudspeakers may produce a first DSSS signal for estimating the calibration dataD25031W001 and a second DSSS signal for estimating calibration data quality. In some such examples, the second DSSS signal may have a shorter time duration than the first DSSS signal.

[0022] According to some examples, the calibration data may be, or may include, direction data corresponding to a direction of each loudspeaker of the plurality of loudspeakers relative to an estimated position of a person holding the handheld sensing device. In some such examples, the audio processing method may involve estimating, by the control system and based on the sensor signals, the direction data. In some examples, the calibration data may be, or may include, range data corresponding to a distance from each loudspeaker of the plurality of loudspeakers to the handheld sensing device travelled by each audio calibration signal of the one or more audio calibration signals during the data-gathering process. In some such examples, the audio processing method may involve estimating, by the control system and based on the microphone signals, the range data. Some methods also may involve calibrating, by the control system, the audio data rendering process based, at least in part, on the direction data and the range data.

[0023] In some examples, estimating the one or more calibration data quality metrics may involve estimating a signal-to-noise ratio (SNR) of the microphone signals. According to some examples, estimating the one or more calibration data quality metrics may involve estimating a sensor data quality metric based the sensor signals. In some such examples, the sensor data quality metric may be based on an angular rate estimate, an odometry estimate, or both. In some examples, estimating the one or more calibration data quality metrics may involve estimating at least one microphone signal quality metric, estimating at least one sensor data quality metric, and estimating an overall calibration data quality metric based on the at least one microphone signal quality metric and the at least one sensor data quality metric.

[0024] According to some examples, controlling the one or more aspects of the data-gathering process may involve determining when the one or more calibration data quality metrics indicate that a calibration data quality is at or above a calibration data quality threshold. In some such examples, controlling the one or more aspects of the data-gathering process may involve determining whether the calibration data quality has been at or above a calibration data quality threshold for at least a threshold time interval. In some such examples, the threshold time interval may correspond with an estimated sufficient time for obtaining a sufficient quantity of calibration data for which the calibration data quality has been at or above the calibration data quality threshold for a particular loudspeaker of the plurality of loudspeakers.

[0025] In some examples, the audio data rendering process may involve a flexible rendering process. The flexible rendering process may be, or may include, a center of massD25031W001 amplitude panning process, a flexible virtualization process, a vector-based amplitude panning process, or combinations thereof.

[0026] According to some examples, the sensor signals may be, or may include, magnetometer signals, inertial sensor signals, radio signals, camera signals, or combinations thereof. In some such examples, the one or more calibration data quality metrics include at least one calibration data quality metric based on a combination of two or more types of sensor signals from a group of sensor signals that includes magnetometer signals, inertial sensor signals, radio signals and camera signals.

[0027] At least some aspects of the present disclosure may be implemented via apparatus. For example, one or more devices may be capable of performing, at least in part, the methods disclosed herein. In some implementations, an apparatus is, or includes, an audio processing system having an interface system and a control system. The control system may include one or more general purpose single- or multi-chip processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or combinations thereof.

[0028] Some or all of the operations, functions and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more computer-readable, non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. Accordingly, some innovative aspects of the subject matter described in this disclosure can be implemented via one or more non-transitory media having software stored thereon.

[0029] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Note that the relative dimensions of the following figures may not be drawn to scale.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A shows an example of an audio environment.

[0031] Figure IB is a block diagram that shows examples of components of an apparatus capable of implementing various aspects of this disclosure.D25031W001

[0032] Figure 2A shows examples of an audio DSSS signal and an audio playback signal.

[0033] Figure 2B is a graph that shows examples of the powers of two DSSS signals with different bandwidths but located at the same central frequency.

[0034] Figure 3A shows an example of a delay waveform (DW).

[0035] Figure 3B shows an illustrative autocorrelation of an order 11 Gold Code.

[0036] Figure 3C is a flow diagram that outlines blocks of a disclosed adaptive calibration method according to some examples.

[0037] Figure 3D is a graph showing that the SNR of DSSS signals received by matched filtering has an upper bound according to the order of the Gold code.

[0038] Figure 4 shows examples of timelines for a system with two speakers being calibrated.

[0039] Figure 5 shows alternative examples of timelines for a system with two speakers being calibrated.

[0040] Figure 6 is a flow diagram that outlines another example of a disclosed method.DETAILED DESCRIPTION OF EMBODIMENTS

[0041] Flexible rendering is a playback solution which delivers immersive audio experiences from a constellation of loudspeakers that can be flexibly placed around the room, not necessarily in locations that conform to a canonical surround sound layout such as Dolby 5.1 or 7.1. A larger number of loudspeakers allows for greater immersion, because the spatiality of the media presentation may be leveraged. In some instances, flexible rendering may involve rendering to more than one type of loudspeaker. To configure a flexible Tenderer, a map needs to be created that describes the layout of the loudspeakers and optionally the position of a listener in a playback environment.

[0042] Some previously-deployed acoustic mapping solutions for flexible rendering have focused on the use of microphones within the speakers, such as in the case of smart speakers, to build a map of a loudspeaker constellation instead of being listener-centric. Such solutions are constrained in their scope of support to loudspeakers with in-built microphones, such as smart speakers.

[0043] This disclosure provides listener-centric acoustic mapping solutions for flexible rendering. The methods disclosed herein do not require any sound capturing apparatus within the loudspeakers, thereby expanding the scope of mapping beyond smart speakers to all loudspeakers in general.D25031W001Adaptive Calibration

[0044] Some listener-centric acoustic mapping (LCAM) calibration processes involve a user pointing a sensing device at each of the loudspeakers in a playback environment to be calibrated sequentially. The system (e.g., the sensing device) may prompt the user to point the sensing device at a specific loudspeaker, or may allow the user to point the sensing device at the loudspeakers in any order the user chooses, depending on the particular implementation. Various examples of such user prompts are described below. The control system may prompt the user (e.g., via a user interface system of the sensing device) regarding when to point the sensing device at the next loudspeaker in either case. As used herein, the terms “point ” “pointing,” etc., used in the context of pointing the sensing device at a loudspeaker, may refer to various gestures by the user and / or orientations of the sensing device, but generally refer to the user holding the sensing device in the direction of the loudspeaker, relative to the person’s location, such that the sensing device will be between the person and the loudspeaker. In some cases, pointing the sensing device at a loudspeaker may involve the user extending one or both of the user’s hands that is / are holding the sensing device in the direction of the loudspeaker, extending at least one of the user’s arms, a corresponding hand of which is holding the sensing device, in the direction of the loudspeaker, etc.

[0045] Figure 1A shows an example of an audio environment. As with other figures provided herein, the types and numbers of elements shown in Figure 1 A are merely provided by way of example. Other implementations may include more, fewer and / or different types and numbers of elements. For example, other audio environments 115 may include more than three loudspeakers.

[0046] Figure 1 A shows an example of implementing an LCAM process in an audio environment 115. In this example, a user 105 — also referred to herein as a person or a listener — is holding a mobile device 100 that includes a directi onally-sensitive sensor system 130 and a microphone system 140. The mobile device 100 is an example of what may be referred to herein as a “handheld sensing device.” The mobile device 100 may be cellular telephone, a remote control device, a tablet device, or another type of mobile device. The sensor system 130 may include one or more accelerometers, one or more magnetometers, one or more gyroscopes, one or more accelerometers — which may collectively be referred to an inertial measurement unit (IMU) — one or more cameras, one or more antennas, one or more radios or combinations thereof. In some examples, the sensor system 130 may be configured for light detection and ranging (LiDAR).D25031W001

[0047] According to this example, the user 105 provides input 101 to the mobile device 100 to initiate a calibration process. Calibration signals 120A-C then play out from all loudspeakers 110A-C in the constellation. In some examples, the calibration signals 120A-C may be audio signals that are included with rendered audio content that is played back by the loudspeakers 110A-C. Simultaneously, sound data — including the calibration signals 120A-C — is collected by the microphone system 140. In this example, the user 105 is required to reposition (rotate and / or translate, as indicated by the arrow 102) the mobile device 100 for the collection of azimuth data. According to some examples, the mobile device 100 may provide a prompt — such as an audio prompt and / or a visual prompt — to the user 105 indicating when to point the mobile device 100 to each of the loudspeakers during the calibration process. The user may act in response to the prompt with or without expressly responding to the prompt (e.g., with or without responding “OK” or the like), depending on the particular implementation.

[0048] In some examples, the mobile device 100 may indicate the progress of the calibration process, e.g., by displaying a graphical user interface (GUI) that indicates the estimated time remaining for obtaining calibration signals from a particular loudspeaker, the percentage of the calibration process that has been completed for a particular loudspeaker, etc.

[0049] Collecting the azimuth data may involve obtaining azimuth angles for each of the loudspeakers 110A-C in a loudspeaker plane, in a listener plane, etc. In the example shown in Figure 1 A, the azimuth angle ©A corresponding to the loudspeaker 100A is measured relative to the y axis of a coordinate system having horizontal x and y axes parallel to the floor of the audio environment 115 and having its origin inside of the user 105. In other examples, the azimuth angles may be measured relative to the x axis or to another axis. In some such examples, the mobile device 100 may include a user interface system. A control system of the mobile device 100 may be configured to prompt the user 105, via the user interface system, to position and re-position the mobile device 100 for the collection of azimuth data. In some examples, the vertical angles of loudspeakers may also be measured relative to a z axis of the coordinate system that is perpendicular to the horizontal x and y axes and perpendicular to the floor of the audio environment 115. In some such examples, the azimuth and / or vertical angles may be referred to as “direction data” or the like.

[0050] Various examples of calibration processes are disclosed in detail herein. In some examples, a control system of the mobile device 100 may be configured to perform some or all of the calibration process(es). Alternatively, or additionally, one or more other devices may be configured to perform some or all of the calibration process(es). In some suchD25031W001 examples, one or more servers or one or more other devices, such as a television, a laptop computer or a smart home hub, may be configured to perform some or all of the calibration process(es), based at least in part on data obtained by the mobile device 100.

[0051] Figure IB is a block diagram that shows examples of components of an apparatus capable of implementing various aspects of this disclosure. As with other figures provided herein, the types and numbers of elements shown in Figure IB are merely provided by way of example. Other implementations may include more, fewer and / or different types and numbers of elements. According to some examples, the apparatus 100 may be configured for performing at least some of the methods disclosed herein. In some implementations, the apparatus 100 may be, or may include, one or more components of an audio system. For example, the apparatus 100 may be an audio device, such as a smart audio device, in some implementations. In other examples, the apparatus 100 may be a mobile device (such as a cellular telephone or a remote control), a laptop computer, a tablet device, a television or another type of device. The mobile device 100 of Figure 1A is an instance of the apparatus 100 of Figure IB.

[0052] According to some examples, the audio environment 100 of Figure 1A may include an orchestrating device, such as what may be referred to herein as a smart home hub. The smart home hub (or other orchestrating device) may be an instance of the apparatus 100. In some implementations, one or more of the loudspeakers 110A-110C may be capable of functioning as an orchestrating device.

[0053] According to some alternative implementations the apparatus 100 may be, or may include, a server. In some such examples, the apparatus 100 may be, or may include, an encoder. Accordingly, in some instances the apparatus 100 may be a device that is configured for use within an audio environment, such as a home audio environment, whereas in other instances the apparatus 100 may be a device that is configured for use in “the cloud,” e.g., a server.

[0054] In this example, the apparatus 100 includes a sensor system 130, a microphone system 140 and a control system 150. The microphone system 140 includes one or more microphones. According to some examples, the microphone system 140 includes an array of microphones. The array of microphones may, in some instances, be configured for receiveside beamforming, e.g., according to instructions from the control system 150. In some examples, the array of microphones may be configured to determine direction of arrival (DOA) and / or time of arrival (TOA) information, e.g., according to instructions from the control system 150. Alternatively, or additionally, the control system 150 may be configured toD25031W001 determine direction of arrival (DOA) and / or time of arrival (TOA) information, e.g., according to microphone signals received from the microphone system 140. Alternatively, or additionally, the control system 150 may be configured to determine loudspeaker direction data according to camera data, according to radio data, etc.

[0055] In some implementations, the control system 150 may be configured for performing, at least in part, the methods disclosed herein. The control system 150 may, for example, include a general purpose single- or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or discrete hardware components.

[0056] In some implementations, the control system 150 may reside in more than one device. For example, in some implementations a portion of the control system 150 may reside in a device within one of the environments depicted herein and another portion of the control system 150 may reside in a device that is outside the environment, such as a server, a mobile device (e.g., a smartphone or a tablet computer), etc. In other examples, a portion of the control system 150 may reside in a device within one of the environments depicted herein and another portion of the control system 150 may reside in one or more other devices of the environment. For example, control system functionality may be distributed across multiple smart audio devices of an environment, or may be shared by an orchestrating device (such as what may be referred to herein as a smart home hub) and one or more other devices of the environment. In other examples, a portion of the control system 150 may reside in a device that is implementing a cloud-based service, such as a server, and another portion of the control system 150 may reside in another device that is implementing the cloud-based service, such as another server, a memory device, etc. The interface system 145 also may, in some examples, reside in more than one device.

[0057] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. The one or more non-transitory media may, for example, reside in the optional memory system 155 shown in Figure IB and / or in the control system 150. Accordingly, various innovative aspects of the subject matter described in this disclosure can be implemented in one or more non-transitory media having software stored thereon. The software may, for example, include instructions for controlling at least one device to perform some or all of theD25031W001 methods disclosed herein. The software may, for example, be executable by one or more components of a control system such as the control system 150 of Figure IB.

[0058] The sensor system 130 may include one or more accelerometers, one or more magnetometers, one or more gyroscopes, one or more cameras, one or more radios or combinations thereof. In some implementations, the sensor system 130 may include one or more touch sensors, gesture sensors, motion detectors, etc. According to some implementations, the sensor system 130 may include one or more cameras. In some implementations, the cameras may be free-standing cameras. In some examples, one or more cameras of the sensor system 130 may reside in a smart audio device, which may be a single purpose audio device or a virtual assistant. In some such examples, one or more cameras of the sensor system 130 may reside in a television, a mobile phone or a smart speaker. In some examples, the apparatus 100 may be configured to receive sensor data for one or more sensors residing in one or more other devices in an audio environment via the interface system 145.

[0059] The interface system 145 — when present — may, in some implementations, include a wired or wireless interface that is configured for communication with one or more other devices of an audio environment. The audio environment may, in some examples, be a home audio environment. In other examples, the audio environment may be another type of environment, such as an office environment, an automobile environment, a train environment, a street or sidewalk environment, a park environment, etc. The interface system 145 may, in some implementations, be configured for exchanging control information and associated data with audio devices of the audio environment. The control information and associated data may, in some examples, pertain to one or more software applications that the apparatus 100 is executing.

[0060] The interface system 145 may, in some implementations, be configured for receiving, or for providing, a content stream. The content stream may include audio data. The audio data may include, but may not be limited to, audio signals. In some instances, the audio data may include spatial data, such as channel data and / or spatial metadata. Metadata may, for example, have been provided by what may be referred to herein as an “encoder.” In some examples, the content stream may include video data and audio data corresponding to the video data.

[0061] The interface system 145 may include one or more network interfaces and / or one or more external device interfaces (such as one or more universal serial bus (USB) interfaces). According to some implementations, the interface system 145 may include one or more wireless interfaces, e.g., configured for Wi-Fi or Bluetooth™ communication.D25031W001

[0062] The interface system 145 may, in some examples, include one or more devices for implementing a user interface, such as one or more microphones, one or more speakers, a display system, a touch sensor system and / or a gesture sensor system. In some examples, the interface system 145 may include one or more interfaces between the control system 150 and a memory system, such as the optional memory system 155 shown in Figure IB. However, the control system 150 may include a memory system in some instances. The interface system 145 may, in some implementations, be configured for receiving input from one or more microphones in an environment.

[0063] In some implementations, the apparatus 100 may include the display system 160 shown in Figure IB. The display system 160 may include one or more displays, such as one or more light-emitting diode (LED) displays. In some instances, the display system 160 may include one or more organic light-emitting diode (OLED) displays. In some examples, the display system 160 may include one or more displays of a smart audio device. In other examples, the display system 160 may include a television display, a laptop display, a mobile device display, or another type of display. In some examples wherein the apparatus 100 includes the display system 160, the sensor system 130 may include a touch sensor system and / or a gesture sensor system proximate one or more displays of the display system 160. According to some such implementations, the control system 150 may be configured for controlling the display system 160 to present one or more graphical user interfaces (GUIs). In some examples, a user interface system of the apparatus 100 may include the display system 160, a touch sensor system and / or a gesture sensor system proximate one or more displays of the display system 160, the microphone system 140, the loudspeaker system 110, or combinations thereof.

[0064] According to some implementations, the apparatus 100 may include the optional loudspeaker system 110 shown in Figure IB. The optional loudspeaker system 110 may include one or more loudspeakers, which also may be referred to herein as “speakers” or, more generally, as “audio reproduction transducers.” In some examples (e.g., cloud-based implementations), the apparatus 100 may not include a loudspeaker system 110.

[0065] According to some such examples the apparatus 100 may be, or may include, a smart audio device. In some such implementations the apparatus 100 may be, or may include, a wakeword detector. For example, the apparatus 100 may be, or may include, a virtual assistant.Calibration SignalsD25031W001

[0066] Various types of calibration signals may be used in the methods disclosed in this document. Under sequential playback conditions, any calibration signal with a spectrally-rich autocorrelation in the bandwidth of concern could be used to measure an impulse response (IR)-like signal, such as a DW. Examples include time-stretched-pulse (TSP) calibration signals, maximum length sequence (MLS) calibration signals, sine sweep calibration signals, etc. In some implementations, the calibration signals may be Direct Sequence Spread Spectrum (DSSS) signals. DSSS calibration signals are particularly beneficial for implementations involving the simultaneous playback of calibration signals from multiple loudspeakers of a playback environment.

[0067] DSSS signals have previously been deployed in the context of telecommunications. When DSSS signals are used in the context of telecommunications, DSSS signals are used to spread out the transmitted data over a wider frequency range before it is sent over a channel to a receiver. Most or all of the disclosed implementations, by contrast, do not involve using DSSS signals to modify or transmit data. Instead, such disclosed implementations involve sending DSSS signals between audio devices of an audio environment. What happens to the transmitted DSSS signals between transmission and reception is, in itself, the transmitted information. That is one significant difference between how DSSS signals are used in the context of telecommunications and how DSSS signals are used in the disclosed implementations.

[0068] Moreover, the disclosed implementations involve sending and receiving acoustic DSSS signals, not sending and receiving electromagnetic DSSS signals. In many disclosed implementations, the acoustic DSSS signals are inserted into a content stream that has been rendered for playback, such that the acoustic DSSS signals are included in played-back audio. According to some such implementations, the acoustic DSSS signals may not be audible to humans, so that a person in the audio environment would not perceive the acoustic DSSS signals, but would only detect the played-back audio content. However, in some instances the acoustic DSSS signals may be audible to humans. Such implementations have the potential advantages of improved SNR and potentially shortened capture times.

[0069] Another difference between the use of acoustic DSSS signals as disclosed herein and how DSSS signals are used in the context of telecommunications involves what may be referred to herein as the “near / far problem.” In some instances, the acoustic DSSS signals disclosed herein may be transmitted by, and received by, many audio devices in an audio environment. The acoustic DSSS signals may potentially overlap in time and frequency. Some disclosed implementations rely on how the DSSS spreading codes are generated to separate theD25031W001 acoustic DSSS signals. In some instances, the audio devices may be so close to one another that the signal levels may encroach on the acoustic DSSS signal separation, so it may be difficult to separate the signals. That is one manifestation of the near / far problem, some solutions for which are disclosed herein.

[0070] A DSSS signal typically includes data, a carrier signal and a spreading code. To implement DSSS, in some examples binary phase shift keying (BPSK) modulation may be utilized. If the need to transmit data over the channel is omitted, then one can express the BPSK signal, s(t), as follows:s[n] = Ac[n] cos(c / )0n) (1)

[0071] In the foregoing equation, A represents the amplitude of the DSSS signal, c[n] represents the spreading code, and cos() represents a sinusoidal carrier with a digital frequency of < >o derived from a physical carrier frequency of f0.

[0072] Figure 2A shows examples of an audio DSSS signal and an audio playback signal. Acoustic calibration signals, such as acoustic DSSS signals, may also be referred to herein as audio calibration signals . The curve 230 in Figure 2A corresponds to an example of s[n] in the equation above. The DSSS signal 230 includes a main lobe 235, in which the highest-level portions of the DSSS signal 230 can be found. One of the potential advantages of some disclosed implementations involving acoustic DSSS signals is that by spreading the signal one can reduce the perceivability of the DSSS signal component of audio device playback sound, because the amplitude of the DSSS signal component is reduced relative to a given amount of energy in the audio playback signal.

[0073] This allows us to place the DSSS signal component of audio device playback sound (e.g., as represented by the curve 230 of Figure 2A) at a level sufficiently below the levels of the content stream component of the audio device playback sound (e.g., as represented by the curve 225 of Figure 2A) such that the DSSS signal component is not perceivable to a listener. Some disclosed implementations exploit the masking properties of the human auditory system to optimize the parameters of the DSSS signal in a way that maximizes the signal-to-noise ratio (SNR) of the derived DSSS signal observations and / or reduces the probability of perception of the DSSS signal component. Some disclosed examples involve applying a weight to the levels of the content stream component and / or applying a weight to the levels of the DSSS signal component. Some such examples apply noise compensation methods, wherein the acoustic DSSS signal component is treated as the signal and the content stream component is treated as noise. Some such examples involve applying one or more weights according to (e.g., proportionally to) a play / listen objective metric.D25031W001 DSSS Spreading Codes

[0074] As noted elsewhere herein, in some examples calibration signals may be, or may include, one or more DSSS signals based on DSSS spreading codes. The spreading codes used to spread the carrier wave in order to create the DSSS signal(s) are extremely important. The set of DSSS spreading codes is preferably selected so that the corresponding DSSS signals have the following properties:1. A sharp main lobe in the autocorrelation waveform;2. Low sidelobes at non-zero delays in the autocorrelation waveform;3. Low cross-correlation between any two spreading codes within the set of spreading codes to be used if multiple devices are to access the medium simultaneously (e.g., to simultaneously play back modified audio playback signals that include a DSSS signal component); and4. The DSSS signals are unbiased, (have zero DC component).

[0075] The family of spreading codes (e.g., Gold codes, which are commonly used in the GPS context) typically characterizes the above four points. If multiple audio devices are all playing back modified audio playback signals that include a DSSS signal component simultaneously and each audio device uses a different spreading code (with good crosscorrelation properties, e.g., low cross-correlation), then a receiving audio device should be able to receive and process all of the acoustic DSSS signals from all of the loudspeakers producing the acoustic DSSS signals simultaneously by using a code domain multiple access (CDMA) method. By using a CDMA method, multiple audio devices can send acoustic DSSS signals simultaneously, in some instances using a single frequency band. Spreading codes may be generated during run time and / or generated in advance and stored in a memory, e.g., in a data structure such as a lookup table.

[0076] Choosing a particular set of codes, c(t), that have good auto- and crosscorrelation properties allows for employing a code division multiple access (CDMA) scheme. As noted above, one such set of codes that is widely used is the Gold Code family. CDMA schemes, including those utilizing Gold codes, are widely used for telecommunications, positioning, navigation and timing applications. This results in the simultaneous use of set of signals that can be expressed as follows:In Equation 2, the i subscript indicates the ithsignal.

[0077] In the embodiments disclosed in this document, DSSS refers to a set of signals which are, in general, synthesized using Gold Codes as the spreading code c[n]. While onlyD25031W001 BPSK is described in the examples provided in this document, quadrature phase shift keying (QPSK) can also be used.

[0078] A Gold Code of order K, is a binary sequence of length L = 2K— 1 This is then sampled at a rate fc, the chipping rate of the sequence. This chipping rate specifies how fast each chip of the binary sequence is transmitted. Note, the term chip, not bit, is used for the Gold Code as it must be known by both the transmitter and receiver ahead of time and no data is transmitted in the Gold Code itself. If the audio sampling rate is fsthen (pc= fc / fsis the digital frequency of the code. A typical DSSS calibration signal for the purposes of acoustic calibration may have fc= 6000, f0= 12000, K = 13 which results in a code periodicity of approximately 1.365 seconds or M = 65528 samples.

[0079] Figure 2B is a graph that shows examples of the powers of two DSSS signals with different bandwidths but located at the same central frequency. In these examples, Figure 2B shows the spectra of two DSSS signals 235A and 235B that are both centered on the same center frequency 237. In some examples, the DSSS signal 235A may be produced by one audio device of an audio environment (e.g., by the audio device 100A) and the DSSS signal 235B may be produced by another audio device of the audio environment (e.g., by the audio device 100B).

[0080] According to this example, the DSSS signal 235B is chipped at a higher rate (in other words, a greater number of bits per second are used in the spreading signal) than the DSSS signal 235A, resulting in the bandwidth 240B of the DSSS signal 235B being larger than the bandwidth 240A of the DSSS signal 235A. For a given amount of energy for each DSSS signal, the larger bandwidth of the DSSS signal 235B results in the amplitude and perceivability of the DSSS signal 235B being relatively lower than those of the DSSS signal 235A. A higher-bandwidth DSSS signal also results in higher delay-resolution of the baseband data products, leading to higher-resolution estimates of acoustic scene metrics that are based on the DSSS signal (such as time of flight estimates, time of arrival (To A) estimates, range estimates, direction of arrival (DoA) estimates, etc.). However, a higher-bandwidth DSSS signal also increases the noise-bandwidth of the receiver, thereby reducing the SNR of the extracted acoustic scene metrics. Moreover, if the bandwidth of a DSSS signal is too large, coherence and fading issues associated with the DSSS signal may become present.

[0081] The length of the spreading code used to generate a DSSS signal limits the amount of cross-correlation rejection. For example, a 10 bit Gold code has just -26dB rejection of an adjacent code. This may give rise to an instance of the above-described near / far problem, in which a relatively low-amplitude signal may be obscured by the cross correlation noise ofD25031W001 another louder signal. Some of the novelty of the systems and methods described in this disclosure involves orchestration schemes that are designed to mitigate or avoid such problems.Receiving Calibration Signals

[0082] In the CDMA scheme, receiving individual signals can be done using a matched filter, which is optimal in the sense of maximizing the output signal-to-noise ratio (SNR) of the filter. During a calibration process, a microphone of a hand-held sensing device will produce a microphone signal d[n] containing a mixture of noise and a plurality of calibration signals. If multiple loudspeakers are simultaneously transmitting calibration signals, this microphone signal will contain the calibration signals it hears from multiple speakers after travelling through the playback environment to the microphone. After travelling through the playback environment, the calibration signals are altered (lower amplitude, delayed, etc.). The microphone signal also contains noise signals, etc. Moreover, in addition to noise, the calibration signals may be masked by audio content that is also being played back by the loudspeakers. This audio content is not used for the calibration measurement itself, but to improve the user experience.

[0083] According to some examples, as a modulated signal is received, a complex replica r;[n] of the calibration signal for reception such as the following may be used:r n] = AfC n] cos(o in) + J IKI sin(o,i i) (3)In Equation 3, A represents the amplitude of the DSSS signal, c represents the spreading code, < >0represents the digital frequency of the carrier wave and i represents the index of the code and the index of a loudspeaker that is emitting calibration signals based on that code. A control system may use the complex replica rt[n] of the calibration signal to produce yt, the complex matched filter output for the ithcalibration signalyi M = Sm=o n [T - m] d [m] (4)

[0084] Accordingly, j / j represents the circular convolution of rtand d. In Equation 4, T represents the discrete time sample of the circular convolution signal, AT represents the length of the code in samples and m represents the convolution summation variable. In some implementations, the control system may be configured to implement fast versions of this algorithm that utilize the Fourier transform and exploit the convolution theorem.Delay waveforms

[0085] The matched filter output ytis a complex signal. For the purposes of peak picking, level estimation, and range finding, the delay waveform (DW) may be computed by either taking the magnitude or power of the complex matched filter output, yt. The power delay waveform may be computed as follows:D25031W001 D [T] = ReaU i [T])2+ Imager])2(5)

[0086] Figure 3A shows an example of a delay waveform (DW). In the graph shown in Figure 3A, the horizontal axis indicates samples in a time interval and the vertical axis indicates power in Watts. The direct response 305 of the DW 300 corresponds to the earliest peak that is above a noise threshold (e.g., above 1000 units, above 2000 units, etc.). The noise threshold may vary according to characteristics of the particular playback environment. Often, the direct response 305 peak is also the strongest (highest-power) peak. The direct response 305 corresponds to sound waves traversing a direct acoustic path from a loudspeaker to a listening position (or to a proxy for a listening position such as the microphone of the handheld sensing device 100). Accordingly, the direct response 305 corresponds to the direct acoustic path 215i shown in Figure 2A.

[0087] The indirect response 310 is the remainder of the response. It is comprised of the reflections in the acoustic space. The indirect response 310 can, in some instances, carry on for a significant amount of time (such as hundreds of milliseconds).

[0088] The noise region 315 corresponds with samples obtained prior to time of the direct response 305. Any samples occurring before the direct response 305 do not correspond to a physically possible acoustic path. Accordingly, samples in the noise region 315 are suitable for noise level estimation.

[0089] Unlike methods which utilize the Wiener filter to estimate an impulse response, some methods disclosed in this document use a matched filter j / j [T], which produces an impulse response convolved with the autocorrelation of the calibration code, e.g., as follows:

[0090] yt[T] = h [T] ® R [T] (6)

[0091] In Equation 6, ® represents the circular convolution operator and R [T] represents the autocorrelation of the calibration signal. It may be assumed that there is insignificant carrier frequency mismatch and there is only a need to consider the delay component.

[0092] Figure 3B shows an illustrative autocorrelation of an order 11 Gold Code. In particular, Figure 3B illustrates that there is a significant amount of noise in the autocorrelation waveform at all lags (delays). Practically, this results in an upper bound on the SNR of the output of the matched filter process regardless of the input SNR. This upper bound, T, expressed in Decibels, is a function of the Gold Code order K:

[0094] In Equation 7, the floor() function rounds down to the nearest integer.D25031W001

[0095] Estimating Parameters from the Delay Waveform

[0096] The lag, also referred to as a delay, of the direct response corresponds to the Time of Arrival (ToA) of the calibration signal. Because the sensing device and the playback system are normally not synchronized with one another, the delay of the direct response does not correspond to the Time of Flight (ToF) that is linearly related to the distance between the phone and speaker. In other words, one cannot calculate the time of flight in absolute terms, but only the relative arrivals. The ToA of the ithcalibration signal, Tj, in the DW may be estimated using a peak picking algorithm, the most naive of these being the max peak picker:

[0097] fj = argmax DWi') (8)

[0098] More elaborate peak picking schemes can deal with strong reflections and obstructions in the playback environment.

[0099] The level of the direct response Stis a function of the relative loudness of the speaker, Gt, and the distance between the phone and the speaker, di. The relative loudness corresponds to how loud the speaker is at a fixed distance playing the DSSS signal. In some examples, it is assumed that the speakers in the constellation have roughly the same target curve (the approximate frequency response of a loudspeaker).

[0100] The direct response power level elled as follows:

[0102] In Equation 9, k represents a scaling factor associated with the gain of the calibration device. Here, it may be assumed that the calibration device is itself not calibrated and that an unknown scaling factor k means that the absolute level of any signal in the microphone cannot be measured.

[0103] The direct response level can be estimated by taking the value of the earliest peak in the delay waveform. There are many ways of estimating what the earliest peak is. One such approach is to find the absolute peak and then, within a short window (e.g., in the range of 80 to 120 milliseconds) earlier than this, to look for all peaks which are either 1) significantly above the noise floor (e.g., more than twice the power of the noise floor), or 2) not too far down from the absolute peak level (e.g., above 15% of the argmax, above 20% of the argmax, above 25% of the argmax, etc.). Once the earliest peak is found, the maximum of the earliest peak can be taken as the level estimate St.

[0104] Alternatively, knowledge of the autocorrelation waveform (R [T], the autocorrelation of the calibration signal) may be exploited. The shape of the autocorrelation waveform can be used to interpolate the samples surrounding the peak to refine the estimate of the signal level. This interpolation addresses the fact that the peak value corresponding toD25031W001 the direct response is likely to be located at some delay between samples. In some embodiments, the noise in the autocorrelation signal is ignored. The peak response of the DW may then be modelled as a triangle. Recalling that DW [f is the maximum value of the DW, then the estimate of the direct peak level, Sj, using the triangle signal model can be made as follows:

[0105] St = (DW [f + dL + Mel) / 2 (10)

[0106] In Equation 10, MeZrepresents the next largest sample corresponding to the peak and may be expressed as follows:

[0107] Mel= max (DW[ft- l],DW[ti + 1]). (11)

[0108] In Equation 10, dL represents the level change of the delay waveform between the samples and may be expressed as follows:

[0110] This process comprises the autocorrelation shape based interpolation.[OHl] The noise level, N, can be estimated by processing the samples in the noise region 315 of the DW. As j / j [T] can be modelled as a complex Gaussian process, the noise samples in the power DW will follow a chi-squared distribution. If, instead of taking the power DW, the magnitude DW (MDW) is computed, the MDW may be used synonymously with DW (except where differentiation between them is required):

[0112] MDWi[z] = Real(yi[z]')2+ Imag(yi[z]')2(13)

[0113] Then the noise-region follows a Rician distribution, and a Gaussian approximation can be sufficient in typical operating conditions. One way to estimate the noise level, aN, is to take the standard deviation of the noise only region Tw

[0114] dN= std (MDI / ! / ;[r]) VT G TW(14)

[0115] The user and their actions are inherently noisy and can result in low quality data being collected during the calibration process, causing derived parameters to be erroneous. It is therefore useful to create a data quality metric to ascertain whether calibration data can be used. The data quality metric can include a data quality estimate of the quality of the calibration data gathered. Such estimates can include whether the user kept the sensing device being used for obtaining the calibration data steady during the measurement, whether the user performed the measurement technique(s) in the correct way and / or whether the sensors of the sensing device are operating correctly. Some estimates can be based upon the raw sensor values. Some estimates can be based upon a model of what sensor values mean in the physical world. Such estimates can be derived from sensor data such as a camera, microphone, radio or antenna. Such estimates or data quality metrics may, in someD25031W001 implementations, be simplified to single numbers that a threshold can be compared to. In other cases, the data quality metric may be derived from multiple signal properties and / or multiple types of sensor data that can be compared to expected values. In part to compensate for user-related noise, some disclosed adaptive calibration processes involve estimating the quality of the data being collected to ensure that a sufficient amount of good calibration data for every speaker in the playback system has been collected. Accordingly, such processes may involve making data quality estimates and determining whether, and if so when, calibration data of a sufficiently high quality has been received. Using these data quality estimates the adaptive process may control the amount of time the user dwells (points the sensing device) at each speaker (e.g., via one or more GUIs presented on the sensing device or on another device, via one or more audio prompts, etc.). The control system may also automatically control the parameters, e.g. loudspeaker levels and calibration signal parameters, of the calibration system in response to data quality estimates. The control system may provide feedback to the user that is instantaneous, e.g., how steady the phone is and how strong the signal is, to guide the user in a way to increase the quality of the data. The control system also may provide retrospective feedback about the quality of collected calibration data. In some cases where the quality of calibration data is too low or the process has failed, the control system may cause a user interface system to suggest that the user follow a corrective action before restarting the calibration process.

[0116] Figure 3C is a flow diagram that outlines blocks of a disclosed adaptive calibration method according to some examples. The blocks of method 300, like other methods described herein, are not necessarily performed in the order indicated. Moreover, such methods may include more or fewer blocks than shown and / or described. The method 300 may be performed by an apparatus or system, such as by the apparatus 100 that is shown in Figure IB and described above. More specifically, method 300 may be performed at least in part by the control system 150 of Figure IB.

[0117] In this example, the method 300 involves the following stages:

[0118] 310. The starting conditions, where the first loudspeaker to be measured is set. During or prior to this stage, the user may, for example, have selected an adaptive audio calibration application or “app” on a sensing device. According to this example, sensing device includes an instance of the sensor system 130 and the microphone system 140 of Figure IB, and the sensor system 130 includes an IMU system. During this stage, the user may be prompted (e.g., via a visual prompt on a display and / or via an audio prompt) to point the sensing device at the first loudspeaker. For example, the user may be prompted to holdD25031W001 the sensing device in one hand and to extend the corresponding arm towards the first loudspeaker. In some examples, the user may choose which loudspeaker will be the first loudspeaker.

[0119] 320. The sensing device accumulates sensor data, including IMU data and microphone data. The IMU data may include gyroscope data, magnetometer data, accelerometer data, or combinations thereof. The microphone data may include microphone signals corresponding to audio calibration signals emitted by one or more loudspeakers of a playback environment. In some examples, audio calibration signals may be simultaneously emitted by two or more loudspeakers of a playback environment. In some such examples, all of the loudspeakers to be measured will simultaneously play back audio calibration signals during stage 320. If audio calibration signals are being simultaneously emitted by two or more loudspeakers of the playback environment, the audio calibration signals played back by each loudspeaker should be differentiable from the audio calibration signals played back by every other loudspeaker. In some such examples, the audio calibration signals played back by each loudspeaker at any one time may correspond to a different DSSS code. During this stage, the user may be prompted (e.g., via a visual prompt on a display and / or via an audio prompt) to hold the sensing device steady and to continue pointing the sensing device in the direction of the first loudspeaker to be measured.

[0120] 330. After some period of time, the sensor data is processed by a control system, which may be the control system of the sensing device or the control system of another device. During this stage, estimated parameters for loudspeaker calibration may be extracted, along with calibration data quality metrics. Some examples of estimating calibration data quality metrics are described below. During process 330, the control system may estimate the range or distance from the sensing device to each loudspeaker that was, or is, emitting audio calibration signals. In some examples, determining the range data may involve determining a time of arrival of each audio calibration signal of the one or more audio calibration signals emitted by each loudspeaker of the plurality of loudspeakers and received by the sensing device, determining a level of each audio calibration signal of one or more audio calibration signals emitted by each loudspeaker of the plurality of loudspeakers and received by the sensing device, or both. The IMU data obtained while the user is or was pointing at the first loudspeaker may provide direction data, such as azimuth data, attitude data, etc., corresponding to the direction of the first loudspeaker relative to the sensing device, which may be used as a proxy for the direction of the first loudspeaker relative to the user’s position. In some examples, the direction of the first loudspeaker relative to the user’sD25031W001 position may be estimated according to camera data, radio data, or combinations thereof. For example the user "pointing" at the speaker can be ascertained via the camera; The analysis of range and direction can be cross checked with wireless TOA and DOA methods (e.g. range finding via Bluetooth radio). Some examples involve combining one or more such methods.

[0121] 340. If the data quality metric indicates “good” calibration data — e.g., calibration data having a calibration data quality metric that is at or above a threshold — has been accumulated, the control system may cause the user to be informed via some feedback mechanism such as a progress bar displayed on a display, e.g., a display of the sensing device. In some implementations, the control system may increment a counter which counts the number of good calibration data measurements made for the current loudspeaker. In some such examples, the number of good calibration data measurements may be indicated to the user, e.g., on a displayed graphical user interface (GUI). A single calibration data measurement may, for example, correspond to the length of (the time interval for) a calibration data sequence that is periodically played back by loudspeakers of a playback environment and / or may correspond to a time interval of a delay waveform corresponding to the calibration data sequence. Some examples are described below with reference to Figures 4 and 5.

[0122] 350. In this example, stage 350 involves determining whether a sufficient amount of calibration data of a sufficiently high quality has been obtained. If a sufficient amount of good calibration data has not yet been accumulated, the control system may return to accumulating calibration data for the current device (stage 320). The user may continue to be instructed to point at the current loudspeaker.

[0123] 360. Otherwise, if the control system determines that a sufficient number of good calibration data measurements has been collected for the current speaker being measured, then the control system may increment the index of the speaker to be measured and method 300 may proceed to stage 370.

[0124] 370. In this example, stage 370 involves determining whether all of the loudspeakers have been measured. If the control system determines that all loudspeakers have been measured, then the calibration process is over (block 380). If the control system determines that all of the loudspeakers have not been measured, the process reverts to stage 320 and another loudspeaker will be measured. The control system may cause a user interface system to provide a prompt to the user to point the sensing device at the next loudspeaker and the process 300 may continue until the control system determines that all loudspeakers have been measured.D25031WQ01

[0125] 380. This stage may involve, for example, storing in a memory the calibration data obtained during method 300. In some examples, stage 380 may involve calibrating a flexible rendering process based, at least in part, on the calibration data obtained during method 300.

[0126] According to some examples, method 300 may involve counting the number of times (and / or the amount of time) “bad” calibration data, e.g., calibration data that is below a calibration data quality threshold, has been collected for a loudspeaker. In some such examples, if the collection of bad calibration data has exceeded some threshold, the control system may cause the user to be informed (e.g., via an audio or visual prompt) that an issue is present. If the data quality metrics are informative regarding the source of the underlying problem(s), then the control system may cause the user to be prompted to resolve some particular issue, e.g., to avoid or attenuate noise sources, or to hold the sensing device more still. Moreover, in some examples the calibration data quality metrics may also indicate that the sensing device is unable to perform the calibration process.Data Quality Estimation

[0127] During and / or after method 300 or a similar process, the microphone and IMU sensor data signals obtained from the sensing device are processed by a control system in order to extract calibration data for audio rendering purposes. This processing may, for example, involve estimating an impulse response, or computing a cross-correlation of the microphone feed using a copy of the calibration signal emitted by the speakers. Additional metrics, which are indicative of the quality of the microphone data, may be extracted either from the raw microphone signal or from derived products of the raw microphone signal (e.g., the estimated impulse response and / or cross-correlation results). Similarly, for the IMU, in addition to extracting parameters for calibrating an audio Tenderer, the raw IMU signals or derived data products (e.g., the attitude of the sensing device for the purposes of data quality estimation) may be evaluated during this process. Some examples of data quality estimation based on microphone data and IMU data are described below. Alternatively, or additionally, data quality estimation may be based on camera data and / or radio data. For example, the accuracy of the user’s "pointing" at a loudspeaker may be ascertained via camera data. The analysis of range and direction may be cross-checked with wireless ToA and / or DoA methods. For example, Bluetooth radio supports range finding.Using microphone signals

[0128] The data quality metric(s) extracted from the microphone signal may be related to the presence and / or strength of the calibration signal emitted by the speaker in theD25031W001 microphone signal. If the speakers are not simultaneously emitting calibration tones and other interfering / masking signals, then the impulse response can be estimated using standard techniques such as Wiener filters. If the speakers are simultaneously emitting calibration tones, or if there are other interfering / masking signals being played back by the loudspeakers, then the cross-correlation of the calibration signal with the microphone feed produces a delay waveform.

[0129] In some examples, a simple signal -to-noise (SNR) ratio may be used to indicate microphone data quality. The SNR may be determined as follows:SNR = (15)Nv 7

[0130] In Equation 15, S represents the signal level estimate and N represents the noise level estimate. The mapping of SNR to a data quality metric threshold is dependent on what data is being extracted and the signal from which it is being extracted (e.g., impulse response or delay waveform). For example, if some equalizer response is being extracted from an impulse response, then the sufficient or threshold SNR to do so would be much higher than the SNR threshold required for simply extracting the time of arrival from a delay waveform. The former may require 60 dB while the latter may only require 15dB. We may define a data quality metric QSNR based on the SNR as follows:QSNR = f(SNR) (16)

[0131] In some examples, f(SNR) may map the SNR onto a range of 0 to 1. In some instances the mapping of SNR into a range may be a linear mapping. In some examples, for the delay waveform case, f(SNR) may be expressed as follows:

[0132] A suitable threshold for QSNR , which may also be referred to herein as a “calibration data quality threshold,” may, for example, be 0.75 for time of arrival (ToA) estimates in a delay waveform (DW). It will be appreciated that 0.75 is merely one suitable calibration data quality threshold for ToA estimates in a DW, and that the calibration data quality threshold for other workable examples may be higher or lower, e.g., in a range from 0.70 to 0.80. Any monotonic function may be used for f(SNR) that the designer deems suitable. In some examples, an indication of QSNR may be provided to the user in the form of feedback in the user interface, e.g., via a signal quality graph or a virtual signal quality dial in a GUI, via audio prompts indicating whether the current calibration data quality is above or below a calibration data quality threshold, etc.Long and short DSSS calibration signalsD25031W001

[0133] If simultaneous playback of the calibration signals by all loudspeakers is occurring, then the loudspeakers must be emitting audio calibration signals which are suitable for multiple access, e.g., audio calibration signals that are distinguishable from one another. Such a scheme could be a Code division multiple access, utilizing the Gold code family, and binary phase shift keying which is a direct sequence spread spectrum (DSSS) technique. In some examples, audio calibration signals that are generated according to these sequences may be played back periodically for the entire duration of the calibration process, on loop.

[0134] The SNR, and thus quality of the data extracted from the delay waveform, increases as the length of the DSSS signal increases due to the coherent gain associated with the matched filtering process used to receive them. Here, we have a design trade-off. If the DSSS signal is long, then we can expect higher SNRs but a longer time to wait to determine if the data quality is sufficient. This means that relatively longer DSSS signals will produce a delay in adapting the calibration process and providing feedback to the user. According to some examples, the speakers may emit two audio calibration signals, one for the purposes of estimating parameters for calibration of the Tenderer, and another for estimating the quality of those parameter estimates. The latter signal may be a shorter signal (a “pilot signal”) and thus received at faster rates in order to update the user and state machine at faster rates. The difference in signal length, and thus the coherent gain associated with matched filtering, can be used in combination with the SNR estimate of the pilot signal to estimate the SNR of the calibration signal at the faster rate.

[0135] Figure 3D is a graph showing that the SNR of DSSS signals received by matched filtering has an upper bound according to the order of the Gold code. If the SNR estimate is made by picking the peak power levels in the signal region and noise-only regions of the delay waveform, then it will asymptotically approach this upper bound as the input SNR increases. Simulation results are shown in Figure 3, which shows the output SNR 385 as a function of the input SNR 390 for a DSSS signal constructed using a Gold Code of order 13. One may observe that in this example, as the input SNR 390 approaches 60 dB, the output SNR 385 approaches the theoretical SNR limit 395. Some disclosed implementations may use this response, or more specifically the inverse, to produce and alternate definition of f(SNR) based on the properties of the calibration signal and the reception method (matched filtering).

[0136] For example, in figure 3D the input SNR 390 is linearly increased, resulting in a nonlinear received SNR response 385. This relationship may be written as:SNR = g(SNRj) (17.1)D25031W001

[0137] In equation 17, SNRtrepresents the input SNR 390, and $(•) is a function defining the relationship of input SNR 390 to received SNR 385 depending on the properties of the calibration signal and the reception method. In some examples, it may be desirable for f(SNR) to correspond or be proportional to SNR which can be achieved by using the inverse of $(•):f(SNR = a ■ g-^SNR (17.2)

[0138] In equation 17.2, a represents the constant of proportionality of f(SNR) to SNRi. The result is a data quality metric, such as defined in equation 16, that increases linearly with input SNR 390. Other examples may use techniques to further clip this metric to the range from 0 to 1, such as in equation 17, in conjunction with choosing the appropriate constant of proportionality a for such purpose.Using IMU signals

[0139] The IMU signals are, in some examples, a combination of accelerometer, gyroscope and magnetometer data. In some examples, these IMU signals may be fused using techniques such as Kalman filtering to produce attitude estimates (the orientation of the sensing device) and / or odometry estimates, which are position changes of the sensing device from a particular epoch.

[0140] The attitude data contains the pointing vector of the sensing device (the direction in which it is pointing) which may be used to derive calibration data for rendering purposes. The pointing vector may be defined relative to the orientation of the calibration device at the time the first loudspeaker is measured. The part of the calibration device pointing away from the user and toward the first loudspeaker during measurement should also be pointing away from the user and toward every other loudspeaker that is measured, such that the "pointing vector" points in the direction of each loudspeaker during its measurement. One example of the pointing vector may be defined according to what is referred to as the “boresight,” which refers to how the IMU of a calibration device is oriented relative to a camera of the calibration device. The camera may be the part of the calibration device that is pointing away from the user and towards a loudspeaker.

[0141] The angular rate of this pointing vector, co, can be used as a data quality metric as it is generally not desirable to accumulate calibration data whilst the user is moving the sensing device. Similarly, the odometry estimates can be used as a data quality metric as odometry estimates are indicative of the sensing device being moved.

[0142] We may define a calibration data quality metricbased on the angular rate (co) signals as follows:D25031W001 Qa> = f (18)

[0143] In some embodiments, we may define (m) as follows:

[0144] In Equation 19,represents a parameter related to the angular rate co and T^o represents a threshold value for the angular rate co. In some examples, TM0may be 1 degree per second, or 0.017 radians per second,may be 10 degrees per second, or 0.17 radians per second, meaning that the control system will tolerate attitude data to be changing at a rate of 1 degree per second due to noise, but if the attitude data is changing at a rate greater than 10 degrees per second, the control system will determine that the attitude data is not usable. Other examples may implement different values forand / or T^. In one such example,may be 5 degrees. This type of embodiment takes into consideration that the attitude estimate may be noisy, and any angular rates less than the threshold T^Q may be negligible. is a parameter chosen to provide a range where there is some dynamic range in the output of (m) which may be more suited for output to a user. If we set T^Q=then in some examples / (m) simply becomes a binary function which may be more suited for use in thresholding data accumulation. In other words, the middle termwould not be Toimplemented in such examples.

[0145] Some examples may involve similar definitions for Qv, a data quality metric based on odometry (v) data that may be derived from IMU data, e.g., as follows:Qv = f ) (20)

[0146] In Equation 20, (v) is a data quality function based on the odometry data. In such examples, odometry measurements may be based on the position of the calibration device to see what movement of the calibration device there has been. Some such example may involve determining the average of the calibration device’s position over a previous time interval T, then determining the calibration device’s current position and determining the difference between the calibration device’s current position and subtracting that from the calibration device’s average position. In some such examples, the ratio between this difference and a specified threshold must be below 1 for the calibration data to be considered “good.”Using Advanced Spatial signals

[0147] In addition to the IMU, many sensing devices such as mobile phones incorporate light detection and ranging (LiDAR) sensors, radio / antenna-based sensors and / orD25031W001 cameras. The data from these sensors, in addition to the IMU signals, may be fused to provide more accurate attitude and position estimates. Moreover, processing the raw camera and / or LiDAR data can provide visual odometry data that can be used similarly to how the attitude and position rate data is used. Some implementations may include camera systems that incorporate object recognition functionality, which may include the ability to look for known objects and to ascertain whether the user is pointing at the speaker that is being measured. Examples of object recognition systems include Haar cascades and deep machine learning based systems. Information from these object recognition systems may be combined with the other quality metrics to create a cascade of quality metrics where the quality of the audio measurement and / or IMU measurement is augmented. The loudspeakers under test may, in some instances, provide visual cues to the object detection system, e.g., via coded optical signals, such as optical signals from light-emitting diodes (LEDs), to indicate which loudspeaker is which. Other implementations may augment the audio and IMU measurements with the use of radio signals to create a further cascade in quality metrics. For example, a radio ToA may be compared with the audio ToA to ensure they are within a tolerance of, e.g., a time threshold. In some examples, radio DoA measurements may be used to ensure that the IMU-based angle measurements are consistent.Data Quality

[0148] The overall calibration data quality can be described by combining the data quality of the microphone and the IMU signals. In some embodiments, a control system may be configured to determine an overall calibration data quality factor Q non-linearly, for example as follows:Q = min QSNR, Q^, Qv) (21)

[0149] In other implementations, a control system may be configured to weight the individual quality metrics used to determine an overall calibration data quality factor Q. In some such examples, the individual quality metrics may be weighted linearly, for example as follows:Q = ^SNRQSNR + K Qa, + Qv (22)

[0150] In Equation 22, ASNRrepresents a weighting factor for QSNR,represents a weighting factor for Q^, Avrepresents a weighting factor for Qv, and ASNR++ Av= 1. There are trade-offs to be made when choosing how to combine all the individual data quality metrics. For example, by taking the minimum in Equation 21, the control system is being the most conservative with data accumulation. However, this example of an overall calibration data quality metric may not be suitable for the purposes of providing feedback to the userD25031W001 because it is non-linear and therefore the user may not be able to respond to such feedback intuitively.

[0151] In other implementations, hard limits may be applied to a given quality metric, whether individual or combined, to ensure no metric is excessive, for example as follows:Q = max (L1OW, min(Lhigh, Q)) (23) In equation 23, L[owrepresents a lower limit on the quality metric, and Lhighrepresents an upper limit on the quality metric.Data Accumulation

[0152] Figure 4 shows examples of timelines for a system with two speakers being calibrated. According to this example, graph 410 and timelines 420, 430 and 440 are time-aligned. For example, the time corresponding to the dashed line 405 — which will be referred to herein as time 405 — indicates the same time on graph 410 and timelines 420-440.According to this example, microphone data corresponding to audio calibration signals are kept or discarded in contiguous blocks (such as blocks 425a and 425b) corresponding to the length of the calibration sequence that is periodically played back by the loudspeakers during a calibration process. Here, the time is divided into discrete periods corresponding to the blocks 425, each of which has a time interval corresponding to a DSSS period of a calibration data sequence. In this example, the blocks 425 all have a time interval that is equal to time 405 minus time 401. The blocks 425 are examples of what may be referred to as “calibration data observations” or “calibration data measurements.”

[0153] In this example, graph 410 indicates calibration data quality over time and the dashed line 412 indicates a calibration data quality threshold. In this particular example, the required number of “good” calibration data observations — meaning calibration data observations when the calibration data is at or above the calibration data quality threshold 412 — is 3. Here, the timeline 420 indicates when blocks 425a of microphone data corresponding to audio calibration signals for speaker 1 are being obtained, the timeline 430 indicates when blocks 425b of microphone data corresponding to audio calibration signals for speaker 2 are being obtained, and the timeline 440 indicates the loudspeaker for which calibration signals are currently being obtained. In timeline 420, one may observe that the first two blocks 425a, labeled DW1 A and DW IB, are retained. However, we can see that in the third period of the timeline 420, the data quality metric of graph 410 drops below the threshold 412 and the third DSSS sequence (or its corresponding delay waveform) is not retained or accumulated for calibration purposes, as indicated by the dash within the thirdD25031W001 block. However, during the fourth period of the timeline 420, the data quality metric of graph 410 is above the threshold 412 and the therefore the fourth DSSS sequence (or its corresponding delay waveform DW1C) is retained. Additionally, after the system has prompted the user to move to the second device, the data quality estimate drops during the subsequent block due to the user moving the sensing device. After this period, the data quality is high and the second loudspeaker is measured without interruption. This is a commensurate approach, because the control system either keeps or discards contiguous blocks of the microphone data (or equivalently blocks of the corresponding delay waveforms).

[0154] It is also possible to not drop entire blocks corresponding to the length of the DSSS sequence. An example of this is shown in Figure 5.

[0155] Figure 5 shows alternative examples of timelines for a system with two speakers being calibrated. In this example, blocks of microphone data are kept or discarded if the calibration data quality is sufficiently high — at least on average — for an entire calibration sequence length. According to this example, the elements shown in Figure 5, including graph 410 and timelines 420-440, are intended to be the same as the corresponding elements of Figure 4, except as specifically indicated. One important difference is that the time interval between the end of block DW1C and the beginning of block DW2A is less than the length of a complete calibration data sequence. In other words, the time interval between time 507 and time 405 is less than the time interval between time 405 and time 401. In this example, the data quality trace 410 exceeds the quality threshold 412 at time 507, which corresponds to the start of block DW2A. Examples such as the one indicated by Figure 5 have potential advantages. For example, if the time interval corresponding to sensing device movement and the resulting low calibration data quality is less than the length of a complete calibration data sequence — or less than a multiple of time intervals corresponding to the length of a complete calibration data sequence — then the calibration data for the next loudspeaker measurements may begin to be obtained more quickly. However, such processes introduce some additional complexity as the time of arrival and similar, timing based estimates extracted from the periodic calibration signal present in the microphone data will be shifted according to the number of samples that were not accumulated. In the commensurate case that is described with reference to Figure 4, by accumulating or discarding contiguous blocks of length equal to the periodic calibration signal, this resulted in no relative shift in timing.D25031W001

[0156] Figure 6 is a flow diagram that outlines another example of a disclosed method. The blocks of method 600, like other methods described herein, are not necessarily performed in the order indicated. Moreover, such methods may include more or fewer blocks than shown and / or described. The method 600 may be performed by an apparatus or system, such as by the control system 150 of the apparatus 100 that is shown in Figure IB and described above.

[0157] In this example, block 605 involves obtaining, by a control system, sensor signals and microphone signals during a data-gathering process of obtaining calibration data for calibration of an audio data rendering process. In this example, each loudspeaker of the plurality of loudspeakers produces one or more audio calibration signals during the data-gathering process.

[0158] According to this example, block 610 involves estimating, by the control system, during the data-gathering process and based on at least one of the sensor signals or the microphone signals, one or more calibration data quality metrics.

[0159] In this example, block 615 involves controlling one or more aspects of the data-gathering process based, at least in part, on the one or more calibration data quality metrics.

[0160] In some examples, the sensor signals and microphone signals may be obtained by a handheld sensing device. According to some examples, the handheld sensing device may be pointed in a direction of each loudspeaker of a plurality of loudspeakers during the data-gathering process.

[0161] Controlling the one or more aspects of the data-gathering process may involve causing, by the control system, a user interface to provide a user prompt. The user prompt may be, or may include, a user instruction. The user instruction may be, or may include, an instruction to point the handheld sensing device more accurately at a loudspeaker, an instruction to reduce motion of the handheld sensing device, an instruction to wait for a quieter moment or an instruction to point the handheld sensing device at a different loudspeaker. In some examples, the user prompt may be, or may include, a data-gathering progress indication, a data quality indication, or both.

[0162] According to some examples, controlling the one or more aspects of the data-gathering process may involve altering a level of media played back by one or more loudspeakers of the plurality of loudspeakers. In some examples, the one or more audio calibration signals may be simultaneously emitted by two or more loudspeakers during at least a portion of the data-gathering process. Controlling the one or more aspects of the data-gathering process may involve causing only one loudspeaker of the plurality of loudspeakersD25031W001 to produce the one or more audio calibration signals during at least a portion of the data-gathering process.

[0163] In some examples, the one or more audio calibration signals may be, or include, direct sequence spread spectrum (DSSS) signals. In some such examples, the DSSS signals may use orthogonal spreading codes. According to some examples, each loudspeaker of the plurality of loudspeakers may produce a first DSSS signal for estimating the calibration data and a second DSSS signal for estimating calibration data quality. In some such examples, the second DSSS signal may have a shorter time duration than the first DSSS signal.

[0164] According to some examples, the calibration data may be, or may include, direction data corresponding to a direction of each loudspeaker of the plurality of loudspeakers relative to an estimated position of a person holding the handheld sensing device. In some such examples, the audio processing method may involve estimating, by the control system and based on the sensor signals, the direction data. In some examples, the calibration data may be, or may include, range data corresponding to a distance from each loudspeaker of the plurality of loudspeakers to the handheld sensing device travelled by each audio calibration signal of the one or more audio calibration signals during the data-gathering process. In some such examples, the audio processing method may involve estimating, by the control system and based on the microphone signals, the range data. In some examples, method 600 also may involve calibrating, by the control system, the audio data rendering process based, at least in part, on the direction data and the range data.

[0165] In some examples, estimating the one or more calibration data quality metrics may involve estimating a signal-to-noise ratio (SNR) of the microphone signals. According to some examples, estimating the one or more calibration data quality metrics may involve estimating a sensor data quality metric based the sensor signals. In some such examples, the sensor data quality metric may be based on an angular rate estimate, an odometry estimate, or both. In some examples, estimating the one or more calibration data quality metrics may involve estimating at least one microphone signal quality metric, estimating at least one sensor data quality metric, and estimating an overall calibration data quality metric based on the at least one microphone signal quality metric and the at least one sensor data quality metric.

[0166] According to some examples, controlling the one or more aspects of the data-gathering process may involve determining when the one or more calibration data quality metrics indicate that a calibration data quality is at or above a calibration data quality threshold. In some such examples, controlling the one or more aspects of the data-gathering process may involve determining whether the calibration data quality has been at or above a calibration dataD25031W001 quality threshold for at least a threshold time interval. In some such examples, the threshold time interval may correspond with an estimated sufficient time for obtaining a sufficient quantity of calibration data for which the calibration data quality has been at or above the calibration data quality threshold for a particular loudspeaker of the plurality of loudspeakers.

[0167] In some examples, the audio data rendering process may involve a flexible rendering process. The flexible rendering process may be, or may include, a center of mass amplitude panning process, a flexible virtualization process, a vector-based amplitude panning process, or combinations thereof.

[0168] According to some examples, the sensor signals may be, or may include, magnetometer signals, inertial sensor signals, radio signals, camera signals, or combinations thereof. In some such examples, the one or more calibration data quality metrics include at least one calibration data quality metric based on a combination of two or more types of sensor signals from a group of sensor signals that includes magnetometer signals, inertial sensor signals, radio signals and camera signals.

[0169] Some aspects of present disclosure include a system or device configured (e.g., programmed) to perform one or more examples of the disclosed methods, and a tangible computer readable medium (e.g., a disc) which stores code for implementing one or more examples of the disclosed methods or steps thereof. For example, some disclosed systems can be or include a programmable general purpose processor, digital signal processor, or microprocessor, programmed with software or firmware and / or otherwise configured to perform any of a variety of operations on data, including an embodiment of disclosed methods or steps thereof. Such a general purpose processor may be or include a computer system including an input device, a memory, and a processing subsystem that is programmed (and / or otherwise configured) to perform one or more examples of the disclosed methods (or steps thereof) in response to data asserted thereto.

[0170] Some embodiments may be implemented as a configurable (e.g., programmable) digital signal processor (DSP) that is configured (e.g., programmed and otherwise configured) to perform required processing on audio signal(s), including performance of one or more examples of the disclosed methods. Alternatively, embodiments of the disclosed systems (or elements thereof) may be implemented as a general purpose processor (e.g., a personal computer (PC) or other computer system or microprocessor, which may include an input device and a memory) which is programmed with software or firmware and / or otherwise configured to perform any of a variety of operations including one or more examples of the disclosed methods. Alternatively, elements of some embodiments of theD25031W001 inventive system may be implemented as a general purpose processor or DSP configured (e.g., programmed) to perform one or more examples of the disclosed methods, and the system also includes other elements (e.g., one or more loudspeakers and / or one or more microphones). A general purpose processor configured to perform one or more examples of the disclosed methods may be coupled to an input device (e.g., a mouse and / or a keyboard), a memory, and a display device.

[0171] Another aspect of present disclosure is a computer readable medium (for example, a disc or other tangible storage medium) which stores code for performing (e.g., coder executable to perform) one or more examples of the disclosed methods or steps thereof.

[0172] While specific embodiments of the present disclosure and applications of the disclosure have been described herein, it will be apparent to those of ordinary skill in the art that many variations on the embodiments and applications described herein are possible without departing from the scope of the disclosure described and claimed herein. It should be understood that while certain forms of the disclosure have been shown and described, the disclosure is not to be limited to the specific embodiments described and shown or the specific methods described.

Claims

D25031W001CLAIMS1. An audio processing method, comprising:obtaining, by a control system, sensor signals and microphone signals obtained during a data-gathering process of obtaining calibration data for calibration of an audio data rendering process, wherein each loudspeaker of a plurality of loudspeakers produces one or more audio calibration signals during the data-gathering process;estimating, by the control system, during the data-gathering process and based on at least one of the sensor signals or the microphone signals, one or more calibration data quality metrics; andcontrolling one or more aspects of the data-gathering process based, at least in part, on the one or more calibration data quality metrics.

2. The audio processing method of claim 1, wherein the sensor signals and the microphone signals are obtained by a handheld sensing device that is pointed in a direction of each loudspeaker of the plurality of loudspeakers during the data-gathering process.

3. The audio processing method of claim 1 or claim 2, wherein controlling the one or more aspects of the data-gathering process involves causing, by the control system, a user interface to provide a user prompt.

4. The audio processing method of claim 3, wherein the user prompt comprises a user instruction.

5. The audio processing method of claim 4, wherein the user instruction comprises an instruction to point the handheld sensing device more accurately at a loudspeaker, an instruction to reduce motion of the handheld sensing device, an instruction to wait for a quieter moment or an instruction to point the handheld sensing device at a different loudspeaker.

6. The audio processing method of claim 3, wherein the user prompt comprises a data-gathering progress indication, a data quality indication, or both.

7. The audio processing method of any one of claims 1-6, wherein controlling the one or more aspects of the data-gathering process involves altering a level of media played back by one or more loudspeakers of the plurality of loudspeakers.D25031W001 8. The audio processing method of any one of claims 1-7, wherein the one or more audio calibration signals are simultaneously emitted by two or more loudspeakers during at least a portion of the data-gathering process.

9. The audio processing method of any one of claims 1-8, wherein controlling the one or more aspects of the data-gathering process involves causing only one loudspeaker of the plurality of loudspeakers to produce the one or more audio calibration signals during at least a portion of the data-gathering process.

10. The audio processing method of any one of claims 1-9, wherein the one or more audio calibration signals are, or include, direct sequence spread spectrum (DSSS) signals utilizing orthogonal spreading codes.

11. The audio processing method of claim 10, wherein each loudspeaker of the plurality of loudspeakers produces a first DSSS signal for estimating the calibration data and a second DSSS signal for estimating calibration data quality, the second DSSS signal having a shorter time duration than the first DSSS signal.

12. The audio processing method of any one of claims 1-11, wherein the calibration data comprises direction data corresponding to a direction of each loudspeaker of the plurality of loudspeakers relative to an estimated position of a person holding the handheld sensing device and wherein the audio processing method further comprises estimating, by the control system and based on the sensor signals, the direction data.

13. The audio processing method of claim 12, wherein the calibration data comprises range data corresponding to a distance from each loudspeaker of the plurality of loudspeakers to the handheld sensing device travelled by each audio calibration signal of the one or more audio calibration signals during the data-gathering process and wherein the audio processing method further comprises estimating, by the control system and based on the microphone signals, the range data.

14. The audio processing method of claim 13, wherein the audio processing method further comprises calibrating, by the control system, the audio data rendering process based, at least in part, on the direction data and the range data.D25031W001 15. The audio processing method of any one of claims 1-14, wherein estimating the one or more calibration data quality metrics comprises estimating a signal-to-noise ratio (SNR) of the microphone signals.

16. The audio processing method of any one of claims 1-15, wherein estimating the one or more calibration data quality metrics comprises estimating a sensor data quality metric based the sensor signals and wherein the sensor data quality metric is based on an angular rate estimate, an odometry estimate, or both.

17. The audio processing method of any one of claims 1-16, wherein estimating the one or more calibration data quality metrics comprises estimating at least one microphone signal quality metric, estimating at least one sensor data quality metric, and estimating an overall calibration data quality metric based on the at least one microphone signal quality metric and the at least one sensor data quality metric.

18. The audio processing method of any one of claims 1-17, wherein controlling the one or more aspects of the data-gathering process involves determining when the one or more calibration data quality metrics indicate that a calibration data quality is at or above a calibration data quality threshold.

19. The audio processing method of claim 18, wherein controlling the one or more aspects of the data-gathering process involves determining whether the calibration data quality has been at or above a calibration data quality threshold for at least a threshold time interval.

20. The audio processing method of claim 19, wherein the threshold time interval corresponds with an estimated sufficient time for obtaining a sufficient quantity of calibration data for which the calibration data quality has been at or above the calibration data quality threshold for a particular loudspeaker of the plurality of loudspeakers.

21. The audio processing method of any one of claims 1-20, wherein the audio data rendering process comprises a flexible rendering process.

22. The audio processing method of claim 21, wherein the flexible rendering process comprises a center of mass amplitude panning process, a flexible virtualization process, a vector-based amplitude panning process, or combinations thereof.D25031W001 23. The audio processing method of any one of claims 1-22, wherein the sensor signals comprise magnetometer signals, inertial sensor signals, radio signals, camera signals, or combinations thereof.

24. The audio processing method of claim 23, wherein the one or more calibration data quality metrics include at least one calibration data quality metric based on a combination of two or more types of sensor signals from a group of sensor signals that includes magnetometer signals, inertial sensor signals, radio signals and camera signals.

25. One or more non-transitory computer-readable media having instructions stored thereon to control one or more devices to perform operations of any one of claims 1-24.

26. An apparatus configured to perform operations of any one of claims 1-24.

27. A system configured to perform operations of any one of claims 1-24.