Dynamic device speaker tuning for echo control

Abstract

Dynamic device speaker tuning for echo control includes: detecting audio rendering from a speaker on a device; based at least on detecting the audio rendering, capturing an echo of the rendered audio with a microphone on the device; performing a Fourier transform on the echo and the rendered audio; determining a real-time transfer function for at least one signature frequency band; determining a difference between the real-time transfer function and a reference transfer function; and tuning the speaker for audio rendering by adjusting an audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function. For some examples, the signature frequency band represents wall echo or an alternate mounting option. For some examples, the echo is collected during intervals while the audio rendering is ongoing.

Description

[0001] background

[0002] When a speaker is placed near certain objects (such as a wall), the resulting sound field can increase the strength of the echo path from the device speaker to the device microphone. For example, a speaker close to a wall may produce a sound with an increased bass (low frequency) level due to the wall acting as a speaker baffle. If the echo becomes so strong that acoustic echo cancellation / suppression becomes ineffective, this increased echo strength may have a negative impact on the teleconference / call quality of the remote user. Unfortunately, if the speaker amplifier of the device is permanently tuned to produce a high-quality sound field in an open area around the device, the teleconference / call quality may be affected when the device is placed near an object that can enhance the echo path. Therefore, the audio quality for both the remote party and the device user depends on where the user places the device and how the device is installed within the environment. Summary of the invention

[0003] The disclosed examples are described in detail below with reference to the accompanying drawings listed below. The following overview is provided to illustrate some examples disclosed herein. However, this is not meant to limit all examples to any specific configuration or order of operation.

[0004] Some aspects disclosed herein relate to a system for dynamic device speaker tuning for echo control, comprising: a speaker located on a device; a microphone located on the device; a processor; and a computer-readable medium storing instructions, which when executed by the processor are operable to: detect an audio rendering from the speaker; based at least on detecting the audio rendering, capture an echo of the rendered audio with the microphone; perform a Fourier transform (FT) on the echo and perform a FT on the rendered audio; determine a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function includes at least one signature band; determine a difference between the real-time transfer function and a reference transfer function; and tune the speaker for audio rendering by adjusting an audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The disclosed examples are described in detail below with reference to the drawings listed below:

[0007] Figure 1 Devices that can advantageously employ dynamic device speaker tuning for echo control are explained;

[0008] Figure 2 is a flow chart illustrating exemplary operations involved in dynamic device speaker tuning for echo control;

[0009] Figure 3is another flow chart illustrating exemplary operations involved in device characterization in support of dynamic device speaker tuning for echo control;

[0010] Figure 4 is a block diagram of example components involved in dynamic device speaker tuning for echo control;

[0011] Figure 5 An example audio rendering stream signal is shown;

[0012] Figure 6 Shown for use with Figure 5 An example of signal alignment captured by the echo stream;

[0013] Figure 7 shows an exemplary timeline of activities involved in dynamic device speaker tuning for echo control;

[0014] Figure 8 is a block diagram explaining the mathematical relationships associated with reference spectrum capture in supporting dynamic device speaker tuning for echo control;

[0015] Fig. 9 Shows Figure 8 A schematic representation of a block diagram.

[0016] Fig.10 An exemplary spectrum of rendered pink noise is shown;

[0017] Fig.11 Shows Fig.10 An exemplary spectrum of a captured echo of pink noise;

[0018] Fig.12 Shown with Fig.10 and 11 The spectrum shown in is related to the spectrum of the reference transfer function;

[0019] Fig.13 shows the spectrum of an exemplary real-time transfer function vs. Fig.12 Comparison between the spectra 1200;

[0020] Fig.14 An exemplary playback equalization spectrum to be applied to dynamic device speaker tuning is shown;

[0021] Fig.15 shows an exemplary spectral representation of an audio rendering after advantageously employing dynamic device speaker tuning;

[0022] Fig.16 yes Figure 10-15 reproduction of the spectrograms at reduced magnification so that they all fit conveniently on a single page for side-by-side viewing;

[0023] Fig.17 is another flow chart illustrating exemplary operations involved in dynamic device speaker tuning; and

[0024] Fig.18 is a block diagram of an example computing environment suitable for implementing some of the various examples disclosed herein.

[0025] Corresponding reference numerals indicate corresponding parts throughout the drawings.

[0026] Detailed Description

[0027] Various examples will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numerals will be used to refer to the same or similar components across the drawings. References to specific examples and implementations throughout this disclosure are provided for illustrative purposes only and are not meant to limit all examples unless otherwise indicated.

[0028] In a communication device that has a microphone mounted in the device for local voice pickup, the microphone also picks up a speaker signal during a call. This speaker-to-microphone signal can sometimes be heard by the remote person as an echo, even if it is not heard locally by the user of the device. Various devices have acoustic echo cancellation / suppression, but if it is overwhelmed by an overly strong echo, it will lose its effect. Since echoes generally have a main frequency component, reducing the speaker output at the main echo frequency can help maintain the echo cancellation effect. When the speaker is placed near certain objects (such as walls), the resulting sound field may increase this echo path, which in turn may negatively affect the sound quality of the remote party during a conference call in the form of echo bursts / leakage of the remote party's own voice. For example, a speaker close to a wall may produce a sound with an increased bass (low frequency) level due to the wall acting as a baffle for the speaker. This in turn can increase the echo path and can make the audio sound suboptimal to the remote party. Unfortunately, if the speaker amplifier of a device is permanently tuned to cancel the effects of expected echoes so that the audio sounds pleasing to the remote party when the device is placed near a structure that increases the level of the echo path, the device may produce a sound field of less than ideal quality for users around the device when the device is placed in an open area away from any reflective objects (such as on a cart). Thus, the audio quality for both users around the device and the remote party may depend on where the user places the device and how the device is mounted.

[0029] Accordingly, the present disclosure relates to a system for dynamic device speaker tuning for echo control, comprising: a speaker located on a device; a microphone located on the device; a processor; and a computer-readable medium storing instructions, which when executed by the processor are operable to: detect an audio rendering from the speaker; based at least on detecting the audio rendering, capture an echo of the rendered audio using the microphone; perform a Fourier transform (FT) on the echo and perform a FT on the rendered audio; determine a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function includes at least one signature band; determine a difference between the real-time transfer function and a reference transfer function; and tune the speaker for audio rendering by adjusting an audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function.

[0030] Figure 1 Device 100 is illustrated that can advantageously employ dynamic device speaker tuning for echo control. In some examples, device 100 is a version of computing device 1800 that is related to Fig.18 The device 100 has a processor 1814, a memory 1812, and a presentation component 1816, which is related to ( Fig.18 1800 is described in more detail. The device 100 includes a speaker 170 located on the device 100 and a microphone 172 also located on the device 100. Some examples of the device 100 have multiple speakers 170 for stereo or other enhanced audio, such as separate bass and higher pitch (midrange and treble) speakers. Some examples of the device 100 have multiple microphones 172 for stereo audio or noise cancellation. In such systems, the process described herein can be applied to each audio channel. In some examples, using multiple speakers and microphones, audio beamforming can be advantageously adopted. The microphone 172 and the speaker 170 can be considered as part of the presentation component 1816.

[0031] As illustrated, echo path 174 returns the audio rendered from speaker 170 to microphone 172 after reflecting from wall 176. When the device moves away from wall 176, another echo path may exist due to base 178 and / or other nearby objects. Some examples of device 100 are mounted on walls, while other examples are mounted on transportable carts, while other examples are placed on tables. Some examples of device 100 move between various locations. Some examples of device 100 include video screens with audio capabilities exceeding 50 inches. Therefore, the speaker tuning described herein can dynamically compensate for different sound environments. In some examples, dynamic tuning enhances audio quality and also reduces echo and noise. In some examples, dynamic tuning is optimized for speech, although in some examples, dynamic tuning can be selectively controlled to optimize for speech or music.

[0032] Memory 1812 stores application logic 110 and data 140, which includes components (instructions and data) for performing the operations described herein. Audio rendering component 112 uses audio amplifier 160 to render audio from audio data 142 on speaker 170. Audio may include music, voice conversations (e.g., conference calls routed on wireless component 188), or audio tracks stored in audio data 142. A copy of the rendered audio is stored in data 140 as rendered audio 146. Some examples of audio amplifier 160 support parametric equalization or some other means of adjusting a specific frequency band, including bandpass filtering. Some examples of audio amplifier 160 support audio compression. Audio detection component 114 detects audio rendering picked up by microphone 172 from speaker 170 and passed through microphone equalizer 162. Some examples of microphone equalizer 162 support audio compression. Based at least on the detection of audio rendering, audio capture component 116 captures echoes of rendered audio using microphone 172. A copy of the captured echo is stored in data 140 as captured echo 144 .

[0033] The capture control 118 controls the audio capture component 116 (e.g., using the timer 186). In some examples, capturing the echo includes capturing the echo during a first time interval within a second time interval, the second time interval being longer than the first time interval; and repeating the capture at the completion of each second interval while the audio rendering is in progress (e.g., Figure 7 18). In some examples, audio capture is triggered by user input through presentation component 1816. In some examples, one or more of sensors 182 and 184 indicates that device 100 has moved, and this triggers audio capture. Sensor 182 is illustrated as an optical sensor, but it should be understood that other types of sensors, such as proximity sensors, may also be used. Additional aspects regarding the operation of capture control 118 will be described in detail below. Figure 7Describe in more detail.

[0034] The signal component 120 aligns the captured echo 144 with the rendered audio 146 when necessary to obtain a better synchronized frequency response between the two signals. The signal windowing component windows segments of the captured echo 144 and also segments of the rendered audio 146. The FT logic component 124 performs FT on the captured echo 144 and also performs FT on the rendered audio 146. In some examples, the FT is a fast Fourier transform (FFT). In some examples, the FT logic component 124 is implemented on a digital signal processing (DSP) component. In Figure 6 And the additional description of signal alignment, signal windowing and FT operation is described in the subsequent figures. In some examples, the captured echo 144 may include local voice pickup. In some examples, the captured echo 144 may include local noise from the environment. In such examples, energy calculations (such as coherence calculations) can determine whether the captured audio mainly includes echoes rendered from the speaker 170. The coherence calculation compares the power spectrum of the captured echo 144 with the rendered audio 146 to determine whether the power transfer between the signals meets a threshold. The transfer function generator 126 determines the real-time transfer function 148 based on at least the FT of the captured echo 144 and the FT of the rendered audio 146 and stores it in the data 140. In some examples, determining the real-time transfer function 148 includes dividing the amplitude of the FT of the captured echo 144 by the FT of the rendered audio 146.

[0035] The real-time transfer function 148 is compared with the reference transfer function 150 by the transfer function comparison component 128. In some examples, a spectrum mask 152 is applied to the real-time transfer function 148 and the reference transfer function 150 for comparison to isolate a specific frequency band of interest. In some examples, the spectrum mask 152 includes at least one signature band identified in the signature band data 154. The signature band is a portion (band) of the audio spectrum that is particularly affected by specific environmental factors. In some examples, the signature band includes a signature band for wall echo, which is approximately 300 hertz (Hz). In some examples, the signature band includes a signature band for base echo (e.g., echo from the base 178). The transfer function comparison component 128 determines the difference between the real-time transfer function 148 and the reference transfer function 150. In some examples, a band threshold 156 is used to determine whether any tuning will occur within a specific frequency band. For example, if the difference is below a threshold for a frequency band, there will not be any tuning changes in the specific frequency band. Thus, in some examples, the transfer function comparison component 128 is further operable to determine whether the difference between the real-time transfer function 148 and the reference transfer function 150 exceeds a threshold value within a first frequency band. In such examples, tuning the speaker 170 for audio rendering includes tuning the speaker 170 for audio rendering within the first frequency band based at least on the difference between the real-time transfer function 148 and the reference transfer function 150 exceeding the threshold value. In some examples, the transfer function comparison component 128 is further operable to determine whether the difference between the real-time transfer function 148 and the reference transfer function 150 exceeds a threshold value within a second frequency band different from the first frequency band. In such examples, tuning the speaker 170 for audio rendering includes tuning the speaker 170 for audio rendering within the second frequency band based at least on the difference between the real-time transfer function 148 and the reference transfer function 150 exceeding the threshold value (for the second frequency band).

[0036] When tuning is indicated by the output of the transfer function comparison component 128, the tuning control component tunes the speaker 170 for audio rendering by adjusting the audio amplifier 160 equalization based at least on the difference between the real-time transfer function 148 and the reference transfer function 150. Other logic 132 and other data 158 include other logic and data required to perform the operations described herein. Some examples of other logic 132 include artificial intelligence (AI) or machine learning (ML) capabilities. The ML capabilities can be advantageously employed (e.g., using sensors 182 and 184 and tuning control history) to identify environmental factors to improve the equalization of the audio amplifier 160. In some examples, user control of the equalization is also input into the ML capabilities to predict the desired adjustment parameters.

[0037] Figure 22 is a flowchart 200 illustrating exemplary operations of device 100 involved in dynamic device speaker tuning for echo control. Flowchart 200 begins with operation 202, where a sound engineer develops the audio components of device 100 into a target audio profile so that the device provides pleasing sound in an appropriate environment. Operation 204 characterizes the audio components of device 100 and describes the operation of the device 100. Figure 3 178 is described in more detail. In operation 206, a usage scenario category is determined (e.g., operation of the device 100 on a particular base 178 near a wall). In operation 208, signature bands for different usage scenario categories are determined, which may be loaded onto the device 100 (e.g., in signature band data 154). This permits the device 100 to determine certain environmental conditions (e.g., the device 100 is near a wall) by comparing the echo spectral characteristics with the signature band data 154. The spectrum mask 152 is generated in operation 210 using the signature bands. This permits the tuning operation to have a more significant effect by focusing attention on frequency bands that show more significant environmental dependence.

[0038] The reference transfer function 150 and the spectral mask 152 are loaded onto the device 100 in operation 212. The reference transfer function 150 describes the target audio profile as it is the result of tuning by an audio engineer in a favorable environment. The device 100 is deployed in operation 214, and an ongoing dynamic speaker tuning loop 216 begins each time audio is being rendered by the device 100. Loop 216 includes real-time audio capture in operation 218, spectral analysis of the captured echo 144 in 220, and playback equalization (of the audio amplifier 160) in operation 222. Loop 216 then returns to operation 218 and continues as audio is rendered.

[0039] Figure 3 304 is a flowchart explaining further details of operation 204. After the audio engineer has ensured that the features of the device 100 are complete and all hardware and firmware have been verified, operation 204 begins. In addition to loading the tuning profile data, the device 100 should be in a state where it will be deployed (e.g., delivered to the user). In operation 302, the device 100 is placed in an echo-free environment where reverberation and reflection do not interfere with the echo path. In operation 304, the device 100 is turned on and operation 306 begins to capture (record) audio using the microphone 172. In operation 308, pink noise is rendered (played through the speaker 170). In operation 310, pink noise of a certain time length (e.g., several seconds) picked up by the microphone 172 is captured and saved. Subsequently, operation 312 uses the FT of the pink noise and the FT of the audio captured in operation 310 to generate (calculate) a reference transfer function 150. In some examples, a portion of the calculation is processed remotely, rather than being processed entirely on the device 100.

[0040] Figure 4 4 is a block diagram 400 of example components involved in dynamic device speaker tuning for echo control for device 100. Reference source 402 provides white noise or pink noise, such as for Figure 3 described during device characterization. In some examples, reference source 402 is an external source or a software component running on device 100. Calibration noise is provided to audio amplifier 160 and rendered (played) by speaker 170. During device characterization, this occurs in a calibration quality anechoic environment 406. Sound energy is captured by microphone 172, passed through microphone equalizer 162, and saved in reference capture 410. Both reference source 402 and reference capture 410 each provide their respective signals to alignment and windowing component 414, which includes both signal alignment component 120 and signal windowing component 122. To assist in tracking Figure 4 4. In the signal paths of FIG. 4, the signal from the reference source 402 is shown as a dashed line and the signal from the reference capture 410 is shown as a dot-dash line.

[0041] The alignment and windowing component 414 sends the aligned and windowed signals to the FT and amplitude calculation component 416. The signals originating from the reference source 402 and the reference capture 410 are still depicted as dashed and dotted lines, respectively. The FT and amplitude calculation component 416 performs a Fourier transform and finds the amplitude for each signal and passes these signals to the comparator component 418, which performs the amplitude of the FT of the reference capture 410 signal divided by the amplitude of the FT of the reference source 402 signal. This provides (generates or calculates) the reference transfer function 150 stored on the device 100, as described above.

[0042] When the end user possesses the device 100, dynamic speaker tuning utilizing the reference transfer function 150 may be advantageously employed. With respect to a similar signal path, a real-time source 404 (e.g., playing audio data 142) provides an audio signal to an audio amplifier 160, which is then rendered by a speaker 170. This occurs in the user's environment 408, which may be near a wall 176, on a base 178, or some other environment that may not be conducive to sound reproduction. The sound energy in the echo is captured by the microphone 172, passes through the microphone equalizer 162, and is saved in the real-time capture 412 as a captured echo 144. A copy of the rendered audio 146 (from the real-time source 404) is saved. Each of the rendered audio 146 and the captured echo 144 is provided to an alignment and windowing component 414. To assist in tracking Figure 4 , the signal from the rendered audio 146 is shown as a dashed line and the signal from the captured echo 144 is shown as a solid line.

[0043] The alignment and windowing component 414 sends the aligned and windowed signals to the FT and amplitude calculation component 416. The signals originating from the rendered audio 146 and the captured echo 144 are still depicted as dashed and solid lines, respectively. The FT and amplitude calculation component 416 performs a Fourier transform and finds the amplitude for each signal and passes these signals to the comparator component 420, which performs the amplitude of the FT of the captured echo 144 divided by the amplitude of the FT of the rendered audio 146. This provides (generates or calculates) a real-time transfer function 148. Because the FT assumes a periodic signal, windowing simulates the real-time signal as periodic and provides a good approximation of the frequency domain content. Both the real-time transfer function 148 and the reference transfer function 150 are provided to the transfer function comparison component 128, which drives the tuning control 130 to adjust the audio amplifier 160 equalization. In some examples, a portion of the calculation is processed remotely, rather than being processed entirely on the device 100.

[0044] The technique provides a continuous closed loop (feedback loop) that adapts to the environment in which the device 100 is placed. The four overall stages are: (1) device characterization, (2) data capture, (3) spectrum analysis, and (4) equalization. The device characterization stage addresses the problem that the acoustic echo characteristics will be unique to the device form factor due to the microphone and speaker locations. A desired echo spectrum characterization is required to serve as a reference for adaptive tuning. However, if there is no device form factor change, it is only required once. During the data capture stage, the device 100 periodically polls the echo from the speaker 170 to the microphone 170 (or from multiple speakers 170 to multiple microphones 170). This requires simultaneous capture and rendering of audio streams, which is common in Voice over Internet Protocol (VOIP) calls. During the spectrum analysis stage, a DSP component, whether through the cloud or embedded in the device 100, converts the time domain audio data to the frequency domain. The DSP compares the energy spectrum of the audio to the reference mask from the device characterization stage. During the equalization stage, deviations from the predetermined frequency mask will be corrected by the DSP by applying filters to bring the captured audio closer to the mask.

[0045] Figure 5 An example rendered audio signal 500 is shown having a start point 602 at the same time as the start point 602. Figure 6 The signal 600 is aligned with the starting point 502 before. The starting points 502 and 602 are signals that are above any noise 504 and 604 that may be present. To align, the signals 500 and 600 are shifted in time relative to each other so that the starting points 502 and 602 coincide.

[0046] Figure 7 The activities involved in dynamic device speaker tuning are shown (e.g., by ( Figure 1) capture control 118 controlled activities). In some examples, capturing the echo (e.g., via capture echo 144) includes capturing the echo during a first time interval 702a or 702b within a second time interval 704a or 704b, wherein the second time interval (704a or 704b) is longer than the first time interval (702a or 702b, respectively); and repeating the capture upon completion of each second interval (704a or 704b) while audio rendering is ongoing. Figure 1 The timer 186 is used to time the various intervals. As indicated, the rendered audio is stored (e.g., as rendered audio 146) during the time that the captured echo 144 is stored. Each of the rendered audio 146 and the captured echo 144 is provided to the alignment and windowing component 414. Figure 4 Consistently, the signal from the rendered audio 146 is shown as a dashed line and the signal from the captured echo 144 is shown as a solid line.

[0047] Figure 8 is a block diagram 800 explaining the mathematical relationships associated with reference spectrum capture, and Fig. 9 A schematic representation 900 of the block diagram 800 is shown. In the time domain representation, the source x(t) is convolved with the time domain transfer function h(t) to give the result (here the captured echo) captured y(t). However, applying the FT 802 in the frequency domain representation, i.e., the source X(f) multiplied by the frequency domain transfer function H(f) gives captured Y(f). Therefore, the division operation 902 shown in the schematic representation 900 generates (calculates) H(f) as captured Y(f) divided by the source X(f). This is also shown in equations (1) and (2):

[0048] X(f)xH(f)=Y(f) Formula (1)

[0049]

[0050] Fig.10 An exemplary spectrum 1000 of rendered pink noise is shown, and Fig.11 Shows Fig.10 An exemplary spectrum 1100 of a captured echo of pink noise. Fig.12 A spectrum 1200 of a reference echo system (in this case, reference transfer function 150) is shown. A signature band 1202 of the spectral power response that can be expected to increase when device 100 is placed near wall 176 is identified. In some examples, the wall signature band ranges from about 200 Hz to about 600 Hz. Spectrum 1200 is calculated by dividing spectrum 1100 by spectrum 1000. Because the figures are scaled in decibels (dB), multiplication and division are shown as addition and subtraction in the figure.

[0051] Fig.13 A comparison between a spectrum 1300 for an exemplary real-time transfer function (eg, real-time transfer function 148) and a spectrum 1200 for a reference echo system (eg, reference transfer function 150) is shown. Fig.13 , spectrum 1300 has an elevated amplitude within signature band 1202 relative to spectrum 1200. This indicates that device 100 is operating near a wall (eg, wall 176). Fig.14 A calculated playback equalization spectrum 1400 is shown to be applied to 160 by tuning control 130. Reduction 1402 in spectrum 1400 is evident due to proximity to a wall to help reduce the effects of excess bass.

[0052] Fig.15 An exemplary spectrum representation of an audio rendering is shown after dynamic device speaker tuning has been advantageously employed. The rendered spectrum 1500, while not perfect, is still fairly close to spectrum 1200 and exhibits fewer wall echo effects. Fig.16 yes Figure 10-15 1000, 1100, 1200, 1300, 1400, and 1500 are reproduced at reduced magnifications so that they all conveniently fit on a single page for side-by-side viewing. Although the above process compares the energy of signals (e.g., rendered audio signals and echo audio signals (such as within a particular frequency band)), it should be noted that there are alternative methods to compare the energy of signals based on the location where the device 100 is placed. In some examples, time domain energy analysis is used to determine the signal energy remaining after bandpass filtering. In such examples, the passband is centered on the frequencies of interest in the signature band based on device characteristics and certain echo scenarios (e.g., wall echoes). Both the rendered and captured echo signals are subjected to bandpass filtering and energy detection, and the ratio of the signal energies can then be used to pinpoint the presence of significant echoes.

[0053] Fig.17 1700 is a flowchart illustrating exemplary operations involved in dynamic device speaker tuning. In some examples, the operations described with respect to flowchart 1700 are performed by Fig.18 1800. Flowchart 1700 begins at operation 1702 with a user rendering an audio stream (e.g., by starting a VOIP call or playing music on a device). Operation 1704 includes detecting the rendering of audio from a speaker on the device. Decision operation 1706 continues the adaptive tuning algorithm described herein or ends the tuning activity when the rendering is complete. Operation 1708 detects environmental changes using sensors (such as accelerometers that sense movement).

[0054] In operation 1710, a timer is started to determine when an audio capture event will begin and end. The timer determines how often the algorithm will begin recording the loopback audio and the captured audio and how often the playback tuning is adjusted. Operation 1712 includes capturing the echo of the rendered audio using a microphone on the device based at least on the detection of the audio rendering. The captured echo is stored in a buffer in the memory. In some examples, capturing the echo includes capturing the echo during a first time interval within a second time interval, the second time interval being longer than the first time interval; and repeating the capture at the completion of each second interval while the audio rendering is in progress. Operation 1714 includes aligning the echo with a copy of the rendered audio. Because the captured audio is processed and the transmission time of the round-trip reflective surface, it will be delayed relative to the loopback captured directly from the source. Signal alignment is applied to the two signals (usually using cross-correlation techniques) so that they are synchronized with each other sample by sample. If necessary, the audio samples are windowed in operation 1716. In general, it is recommended to window to calculate an accurate FT, for example to avoid spectral leakage.

[0055] Operation 1718 includes performing FT on the echo and performing FT on the rendered audio. The two signals are now in the frequency domain. In some examples, the FT includes an FFT. Operation 1720 calculates the FT amplitude to provide a frequency response. Operation 1722 determines whether the captured audio mainly contains noise, or alternatively whether a significant portion of the captured audio comes from the audio that has been rendered from the speaker. That is, operation 1722 includes determining whether the portion above the threshold in the captured audio includes the echo of the rendered audio. If the captured audio mainly contains noise (as determined in decision operation 1724), audio tuning may not be required at this time. However, if the captured audio contains the echo of the rendered audio, operation 1726 includes determining a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function includes at least one signature band. In some examples, determining the real-time transfer function includes dividing the amplitude of the FT of the echo by the FT of the rendered audio. In some examples, the signature band includes a signature band for wall echoes. In some examples, the signature band includes a signature band for base echoes. Operation 1728 then includes determining the difference between the real-time transfer function and the reference transfer function. To do this, the frequency response of the captured signal is divided by the frequency response of the source signal. This is the real-time transfer function.

[0056] In some examples, the difference is determined by the energy within a signature band (e.g., a 200 Hz to 400 Hz or 600 Hz band or some other band). The energy change in the signature band is compared to the ideal energy change for the same band in the reference transfer function. The energy comparison between the real-time and reference transfer functions determines how the amplifier equalization is adjusted. If the real-time energy is higher, the equalization is adjusted to reduce it to more closely match the reference energy. The process depends on the equalization architecture and how easily it can be adjusted. Some equalizers are parameterized, which simplifies adjusting the gain in a particular band. Decision operation 1730 determines whether to check another band for the difference, and if necessary, repeats operation 1728.

[0057] Operation 1732 includes determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold value in a first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering in the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold value. If more than one frequency band is used to determine the transfer function difference, operation 1732 is repeated for additional frequency bands. Some examples of operation 1732 include determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold value in a second frequency band different from the first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering in the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold value. If the difference is below the threshold value (e.g., the transfer response is sufficiently similar) (as determined in decision operation 1734) or no longer changes, the tuning is complete.

[0058] If tuning is required, operation 1736 includes tuning the speaker for audio rendering by adjusting the audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function. The timer is reset in operation 1738, and flowchart 1700 returns to operation 1704 to find out whether the speaker is still rendering audio.

[0059] Additional Examples

[0060] Some aspects and examples disclosed herein relate to a system for dynamic device speaker tuning for echo control, comprising: a speaker located on a device; a microphone located on the device; a processor; and a computer-readable medium storing instructions, which when executed by the processor are operable to: detect an audio rendering from the speaker; based at least on detecting the audio rendering, capture an echo of the rendered audio using the microphone; perform FT on the echo and perform FT on the rendered audio; determine a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function includes at least one signature band; determine a difference between the real-time transfer function and a reference transfer function; and tune the speaker for audio rendering by adjusting audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function.

[0061] Additional aspects and examples disclosed herein relate to a method for dynamic device speaker tuning for echo control, comprising: detecting audio rendering from a speaker on a device; capturing an echo of the rendered audio using a microphone on the device based at least on detecting the audio rendering; performing FT on the echo and performing FT on the rendered audio; determining a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function includes at least one signature band; determining a difference between the real-time transfer function and a reference transfer function; and tuning the speaker for audio rendering by adjusting audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function.

[0062] Additional aspects and examples disclosed herein relate to one or more computer storage devices having stored thereon computer executable instructions for dynamic device speaker tuning for echo control, which when executed by a computer cause the computer to perform operations including: detecting audio rendering from a speaker on a device; capturing an echo of the rendered audio using a microphone on the device based at least on detecting the audio rendering, wherein capturing the echo comprises capturing the echo during a first time interval within a second time interval, wherein the second time interval is longer than the first time interval; and repeating the capturing upon completion of each second interval while the audio rendering is ongoing; capturing the echo of the rendered audio using a microphone on the device ... The echo is aligned with a copy of the rendered audio; FT is performed on the echo, and FT is performed on the rendered audio; based at least on the FT of the echo and the FT of the rendered audio, a real-time transfer function is determined, wherein determining the real-time transfer function comprises dividing the magnitude of the FT of the echo by the magnitude of the FT of the rendered audio, and wherein the real-time transfer function comprises at least one signature band, and wherein the signature band comprises a signature band for wall echo; determining a difference between the real-time transfer function and a reference transfer function; and based at least on the difference between the real-time transfer function and the reference transfer function, tuning the speaker for audio rendering by adjusting an audio amplifier equalization.

[0063] As an alternative or in addition to other examples described herein, examples include any combination of the following:

[0064] Capturing the echo includes capturing the echo during a first time interval within a second time interval, the second time interval being longer than the first time interval; and repeating the capturing upon completion of each second interval while the audio rendering is in progress;

[0065] The instructions are further operable to align the echo with a copy of the rendered audio;

[0066] aligning the echo with a copy of the rendered audio;

[0067] The FT includes FFT;

[0068] determining whether a portion of the captured audio above a threshold includes an echo of the rendered audio;

[0069] Determining the real-time transfer function includes dividing the magnitude of the FT of the echo by the magnitude of the FT of the rendered audio;

[0070] The signature bands include signature bands for wall echoes;

[0071] The signature bands include a signature band for a base echo;

[0072] The instructions are further operable to determine whether a difference between the real-time transfer function and the reference transfer function exceeds a threshold within a first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering within the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

[0073] Determining whether a difference between the real-time transfer function and the reference transfer function exceeds a threshold within a first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering within the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

[0074] The instructions are further operable to determine whether a difference between the real-time transfer function and the reference transfer function exceeds a threshold within a second frequency band different from the first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering within the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

[0075] Determining whether a difference between the real-time transfer function and the reference transfer function exceeds a threshold in a second frequency band different from the first frequency band; and tuning the speaker for audio rendering includes tuning the speaker for audio rendering in the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

[0076] Although aspects of the present disclosure have been described in terms of various examples and their associated operations, those skilled in the art will appreciate that combinations of operations from any number of the different examples are also within the scope of aspects of the present disclosure.

[0077] Example operating environment

[0078] Fig.18It is a block diagram of an example computing device 1800 for implementing various aspects disclosed herein, and is generally designated as computing device 1800. Computing device 1800 is only an example of a suitable computing environment, and is not intended to impose any restrictions on the scope of use or functionality of the examples disclosed herein. Computing device 1800 should also not be interpreted as having any dependency or requirement on any component / module or combination thereof illustrated. The examples disclosed herein can be described in the general context of computer code or machine-usable instructions (including computer executable instructions such as program components) executed by a computer or other machines such as a personal data assistant or other handheld devices. In general, program components including routines, programs, objects, components, data structures, etc. refer to codes that perform specific tasks or implement specific abstract data types. The disclosed examples can be implemented in various system configurations, including personal computers, laptop computers, smart phones, mobile tablets, handheld devices, consumer electronics, professional computing devices, etc. When the task is performed by a remote processing device linked by a communication network, the disclosed examples can also be implemented in a distributed computing environment.

[0079] The computing device 1800 includes a bus 1810 that directly or indirectly couples the following devices: computer storage memory 1812, one or more processors 1814, one or more presentation components 1816, input / output (I / O) ports 1818, I / O components 1820, power supply 1822, and network components 1824. Although the computer device 1800 is depicted as a seemingly single device, multiple computing devices 1800 can work together and share the depicted device resources. For example, the memory 1812 can be distributed across multiple devices, the processor 1814 can be installed on different devices, and so on.

[0080] Bus 1810 may represent one or more buses (such as an address bus, a data bus, or a combination thereof). Although shown with lines for clarity, Fig.18 , but in reality, the outlines of the components are not that clear, and metaphorically speaking, the lines would more accurately be gray and fuzzy. For example, presentation components such as display devices can be considered I / O components. Moreover, processors also have memory. This is a feature of the art, and to reiterate, Fig.18 The illustrations are merely illustrative of exemplary computing devices that can be used in conjunction with one or more of the disclosed examples. No distinction is made between categories such as "workstation," "server," "laptop," "handheld device," etc., all of which are considered to be within the meaning of the present disclosure. Fig.181800 and is referred to herein as a "computing device". Memory 1812 may take the form of the following computer storage media references and is operable to provide storage of computer-readable instructions, data structures, program modules, and other data for computing device 1800. In some examples, memory 1812 stores one or more of an operating system, a general-purpose application platform, or other program modules and program data. Thus, memory 1812 is capable of storing and accessing instructions configured to perform various operations disclosed herein.

[0081] In some examples, memory 1812 includes computer storage media in the form of volatile and / or nonvolatile memory, removable or non-removable memory, data disks in a virtual environment, or a combination thereof. Memory 1812 may include any number of memories associated with or accessible to computing device 1800. Memory 1812 may be internal to computing device 1800 (e.g., Fig.18 1800 ), external to the computing device 1800 (not shown), or both (not shown). Examples of memory 1812 include, but are not limited to, random access memory (RAM); read-only memory (ROM); electronically erasable programmable read-only memory (EEPROM); flash memory or other memory technology; CD-ROM, digital versatile disk (DVD) or other optical or holographic media; magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices; memory wired to an analog computing device; or any other medium for encoding the desired information and accessed by the computing device 1800. Additionally or alternatively, memory 1812 may be distributed across multiple computing devices 1800, for example, in a virtualized environment where instruction processing is performed on multiple devices 1800. For purposes of this disclosure, "computer storage media," "computer storage memory," "memory," and "memory device" are synonymous terms for computer storage memory 1812, and none of these terms include carrier waves or propagating signaling.

[0082] Processor 1814 may include any number of processing units that read data from various entities (such as memory 1812 or I / O components 1820). Specifically, processor 1814 is programmed to execute computer executable instructions for implementing various aspects of the present disclosure. These instructions can be executed by a processor, by multiple processors within computing device 1800, or by a processor outside client computing device 1800. In some examples, processor 1814 is programmed to execute instructions such as those shown in the flowcharts discussed below and depicted in the accompanying drawings. Moreover, in some examples, processor 1814 represents an implementation of simulation technology for performing the operations described herein. For example, these operations can be performed by simulated client computing device 1800 and / or digital client computing device 1800. Presentation component 1816 presents data indication to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc. Those skilled in the art will appreciate and understand that computer data can be presented in a variety of ways, such as visually in a graphical user interface (GUI), audibly through a speaker, wirelessly between computing devices 1800, through a wired connection, or otherwise. I / O ports 1818 allow computing device 1800 to be logically coupled to other devices including I / O components 1820, some of which may be built-in. Example I / O components 1820 include, for example, but are not limited to, microphones, joysticks, game controllers, satellite dishes, scanners, printers, wireless devices, and the like.

[0083] The computing device 1800 can operate in a network environment using logical connections to one or more remote computers via a network component 1824. In some examples, the network component 1824 includes a network interface card and / or computer executable instructions (e.g., a driver) for operating a network interface card. Communications between the computing device 1800 and other devices can occur over any wired or wireless connection using any protocol or mechanism. In some examples, the network component 1824 can be operable to communicate between public, private, or hybrid (public and private) devices using a transport protocol by using short-range communication technologies (e.g., near field communication (NFC), Bluetooth, etc.). TM Brand communication, etc.) or a combination thereof wirelessly communicates data. For example, network component 1824 communicates with network 1830 on communication link 1832.

[0084] Although described in conjunction with an example computing device 1800, the examples of the present disclosure can be implemented with numerous other general or special computing system environments, configurations, or devices. Examples of well-known computing systems, environments, and / or configurations suitable for various aspects of the present disclosure include, but are not limited to: smart phones, mobile tablets, mobile computing devices, personal computers, server computers, handheld or laptop devices, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, mobile computing and / or communication devices with wearable or accessory form factors (e.g., watches, glasses, headphones or earbuds), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, VR devices, holographic devices, etc., and the like. Such systems or devices can accept input from a user in any manner, including from an input device such as a keyboard or pointing device, through gesture input, proximity input (such as through hovering), and / or through voice input.

[0085] Each example of the present disclosure may be described in the general context of computer executable instructions (such as program modules) executed by one or more computers or other devices in software, firmware, hardware or a combination thereof. Computer executable instructions may be organized into one or more computer executable components or modules. In general, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform specific tasks or implement specific abstract data types. Any number of such components or modules and any organization thereof may be utilized to implement various aspects of the present disclosure. For example, various aspects of the present disclosure are not limited to specific computer executable instructions or specific components or modules illustrated in the accompanying drawings and described herein. Other examples of the present disclosure may include different computer executable instructions or components having more or less functions than those shown and described herein. In an example involving a general-purpose computer, when configured to execute instructions described herein, various aspects of the present disclosure convert the general-purpose computer into a special-purpose computing device.

[0086] As an example and not limitation, computer-readable media include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable memory implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, etc. Computer storage media are tangible and mutually exclusive with communication media. Computer storage media are implemented in hardware and exclude carrier waves and propagation signals. Computer storage media for the purposes of this disclosure are not signals themselves. Exemplary computer storage media include hard disks, flash drives, solid-state memories, phase change random access memories (PRAM), static random access memories (SRAM), dynamic random access memories (DRAM), other types of random access memories (RAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROM), flash memory or other memory technologies, compact disk read-only memories (CD-ROM), digital versatile disks (DVD) or other optical memories, cassettes, tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computing device. In contrast, communication media typically embodies computer readable instructions, data structures, program modules, etc. in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0087] The execution or completion order of the operations in each example of the present disclosure illustrated and described herein is not necessary, but can be executed in different sequential ways in various examples. For example, it is contemplated that before, at the same time, or after a certain operation, another operation is executed or completed, which is also within the scope of the various aspects of the present disclosure. When introducing the elements of various aspects of the present disclosure or its examples, the articles "one", "one", "the", "said" are intended to represent one or more of the elements. The terms "include", "comprise", and "have" are intended to be inclusive, and represent that additional elements may be present in addition to the listed elements. The term "exemplary" is intended to represent "an example of...". The phrase "one or more of the following: A, B, and C" refers to "at least one A and / or at least one B and / or at least one C".

[0088] Having described various aspects of the disclosure in detail, it is apparent that various modifications and variations may be made without departing from the scope of the various aspects of the disclosure as defined in the appended claims. Various changes may be made in the above constructions, products, and methods without departing from the scope of the various aspects of the disclosure, it being intended that all subject matter contained in the above description and shown in the accompanying drawings should be interpreted as illustrative rather than restrictive.

Claims

1. A system for dynamic device speaker tuning for echo control, the system comprising: Speakers located on the device; a microphone located on said device; processor; as well as a computer readable medium storing instructions that, when executed by the processor, are operable to: detect audio rendering from the speaker; Based at least on detecting the audio rendering, capturing an echo of the rendered audio with the microphone, comprising: capturing the echo during a first time interval within a second time interval, wherein the second time interval is longer than the first time interval; and repeating the capturing upon completion of each second time interval while the audio rendering is ongoing; performing a Fourier transform FT on the echo and performing a FT on the rendered audio; determining a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function comprises at least one signature band; determining a difference between the real-time transfer function and a reference transfer function describing a target audio profile and generated based on tuning in a favorable environment; and tuning the speaker for audio rendering by adjusting audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function; The echo is aligned with a copy of the rendered audio.

2. The system of claim 1, wherein the FT comprises a Fast Fourier Transform (FFT).

3. The system of claim 1, wherein determining the real-time transfer function comprises dividing a magnitude of the FT of the echo by a magnitude of the FT of the rendered audio.

4. The system of claim 1, wherein the signature frequency band comprises a signature frequency band for wall echoes.

5. The system of claim 1, wherein the instructions are further operable to: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold within a first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

6. The system of claim 5, wherein the instructions are further operable to: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold in a second frequency band different from the first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

7. A method of dynamic device speaker tuning for echo control, the method comprising: Detect audio rendering from speakers on the device; Based at least on detecting the audio rendering, capturing an echo of the rendered audio using a microphone on the device, including capturing the echo during a first time interval within a second time interval, wherein the second time interval is longer than the first time interval; and repeating the capturing upon completion of each second time interval while the audio rendering is ongoing; performing a Fourier transform FT on the echo and performing a FT on the rendered audio; determining a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein the real-time transfer function comprises at least one signature band; determining a difference between the real-time transfer function and a reference transfer function describing a target audio profile and generated based on tuning in a favorable environment; tuning the speaker for audio rendering by adjusting audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function; as well as The echo is aligned with a copy of the rendered audio.

8. The method of claim 7, wherein the FT comprises a Fast Fourier Transform (FFT).

9. The method of claim 7, wherein determining the real-time transfer function comprises dividing the magnitude of the FT of the echo by the magnitude of the FT of the rendered audio.

10. The method of claim 7, wherein the signature frequency band comprises a signature frequency band for wall echoes.

11. The method of claim 7, further comprising: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold within a first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

12. The method of claim 11, further comprising: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold in a second frequency band different from the first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

13. One or more computer storage devices having stored thereon computer executable instructions for dynamic device speaker tuning for echo control, the computer executable instructions, when executed by a computer, causing the computer to perform operations comprising: Detect audio rendering from speakers on the device; Based at least on detecting the audio rendering, capturing an echo of the rendered audio using a microphone on the device, wherein capturing the echo comprises: capturing the echo during a first time interval within a second time interval, wherein the second time interval is longer than the first time interval; and repeating said capturing upon completion of each second time interval while audio rendering is ongoing; aligning the echo with a copy of the rendered audio; performing a Fourier transform FT on the echo, and performing a FT on the rendered audio; determining a real-time transfer function based at least on the FT of the echo and the FT of the rendered audio, wherein determining the real-time transfer function comprises dividing the magnitude of the FT of the echo by the magnitude of the FT of the rendered audio, and wherein the real-time transfer function comprises at least one signature band, and wherein the signature band comprises a signature band for wall echo; determining a difference between the real-time transfer function and a reference transfer function describing a target audio profile and generated based on tuning in a favorable environment; and The speaker is tuned for audio rendering by adjusting audio amplifier equalization based at least on the difference between the real-time transfer function and the reference transfer function.

14. The one or more computer storage devices of claim 13, wherein the FT comprises a Fast Fourier Transform (FFT).

15. The one or more computer storage devices of claim 13, wherein the operations further comprise: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold within a first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the first frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

16. The one or more computer storage devices of claim 15, wherein the operations further comprise: determining whether the difference between the real-time transfer function and the reference transfer function exceeds a threshold in a second frequency band different from the first frequency band; and Wherein tuning the speaker for audio rendering comprises: The speaker is tuned for audio rendering within the second frequency band based at least on the difference between the real-time transfer function and the reference transfer function exceeding the threshold.

Images