Semi-adaptive beamformer

Abstract

To provide semi-adaptive beamforming methods, devices and systems.SOLUTION: A semi-adaptive beamformer 300 determines a relative transfer function (RTF) vector (instantaneous RTF vector 304) based on an audio signal 302 received through a microphone array, and further determines a minimum variance distortion-free response (MVDR) beamforming filter for the microphone array based on a composition of the instantaneous RTF vector and a fixed RTF vector 305. The fixed RTF vector may include a set of RTFs known to produce a relatively accurate MVDR beamforming filter for any user of the microphone array. Further, the semi-adaptive beamformer determines the MVDR beamforming filter based on a weighted average of the instantaneous RTF vector and the fixed RTF vector, where the weighting can be dynamically adjusted based on the quality of the received audio signal or various other conditions.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present implementation relates generally to signal processing, and more particularly to semi-adaptive beamformers for signal processing. [Background technology]

[0002] Beamforming is a signal processing technique that can focus the energy of transmitted and received signals in a spatial direction. For example, a beamformer can improve the quality of voice detected by a microphone array through signal synthesis at the microphone outputs. More specifically, when synthesizing audio signals, the beamformer may apply respective weights to the audio signals output by each microphone of the microphone array such that the signal strength is enhanced in the direction of voice (or suppressed in the direction of noise). An adaptive beamformer is capable of dynamically adjusting the weights of the microphone outputs to optimize the quality or signal-to-noise ratio (SNR) of the synthesized audio signal. In this way, an adaptive beamformer can adapt to changes in the environment. Examples of adaptive beamforming techniques include Minimum Mean Square Error (MMSE) beamforming, Minimum Variance Distortionless Response (MVDR) beamforming, and Generalized Eigenvalue (GEV) beamforming, among other examples.

[0003] The adaptive beamformer requires time to converge to an optimal set of weights. Before convergence, the adaptive beamformer may distort or even suppress the audio signal in the direction from which the voice is coming. Furthermore, in a low SNR environment, the adaptive beamformer may converge in a direction other than the direction of the voice (e.g., in the direction of a major noise source). Therefore, there is a need to reduce the delay required for the adaptive beamformer to converge while preventing the beamformer from converging in the wrong direction. Summary of the Invention

[0004] This Summary introduces selected concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to delineate key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0005] One innovative aspect of the subject matter in this disclosure may be implemented in a method of audio signal processing, the method including receiving an audio signal via a plurality of microphones, the audio signal including a plurality of frames, each having a respective speech component and a respective noise component, determining a plurality of first relative transfer functions (RTFs) for the plurality of microphones based on a first frame of the plurality of frames, respectively, and determining a first minimum variance distortionless response (MVDR) beamforming filter in the first frame based at least in part on the first plurality of RTFs, the plurality of fixed RTFs for the plurality of microphones, and a covariance of the noise component for the first frame, the first minimum variance distortionless response (MVDR) beamforming filter that reduces power of the noise component without distorting the speech component.

[0006] Another innovative aspect of the subject matter in this disclosure may be implemented in a beamformer that includes a processing system and a memory, where the memory stores instructions that, when executed by the processing system, cause the beamformer to receive an audio signal via a plurality of microphones, the audio signal including a plurality of frames each having a respective speech component and a respective noise component, determine a plurality of first RTFs for the plurality of microphones based on a first frame of the plurality of frames, respectively, and determine a first MVDR beamforming filter in the first frame that reduces power of the noise component without distorting the speech component based at least in part on the plurality of first RTFs, the plurality of fixed RTFs for the plurality of microphones, and a covariance of the noise component for the first frame.

[0007] Another innovative aspect of the subject matter in this disclosure may be implemented in a headset that includes multiple microphones and a beamformer, the beamformer configured to receive an audio signal via the multiple microphones, the audio signal including multiple frames each having a respective speech component and a respective noise component, determine multiple RTFs for the multiple microphones respectively based on a first frame of the multiple frames, and determine an MVDR beamforming filter in the first frame that reduces power of the noise component without distorting the speech component based at least in part on the multiple RTFs, the multiple fixed RTFs for the multiple microphones, and a covariance of the noise component for the first frame. [Brief description of the drawings]

[0008] The present implementations are illustrated by way of example and are not intended to be limited by the figures of the accompanying drawings.

[0009] [Figure 1] FIG. 1 illustrates an example of an environment in which beamforming may be implemented.

[0010] [Diagram 2] FIG. 2 illustrates an example of an audio receiver that can be configured for beamforming, according to some implementations.

[0011] [Diagram 3] FIG. 3 illustrates a block diagram of an example of a semi-adaptive beamformer according to some implementations.

[0012] [Figure 4] FIG. 4 illustrates another block diagram of an example of a semi-adaptive beamformer according to some implementations.

[0013] [Diagram 5] FIG. 5 shows an illustrative flowchart depicting an example of operations for processing an audio signal according to some implementations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] In the following description, numerous specific details are set forth, such as examples of specific components, circuits, processes, etc., to provide a thorough understanding of the details of the present disclosure. As used herein, the term "connected" means directly connected or connected through one or more intervening components or circuits. The terms "electronic system" and "electronic device" may be used interchangeably to refer to any system capable of electronically processing information. Also, in the following description, for purposes of explanation, specific names are set forth to provide a thorough understanding of aspects of the present disclosure. However, it will be apparent to one skilled in the art that these specific details are not required to practice the exemplary embodiments. In other instances, well-known circuits and devices are shown in block diagram form in order to avoid obscuring the disclosure. Some portions of the following detailed description are described in terms of procedures, logic blocks, processes, and other symbolic representations of operations on data bits within a computer memory.

[0015] These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. In this disclosure, a procedure, logic block, process, or the like, is intended to conceive of a self-consistent sequence of steps or instructions leading to a desired result. The steps are physical manipulations of physical quantities that they require. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It should be borne in mind, however, that all of these and similar terms can be associated with the appropriate physical quantities and are merely labels applied to these quantities.

[0016] As will become apparent from the discussion below, unless specifically stated otherwise, throughout this application, discussions utilizing terms such as "access," "receive," "send," "use," "select," "determine," "generalize," "multiply," "average," "monitor," "compare," "apply," "update," "measure," "drive," and the like can be recognized as relating to the implementation and operation of a computer system or similar electronic computing device that manipulates and converts data represented as physical (electronic) quantities in the computer system's registers and memory to other data similarly represented as physical quantities in the computer system's memory or registers, or other such information storage, communication, or display device.

[0017] In the figures, a block may be described as performing one or more functions, but in actual implementation, the functions performed by the block may be performed by one component or multiple components in total, and / or may be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability between hardware and software, various example components, blocks, modules, circuits, and steps are described below generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and specification constraints imposed on the overall system. Skilled artisans may implement the described functionality in various ways for each particular application, and such implementation decisions should not be interpreted as departing from the scope of the present disclosure. Additionally, examples of input devices may include components other than those depicted, including well-known components such as computing devices, memories, and the like.

[0018] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as implemented in a particular manner. Also, any features described as modules or components may similarly be implemented together as an integrated logical device or separately as separate but compatible logical devices. When implemented in software, these techniques may be realized, at least in part, by a non-transitory processor-readable storage medium that includes instructions that, when executed, perform one or more of the methods described above. The non-transitory processor-readable data storage medium may form part of a computer program product, which may include packaging materials.

[0019] The non-transitory processor-readable storage medium may comprise random access memory (RAM), such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, other known storage media, etc. The techniques may additionally or alternatively be embodied at least in part by a processor-readable communication medium that conveys or communicates code in the form of instructions or data structures and that is accessible, readable, and / or executable by a computer or other processor.

[0020] The illustrative logic blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors (or processing systems). As used herein, the term "processor" may refer to any general-purpose processor, special purpose processor, conventional processor, controller, microcontroller, and / or state machine capable of executing scripts or instructions of one or more software programs stored in memory.

[0021] As described above, a beamformer can improve the quality of the voice detected by the microphone array through the signal combining at the microphone outputs. For example, the beamformer may apply individual weights to the audio signals output by each microphone of the microphone array such that when the audio signals are combined, the signal strength is enhanced in the direction of the voice (or suppressed in the direction of the noise). An adaptive beamformer can dynamically adjust the weights of the microphone outputs to optimize the quality, or signal-to-noise ratio (SNR), of the combined audio signal. Examples of adaptive beamforming techniques include Minimum Mean Square Error (MMSE) beamforming, Minimum Variance Distortionless Response (MVDR) beamforming, Generalized Eigenvalue (GEV) beamforming, among other examples.

[0022] The MVRD beamformer determines a set of weights (also called an MVDR beamforming filter) that reduces or minimizes the noise component of a received audio signal without distorting the speech component. More specifically, the MVDR beamforming filter coefficients may be determined as a function of the covariance of the noise component of the received audio signal and a set of relative transfer functions (RTFs) (also called an RTF vector) between the microphones of the microphone array. In contrast, the GEV beamformer determines a set of weights (also called a GEV beamforming filter) that maximizes the SNR of the received audio signal. Through generalized eigenvalue decomposition, GEV beamforming may similarly determine an RTF vector associated with the microphone array.

[0023] The adaptive beamformer takes time to converge to an optimal set of weights. Before convergence, the adaptive beamformer may distort or even suppress the audio signal in the direction from which the voice is coming. Furthermore, in low SNR environments, the adaptive beamformer may converge in a direction other than the direction of the voice (e.g., in the direction of the main noise source). Aspects of the present disclosure accept that in some embodiments, the position of the microphone array may be fixed relative to the target audio source. For example, a microphone mounted on a headset may detect voice from approximately the same direction when the headset is worn by a user (or speaker). As a result, the RTF vector for a microphone array mounted on a headset may exhibit little (if any) change in response to audio signals received from other users.

[0024] Various aspects relate generally to beamforming, and more particularly to semi-adaptive beamforming techniques. In some aspects, the semi-adaptive beamformer may determine an RTF vector (also referred to as an "instantaneous" RTF vector) based on an audio signal received via a microphone array, and may further determine an MVDR beamforming filter for the microphone array based on a combination of the instantaneous RTF vector and a "fixed" RTF vector. The fixed RTF vector may include a set of RTFs that are known (or "trained to" generate) a relatively accurate MVDR beamforming filter for any user of the microphone array. In some implementations, the semi-adaptive beamformer may determine the MVDR beamforming filter coefficients based on a weighted average of the instantaneous RTF vector and the fixed RTF vector. This weighting may be dynamically adjusted based on the quality of the received audio signal or various other conditions. For example, the weighting may emphasize the instantaneous RTF vector when the SNR of the received audio signal is relatively high, and may emphasize the fixed RTF vector when the SNR of the received audio signal is relatively low.

[0025] Particular implementations of the subject matter described in this disclosure may be implemented to achieve one or more of the following possible advantages: The semi-adaptive beamformer of this implementation can rapidly converge to an appropriate set of weights while limiting convergence in the wrong direction. For example, by training a fixed RTF vector that can be applied to audio signals received from various users, aspects of the present disclosure can determine a relatively accurate starting point for initiating the adaptive beamforming procedure. By combining the fixed RTF vector with the instantaneous RTF vector, aspects of the present disclosure may further apply the beamforming procedure to a particular user in a controlled manner. For example, by emphasizing the instantaneous RTF vector when the SNR is low, the semi-adaptive beamformer can determine an MVDR beamforming filter that more accurately tracks the direction of the desired voice. In contrast, by emphasizing the fixed RTF vector when the SNR is high, the semi-adaptive beamformer prevents convergence in the direction of the main noise source.

[0026] 1 illustrates an example of an environment 100 in which beamforming may be implemented. The example environment 100 includes a headset 110 and a user 120. In some aspects, the headset 110 may include multiple microphones 112-116 (also referred to as a "microphone array"). In the example of FIG. 1, the headset 110 is depicted as including three microphones 112-116. However, in some other implementations, the headset 110 may include fewer or more microphones than those depicted in FIG. 1.

[0027] The microphones 112-116 are positioned or otherwise configured to detect sound 122 (detected as a series of acoustic waves) propagating from the mouth of the user 120. For example, each microphone 112-116 may convert the detected sound 122 into an electronic signal (also referred to as an "audio signal") that represents an acoustic waveform. Each audio signal may include a voice component (representing the user's voice 122) and a noise component (representing noise from the headset 110 and the surrounding environment). Due to the spatial location of the microphones 112-116, the sound 122 detected by some of the microphones in the microphone array may be delayed relative to the sound 122 detected by some of the other microphones in the microphone array. In other words, the microphones 112-116 may generate multiple audio signals with different phase offsets.

[0028] In some aspects, the audio signals generated by each of the microphones 112-116 may be weighted and combined to emphasize speech components or suppress noise components. More specifically, the weights applied to the audio signals may be configured to enhance the signal strength in the direction of the voice 122. Such a signal processing technique is called "beamforming". In some implementations, an adaptive beamformer may estimate (or predict) a set of weights (also called a "beamforming filter") that may be applied to the audio signals to enhance the signal strength in the direction of the voice. The quality of the voice in the resulting signal depends on the accuracy of the beamforming filter coefficients. For example, the voice may be emphasized when the beamforming filter is adjusted in the direction of the user's mouth. On the other hand, the voice may be distorted or suppressed when the beamforming filter is adjusted in the direction of a noise source.

[0029] An adaptive beamformer can dynamically adjust beamforming filter coefficients to optimize the quality, or signal-to-noise ratio (SNR), of the synthesized audio signal. Examples of adaptive beamforming techniques include minimum variance distortion-free response (MVDR) beamforming and generalized eigenvalue (GEV) beamforming, among others. The MVDR beamformer determines a beamforming filter that reduces or minimizes the noise component of the audio signal without distorting the speech component. MVDR beamforming assumes that a delay-only propagation path exists between the microphones 112-116 and the audio source. However, in a microphone array mounted on a headset, the audio signals generated by the microphones 112-116 may include acoustic background noise from reverberating walls or the headset 110 enclosure. Such reverberations may lead to the elimination of important speech by the MVDR beamformer.

[0030] In contrast, the GEV beamformer determines a beamforming filter that maximizes the signal-to-noise ratio (SNR) of the audio signal. More specifically, GEV beamforming adaptively derives a dominant stationary vector that incorporates the cross power spectral density matrix of the speech-noise and noise-only components of the audio signal generated by the microphones 112-116. This adaptive algorithm does not require any information about the positions of the microphones 112-116 or the audio sources. However, the algorithm requires time to converge to an optimal set of filter coefficients. Before convergence, the GEV beamforming filter may distort or even suppress the audio signal in the direction from which the speech is coming. Furthermore, in low SNR environments, the GEV beamformer may converge in a direction other than the direction of the speech (such as the direction of the dominant noise source).

[0031] In some aspects, the headset 110 may include a semi-adaptive beamformer (not shown for simplicity) that can rapidly converge to an optimal beamforming filter while limiting convergence in the wrong direction. In some implementations, the semi-adaptive beamformer may leverage the known ability of the headset 110 to determine relatively accurate beamforming filters for various users. For example, as shown in FIG. 1, the headset 110 is designed to be worn on the head of a user. More specifically, the headset 110 includes a pair of ear cups designed to cover the ears of the user 120, and the microphones 112-116 are provided in the ear cups of the headset 110. Aspects of the present disclosure accommodate that for many intended users of the headset 110, the distance between the ears and the mouth is substantially the same. As a result, the semi-adaptive beamformer may determine the beamforming filter based at least in part on a priori information of the relative positions of the microphones 112-116 and the mouth of the user 120.

[0032] 2 illustrates an example of an audio receiver 200 that may be configured for beamforming in some implementations. The audio receiver 200 includes several (M) microphones 210(1)-210(M) arranged in a microphone array and a beamforming filter 220. In some implementations, the audio receiver 200 may be an example of the headset 110 of FIG. 1. For example, referring to FIG. 1, each of the microphones 210(1)-210(M) may be an example of one of the microphones 112-116.

[0033] The microphones 210(1)-210(M) are configured to convert a series of sound waves 201 (also referred to as "acoustic waves") into audio signals 202(1)-202(M), respectively. As shown in FIG. 2, the sound waves 201 are incident on the microphones 210(1)-210(M) at an angle (θ). In some implementations, the sound waves 201 may include a user's voice (such as the voice 122 in FIG. 1) mixed with noise or interference (such as noise bouncing off the walls of a headset). Thus, each audio signal 202(1)-202(M) may include a voice component (s) and a noise component (u). Due to the spatial location of the microphones 210(1)-210(M), each audio signal 202(1)-202(M) may represent a delayed version of the same audio signal. For example, the first audio signal 202(1) may be used as a reference audio signal, and each of the other audio signals 202(2)-202(M) may be represented as a phase-delayed version of the first audio signal 202(1). Thus, the audio signals 202(1)-202(M) may be modeled as a vector (y). TIFF2024066473000002.tif35145 where l is a frame index that represents one of the number of audio frames (L), k is a frequency index that represents one of the number of frequency ranges (K), and a(θ, k) is a steering vector that represents a set of phase delays due to sound wave 201 incident on microphones 210(1)-210(M).

[0034] The beamforming filter 220 is a weight vector w=[w1,...,w M ] T (where w1-w M are referred to as filter coefficients) are applied to the audio signals 202(1)-202(M) to generate weighted audio signals 204(1)-204(M), respectively. The weighted audio signals 204(1)-204(M) are further combined (such as by adding) to generate the output audio signal 206. Thus, the output audio signal 206 may be modeled as a vector (S). TIFF2024066473000003.tif24161 where w represents a beamforming filter (or vector) 220. In some aspects, a beamformer (not shown for simplicity) may determine a weight vector w that optimizes the output audio signal 206 for one or more states.

[0035] For example, the MVDR beamformer is configured to determine a weighting vector w that reduces or minimizes various noise components of the output audio signal 206 without distorting the speech components of the output audio signal 206. In other words, the weighting vector w may satisfy the following condition: TIFF2024066473000004.tif22142 where R u (k) is the covariance of the noise component u(l,k) of the received audio signal y(l,k). The resulting weight vector w is the MVDR forming filter (w MVDR (k)). TIFF2024066473000005.tif47161

[0036] On the other hand, the GEV beamformer (also called a “maximizing SNR beamformer”) is configured to determine a weighting vector w that increases or maximizes the SNR of the output audio signal 206. For example, the SNR is determined by the noise component R u The covariance of (k) and the speech component (R s (k)) as a function of the covariance of TIFF2024066473000006.tif32142 where R y (k) is the covariance of the received audio signal y(l,k). The resulting weight vector w is u -1 (k)R y (k) Principal fixed vector (v max (k)) and the GEV beamforming filter (w GEV (k)).

[0037] Through generalized eigenvalue decomposition, the GEV beamformer can determine a relative transfer function (RTF) between each microphone 210(1)-210(M) in the microphone array and a reference microphone (such as the first microphone 210(1). For example, the RTF can be modeled as an RTF vector (h(k)): TIFF2024066473000007.tif60133Here, (R y (k)w GEV (k))1 is R y (k)w GEV It is the first element of (k).

[0038] As described above, the GEV beamformer may adaptively determine the weight vector w based on the received audio signal y(l,k). However, the GEV beamformer takes time to converge to an appropriate weight vector w, and may converge in the wrong direction when the SNR of the audio signal y(l,k) is very low. In contrast, the MVDR beamformer generally relies on the shape (such as the steering vector a(θ,k)) to determine the weight vector w. Although the MVDR beamformer does not take time to converge, the MVDR beamforming filter w MVDR The accuracy of (k) depends on the accuracy of the estimation of the steering vector a(θ,k), which may be difficult to adapt to different users. Aspects of the present disclosure accept that the steering vector a(θ,k) may also be defined as the RTF vector h(k). Thus, using a(θ,k) instead of h(k) in equation (3) yields the following equation: TIFF2024066473000008.tif58167

[0039] In some embodiments, the semi-adaptive beamformer uses a fixed RTF vector h * Based at least in part on (k), a weight vector w for the beamforming filter 220 may be determined. In some implementations, the fixed RTF vector h *(k) may be learned as part of a training operation based on audio signals 202(1)-202(M) received via microphones 210(1)-210(M). Thus, such audio signals may also be referred to as "training signals." For example, the training signals may represent speech from one or more known users detected by microphones 210(1)-210(M). In some implementations, the fixed RTF vector h * (k) may be determined using GEV beamforming (such as according to equation (4)). For example, the GEV beamformer may determine one or more RTF vectors h(k) based on each of the received training signals, and may use a fixed RTF vector h * (k) may be determined as the average of the RTF vector h(k). As a result, the fixed RTF vector h * (k) may be generally adjusted to suit different users of the audio receiver 200 .

[0040] But the fixed RTF vector h * (k) may not be optimized for a particular user of the audio receiver 200. For example, referring to FIG. 1, each user 120 of the headset 100 may have a unique head shape and head size, resulting in differences in the optimal RTF vector for each user. In some aspects, the semi-adaptive beamformer uses filter weights w1-w M The RTF vector used to determine h may be fine-tuned, for example, to adapt the beamforming filter 220 to the actual user of the audio receiver 200. In some implementations, the semi-adaptive beamformer may determine the instantaneous RTF vector ĥ(k) based on the audio signals 202(1)-202(M) received from the current user of the audio receiver 200, and may use the fixed RTF vector h * (k) and the instantaneous RTF vector h^(k) based on the filter weights w1-w M For example, the semi-adaptive beamformer may further determine the filter weights w1-w Mwhere h(k) is a combination of h*(k) and ĥ(k).

[0041] FIG. 3 illustrates a block diagram of an example of a semi-adaptive beamformer 300 in some implementations. The semi-adaptive beamformer 300 is configured to determine a beamforming (BF) filter 308 based on an audio signal 302 received via a microphone array. In some implementations, the microphone array may include any of the microphones 112-116 of FIG. 1 or any of the microphones 210(1)-210(M) of FIG. 2. For example, referring to FIG. 2, the audio signal 302 may be an example of an audio signal y(l,k) (including audio signals 202(1)-202(M), respectively) received via the microphones 210(1)-210(M), and the beamforming filter 308 may be an example of the beamforming filter 220.

[0042] The semi-adaptive beamformer 300 includes a GEV beamforming component 310, a dynamic RTF adjustment component 320, and an MVDR beamforming component 330. The GEV beamforming component 310 is configured to generate a respective instantaneous RTF vector 304 based on each frame of a received audio signal 302. For example, the GEV beamforming component 310 may select a GEV beamforming filter w that maximizes the SNR of the audio signal 302 (as described with reference to FIG. 2). GEV (For simplicity, the frequency index k is omitted below). The GEV beamforming component 310 may further map the instantaneous RTF vector 304 to the GEV beamforming filter w GEV and covariance (R y ) may be determined as a function of

[0043] The dynamic RTF adjustment component 320 is configured to generate a composite RTF vector 306 based on the instantaneous RTF vector 304 and the fixed RTF vector 305. For example, the fixed RTF vector 305 may include a set of RTFs determined to reasonably fit different users of the microphone array (such as as part of a training procedure). In some implementations, the semi-adaptive beamformer 300 may learn the fixed RTF vector 305 based on audio signals (or training signals) previously received via the microphone array (as described with reference to FIG. 2). In some aspects, the dynamic RTF adjustment component 320 generates a composite RTF vector 306 (h l ) for the l-th frame as the instantaneous RTF vector 304(h^ l ) and fixed RTF vector 305(h * ) may be determined as a weighted average. TIFF2024066473000009.tif43161 where μ l is the correlation coefficient for the lth frame of the audio signal 302.

[0044] In some implementations, the dynamic RTF adjustment component 320 may be configured to calculate the correlation coefficient μ l Dynamically adjust the instantaneous RTF vector h^ l or a fixed RTF vector h * Either of the two may be emphasized. For example, (μ l >0.5) l is the instantaneous RTF vector h^ l Compared with the fixed RTF vector h * can be emphasized, whereas (μ l A lower correlation coefficient μ (such as <0.5) l is a fixed RTF vector h * Compared with the instantaneous RTF vector h^ l In some embodiments, the dynamic RTF adjustment component 320 may emphasize the fixed RTF vector h *a correlation coefficient μ based at least in part on the amount of movement (relative to the user's mouth position) of one or more microphones in the microphone array compared to a "default" position of the microphone in l For example, the correlation coefficient μ l can be expressed as follows: TIFF2024066473000010.tif55164 where F, less than or equal to K, is the number of frequency bins used for averaging in equation (7). As shown in equation (7), the fixed RTF vector h * is the instantaneous RTF vector h^ l When there is a high correlation between and (most of the frequency range is above 0 and below F-1), the correlation coefficient μ l is higher (closer to 1).

[0045] In some other aspects, the dynamic RTF adjustment component 320 may be configured to adjust the correlation coefficient μ based at least in part on the SNR of the audio signal 302. l As described with reference to FIG. 2, the GEV beamforming component 310 may dynamically adjust the SNR 307 for the audio signal 302 using the GEV beamforming filter w GEV As part of the procedure for determining, the dynamic RTF adjustment component 320 may receive the SNR 307 from the GEV beamforming component 310. In some implementations, the dynamic RTF adjustment component 320 may determine a correlation coefficient μ that is lower when the SNR 307 is relatively high (to better adapt the microphone array to the current user). l In some other implementations, the dynamic RTF adjustment component 320 may select a higher correlation coefficient μ when the SNR 307 is relatively low (to prevent the composite RTF vector 306 from converging in the wrong direction). l may be selected.

[0046] The MVDR beamforming component 330 is configured to generate a beamforming filter 308 based on the received audio signal 302 and the composite RTF vector 306. More specifically, the MVDR beamforming component 330 generates an MVDR beamforming filter w that reduces or minimizes the power of the noise components without distorting the speech components in the l-th frame of the received audio signal 302 (as described with reference to FIG. 2). MVDR More specifically, the MVDR beamforming filter (w MVDR,l ) in equation (4), h (from equation (6)) l can be determined by substituting TIFF2024066473000011.tif59156 where R u,l is the covariance of the noise component in the l-th frame of the received audio signal 302. The resulting MVDR beamforming filter w MVDR,l contains a weight vector w that can be used to weight the audio signals received via each microphone of a microphone array (such as audio signals 202(1)-202(M) in FIG. 2).

[0047] As described above, the dynamic RTF adjustment component 320 calculates a respective correlation coefficient μ for each frame of the received audio signal 302. l (Thus, each composite RTF vector h l For example, if more noise is detected in the l-th frame of the audio signal 302 than in the (l-1)-th frame, the dynamic RTF adjustment component 320 may determine the composite RTF vector h l In the fixed RTF vector h * is the instantaneous RTF vector h^ l To be weighted more heavily, (μ l >μ l-1 (at which point) the correlation coefficient μ lOn the other hand, if less noise is detected in the l-th frame of the audio signal 302 than in the (l-1)-th frame, the dynamic RTF adjustment component 320 may increase the synthetic RTF vector h l In the instantaneous RTF vector h^ l is a fixed RTF vector h * To be weighted more heavily, (μ l <μ l-1 (at which point) the correlation coefficient μ l As a result, the semi-adaptive beamformer 300 may dynamically adjust the beamforming filter 308 on a frame-by-frame basis such that the weighting vector w adapts to real-time changes in the user's mouth position, the position of one or more microphones, or the SNR of the received audio signal 302.

[0048] 4 illustrates another block diagram of an example of a semi-adaptive beamformer 400 according to some implementations. More specifically, the semi-adaptive beamformer 400 may determine a beamforming filter based on audio signals received via a microphone array. In some implementations, the semi-adaptive beamformer 400 may be an example of the semi-adaptive beamformer 300 of FIG. 3. The semi-adaptive beamformer 400 includes a device interface 410, a processing system 420, and a memory 430.

[0049] The device interface 410 is configured to communicate with one or more components of an audio receiver (such as the audio receiver 200 of FIG. 2). In some implementations, the device interface 410 may include a microphone interface (I / F) 412 configured to receive an audio signal via multiple microphones in a microphone array and apply a beamforming filter (including a set of filter coefficients) to the output of each of the multiple microphones. In some implementations, the received audio signal may be further divided temporally into multiple frames, each having a respective speech component and a respective noise component.

[0050] The memory 430 may include an RTF data store 432 configured to store fixed RTF vectors for the microphone array. For example, the fixed RTF vectors may include a set of RTFs determined to reasonably fit different users of the microphone array (such as as part of a training procedure). The memory 430 may also include a non-transitory computer-readable medium (including one or more non-volatile memory elements such as EPROM, EEPROM, flash memory, or hard drives, among other examples) that may store at least the following software (SW) modules: an RTF adaptation SW module 434 for determining an RTF vector based on a first frame of the plurality of frames, the RTF vector including a plurality of RTFs for each of the plurality of microphones and a reference microphone of the plurality of microphones; a beamforming SW module 436 for determining an MVDR beamforming filter that reduces the power of the noise component of the first frame without distorting the speech component of the first frame based at least in part on the RTF vector, the fixed RTF vector, and the covariance of the noise component of the first frame; Each software module contains instructions that, when executed by the processing system 420, cause the semi-adaptive beamformer 400 to perform a corresponding function.

[0051] The processing system 420 may include one or more suitable processes capable of executing one or more software program scripts or instructions stored in the semi-adaptive beamformer 400 (e.g., memory 430). For example, the processing system 420 may execute an RTF adaptation SW module 434 to determine an RTF vector based on a first frame of the multiple frames, the RTF vector including multiple RTFs for each of the multiple microphones and a reference microphone of the multiple microphones. Additionally, the processing system 420 may also execute a beamforming SW module 436 to determine an MVDR beamforming filter that reduces the power of the noise component of the first frame without distorting the speech component of the first frame based at least in part on the RTF vector, the fixed RTF vector, and the covariance of the noise component of the first frame.

[0052] 5 illustrates an exemplary flow chart illustrating an example of a procedure 500 for processing an audio signal, according to some implementations. In some implementations, the example of the procedure 500 may be performed by a beamformer, such as any of the semi-adaptive beamformers 300, 400 of FIGS. 3 and 4, respectively.

[0053] The beamformer may receive an audio signal via a plurality of microphones, the audio signal including a plurality of frames each having a respective speech component and a respective noise component (510). The beamformer may determine a plurality of first relative transfer functions (RTFs) for the plurality of microphones, respectively, based on a first frame of the plurality of frames (520). The beamformer may further determine a first MVDR beamforming filter that reduces power of the noise component in the first frame without distorting the speech component, based at least in part on the plurality of first RTFs, the plurality of fixed RTFs for the plurality of microphones, and a covariance of the noise component for the first frame (530).

[0054] In some aspects, the beamformer may further receive a training signal having a speech component and a noise component via the multiple microphones, determine a GEV beamforming filter that increases the SNR with respect to a covariance of the speech component of the training signal and a covariance of the noise component of the training signal, and determine multiple fixed RTFs based at least in part on the GEV beamforming filter.

[0055] In some aspects, determining the plurality of first RTFs may include determining a first GEV beamforming filter that increases the SNR with respect to a covariance of the speech component of the first frame and a covariance of the noise component of the first frame.

[0056] In some aspects, the beamformer may further determine the first composite RTFs based on the fixed RTFs, the first RTFs, and the first correlation coefficient. In some implementations, the beamformer may determine the first correlation coefficient based at least in part on the correlation between the fixed RTFs and the first RTFs. In some other implementations, the beamformer may determine the first correlation coefficient based at least in part on the SNR related to the covariance of the speech components of the first frame.

[0057] In some aspects, the beamformer may further determine a plurality of second RTFs for the plurality of microphones based on a second frame of the plurality of frames, determine a plurality of second synthetic RTFs based on the plurality of fixed RTFs, the plurality of second RTFs, and a second correlation coefficient, and determine a second MVDR beamforming filter based on the plurality of second synthetic RTFs and a covariance of the noise component of the second frame.

[0058] In some aspects, determining the second plurality of RTFs may include determining a second GEV beamforming filter that increases an SNR for a covariance of the speech component of the second frame and a covariance of the noise component of the second frame. In some implementations, the SNR for the covariance of the speech component of the second frame may be higher than the SNR for the covariance of the speech component of the first frame, and the second correlation coefficient may be lower than the first correlation coefficient. In some other implementations, the SNR for the covariance of the speech component of the second frame may be lower than the SNR for the covariance of the speech component of the first frame, and the second correlation coefficient may be higher than the first correlation coefficient.

[0059] Those skilled in the art will recognize that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields, magnetic particles, optical fields, photons, or combinations thereof.

[0060] Moreover, those skilled in the art will recognize that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the particular application and specification constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0061] The methods, sequences or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An example of a storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor.

[0062] In the foregoing specification, embodiments have been described with reference to specific examples thereof. It will be apparent, however, that various modifications and changes may be made thereto without departing from the broader scope of the present disclosure as set forth in the appended claims. The specification and drawings are, consequently, to be regarded in an illustrative rather than a restrictive sense.

Claims

1. The audio signal is received via multiple microphones, and the audio signal includes multiple frames, each having its own voice component and its own noise component. Based on the first frame of the plurality of frames, a plurality of first relative transfer functions (RTFs) for each of the plurality of microphones are determined, Based on the plurality of first RTFs, the plurality of fixed RTFs relating to the plurality of microphones, and at least a portion of the covariance of the noise component in the first frame, a first minimum dispersion distortion-free response (MVDR) beamforming filter is determined that reduces the power of the noise component without distorting the audio component in the first frame. A method for processing audio signals that include [specific components / signals].

2. The process involves receiving a training signal having audio components and noise components via the aforementioned multiple microphones, To determine a generalized eigenvalue (GEV) beamforming filter that increases the signal-to-noise ratio (SNR) with respect to the covariance of the audio component of the training signal and the covariance of the noise component of the training signal, Based on at least a portion of the GEV beamforming filter, the plurality of fixed RTFs are determined, The method of claim 1, further comprising:

3. Determining the aforementioned plurality of first RTFs is To determine a first GEV beamforming filter that increases the SNR with respect to the covariance of the audio component of the first frame and the covariance of the noise component of the first frame. The method of claim 1, including the method of claim 1.

4. Based on the aforementioned plurality of fixed RTFs, the plurality of first RTFs, and the first correlation coefficient, a plurality of first combined RTFs are determined. The method of claim 3, further comprising:

5. The first correlation coefficient is determined based on at least a portion of the correlation between the plurality of fixed RTFs and the plurality of first RTFs. The method of claim 4, further comprising:

6. The first correlation coefficient is determined based on at least a portion of the SNR relating to the covariance of the audio components of the first frame. The method of claim 4, further comprising:

7. Based on the second frame of the aforementioned plurality of frames, a plurality of second RTFs for each of the plurality of microphones are determined, Based on the aforementioned multiple fixed RTFs, the aforementioned multiple second RTFs, and the second correlation coefficient, multiple second combined RTFs are determined. Based on the plurality of second combined RTFs and the covariance of the noise components of the second frame, a second MVDR beamforming filter is determined. The method of claim 4, further comprising:

8. Determining the aforementioned multiple second RTFs is To determine a second GEV beamforming filter that increases the SNR with respect to the covariance of the audio component of the second frame and the covariance of the noise component of the second frame. The method of claim 7, including the method of claim 7.

9. Processing system and, When executed by the aforementioned processing system, The audio signal is received via multiple microphones, and the audio signal contains multiple frames, each having its own voice component and its own noise component. Based on the first frame of the plurality of frames, a plurality of first relative transfer functions (RTFs) for each of the plurality of microphones are determined, Based on the plurality of first RTFs, the plurality of fixed RTFs relating to the plurality of microphones, and the covariance and at least a portion of the noise components of the first frame, a first minimum dispersion distortion-free response (MVDR) beamforming filter is determined that reduces the power of the noise components without distorting the audio components in the first frame. Memory to store instructions to execute, A beamformer equipped with this feature.

10. Multiple microphones, It is a beamformer, The system receives audio signals via multiple microphones, and the audio signals include multiple frames, each having its own voice component and its own noise component. Based on the first frame of the plurality of frames, a relative transfer function (RTF) vector is determined, and the RTF vector includes a plurality of RTFs relating to each of the plurality of microphones and the reference microphone of the plurality of microphones. Based on the determined RTF vector, the fixed RTF vector, and at least a portion of the covariance of the noise component in the first frame, a minimum variance-free response (MVDR) beamforming filter is determined that reduces the power of the noise component in the first frame without distorting the audio component in the first frame. The beamformer configured, A headset equipped with [features / equipment].

Images