System and method for covariance smoothing

Abstract

To provide a system and method for covariance smoothing.SOLUTION: A method and system improve signal processing by smoothing a covariance matrix of a multi-channel signal by setting a forgetting factor based on the bins of a band. The method and system for resetting smoothing based on transient detection are also disclosed. The method and system for resampling for smoothing between band transitions are also disclosed.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 62 / 881,825, filed August 1, 2019, and U.S. Provisional Patent Application No. 63 / 057,533, filed July 28, 2020, which are incorporated by reference.

[0002] [Technical field] The present disclosure relates to improvements for processing signals, in particular to processing audio signals to improve covariance smoothing for improved processing. [Background technology]

[0003] One aspect of audio signal processing involves presenting multi-channel audio to a listener so that the listener can determine a virtual spatial location for the audio, giving the listener an immersive experience. The earliest implementation of this was stereo, where the listener can spatially determine the "direction" from which a sound source is coming.

[0004] Recent developments in this technology use the inter-channel dependencies of multi-channel systems to present a more immersive sound experience, which may involve the use of covariance matrices of the audio channels. Summary of the Invention

[0005] Various signal processing systems and methods are disclosed herein, some of which may involve smoothing the covariance values ​​of bands across successive frames.

[0006] In some examples, a system and method for smoothing an estimate of a covariance matrix for a sequence of signal frames in a frequency band includes comparing, for the signal frames, an effective count of bins in the frequency band to a desired count of bins for the frequency band, calculating a forgetting factor for the band as a ratio of the effective count to the desired count, and if the effective count of bins in the frequency band is less than the desired count, generating a current estimate of a value of the covariance matrix value for the current frame using the forgetting factor and a previously generated estimate of a value of the covariance matrix value for a previous frame relative to the current frame, the comparing, calculating and generating steps being performed by a system including one or more computer processors. The smoothed covariance matrix can be used to improve further processing of the signal by reducing artifacts caused by rough transitions in the matrix.

[0007] In some such examples, the systems and methods may include calculating the effective number of bins as a sum of filter bank response values ​​for the frequency band.

[0008] In some such examples, the generating step uses a first order filter.

[0009] In some such examples, the primary filter includes the difference between a value for the current frame and a previously generated estimate for a previous frame, the difference being weighted by a forgetting factor.

[0010] In some such examples, the system and method also include comparing the forgetting factor to a maximum forgetting factor and setting the forgetting factor to the minimum of the calculated forgetting factor and the maximum allowed forgetting factor, the comparing and setting steps being performed prior to generating a current estimate of the value of the covariance matrix for the current frame.

[0011] In some such examples, the systems and methods also include detecting whether a transient has occurred in the monitored frame, and in response to detecting that a transient has occurred, resetting the smoothing by setting a current estimate of the covariance matrix value for the monitored frame to an original value of the covariance matrix value for the monitored frame without using a forgetting factor for the monitored frame. The detecting step can be performed by using aspects of a ducking decorrelator.

[0012] In some such examples, the systems and methods include storing a current estimate of a covariance matrix for the current frame and resampling the current estimate across frequency bands to convert from one banding structure to another.

[0013] The system and method may be part of an audio signal decoder.

[0014] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including, but not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, and the like. Thus, various inventive aspects of the subject matter described in this disclosure may be implemented in a non-transitory medium having software stored thereon. The software may be executable, for example, by one or more components of a control system as disclosed herein. The software may include, for example, instructions for performing one or more of the methods disclosed herein.

[0015] At least some aspects of the present disclosure may be implemented via an apparatus or multiple apparatuses. For example, one or more devices may be configured to at least partially perform the methods disclosed herein. In some implementations, the apparatus may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between the control system and a memory system, one or more interfaces between the control system and other devices, and / or one or more external device interfaces. The control system may include at least one of a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. Thus, in some implementations, the control system may include one or more processors and one or more non-transitory storage media operably coupled to the one or more processors.

[0016] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. It is noted that the relative dimensions of the following drawings may not be drawn to scale. Like reference numbers and names in the various drawings generally indicate similar elements, but different reference numbers do not necessarily indicate different elements between different drawings. [Brief description of the drawings]

[0017] [Figure 1] 1 shows an example of a signal channel separated into samples, frames, bins and bands. [Diagram 2]1 shows a flowchart of an exemplary method for smoothing a signal covariance matrix using a smoothing algorithm. [Diagram 3] 1 shows a flowchart (part) of an exemplary method for smoothing a signal covariance matrix using a smoothing algorithm with a maximum forgetting factor. [Figure 4] 1 shows a flowchart (part) of an example method for smoothing a signal covariance matrix using a smoothing algorithm with transient detection. [Diagram 5] 1 illustrates an exemplary encoder / decoder system having a smoothing algorithm in the encoder. [Figure 6] 1 illustrates an exemplary encoder / decoder system with a smoothing algorithm in the decoder. [Figure 7] 1 illustrates an exemplary encoder / decoder system having a smoothing algorithm as part of the DTX. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] This disclosure describes methods and systems for smoothing the values ​​of an estimation matrix (e.g., a covariance matrix) over time, including when the estimates are calculated from a small number of samples, where unsmoothed fluctuations in the values ​​of the estimation matrix can cause problems such as noticeable artifacts in the output (e.g., audio).

[0019] This disclosure also describes (among other things) the use of transient detection to prevent over-smoothing of values, thereby adding undesirable artifacts from the smoothing.

[0020] As used herein, the term "smoothing" refers to reducing the variation in values ​​of successive frames of a signal, for example reducing the variation in covariance values ​​across successive frames.

[0021] The term "covariance" as used herein refers to a measure of the joint variation of two signals, for example the covariance of the energies of the signals.

[0022] As used herein, a "covariance matrix" refers to a matrix of covariances of multiple signals. A "banded covariance matrix" refers to a covariance matrix that groups several adjacent frequency bins of a bin-by-bin covariance matrix into bands, such that the covariance can be expressed in a significantly smaller number (equal to the number of bands). For example, an input with 960 frequency bins can result in 12 frequency bands with the 960 bins uniformly or non-uniformly distributed among the 12 bands. For the examples provided herein, the "covariance matrix" is considered to be a banded covariance matrix.

[0023] As used herein, an "estimated matrix" refers to a sample covariance matrix. As used herein, the terms "covariance matrix" and "smoothed matrix" include estimated matrices.

[0024] The term "complex advanced coupling (CACPL)" as used herein refers to a method of stereo coding in which the left and right signals are combined into mid (L+R) / 2 and side (LR) / 2 signals, and the side signal is then predicted from the mid signal.

[0025] As used herein, the term "smoothing algorithm" refers to smoothing the covariance matrix in accordance with the present disclosure, and in particular, smoothing based on the effective bin counts of the bands. As used herein, the term "smoothed matrix" refers to the covariance matrix as modified by the smoothing algorithm.

[0026] As used herein, the term "effective bin count" refers to either an actual count of the number of bins in a band or an estimate of the effective bin count in a band. An example of an estimate of the effective bin count in a band includes summing the filter bank response values ​​for that band.

[0027] As used herein, the term "forgetting factor" refers to a weighting value that modifies the importance of a part of a function.

[0028] As used herein, "DTX" refers to an encoder module that processes discrete transmission events.

[0029] This document describes a solution to the problem of smoothing the covariance of input channels, which improves signal processing techniques, e.g., for voice and audio services, such as those that provide immersive experiences to users (e.g., virtual reality, augmented reality).

[0030] In digital signal processing as shown in Figure 1, the input signal (channels) is divided into frames 101 in the time domain. A sequence of consecutive frames can be grouped together into a "frame sequence" 102. In the frequency domain, the signal can be divided into frequency intervals called bins 103. A given bin in a given frame is called a sample 106. These bins can be grouped in frequency bands 104 / 105. This allows statistical averaging over a given band for a given window frame / frame sequence (a "window" is a weighting function applied to a frame or frame sequence). One application of this averaging is to estimate the covariance between audio channels.

[0031] Not all bands necessarily contain the same number of bins. For example, the lower frequency band 104 may have fewer bins per band than the higher frequency band 105. This means that for statistical analysis, the lower frequency bands will require a larger window (more frames) per band than the higher frequency bands do to generate meaningful statistical results. A given bin in a given frame may have signal energy associated with it, and some bin values ​​may be averaged across bands and / or windows.

[0032] [Example of covariance matrix] Taking the complex advanced coupling (CACPL) as an example, the parameters α (complex prediction coefficient) and β (real inverse correlation coefficient) are determined by the (2 × 2) mid-side covariance matrix Σ MS This matrix is ​​computed over a set of analysis bands 1≦b≦B.

[0033]

number

[0034] [Smoothing function and forgetting factor] In general, the smoothing matrix can be calculated using a low pass filter designed to meet the smoothing requirements. For a smoothing algorithm, the smoothing requirements are such that previous estimates are used to artificially increase the number of frequency samples (bins) used to generate the current estimate of the covariance matrix. In some embodiments, the smoothing matrix A is calculated from the input covariance matrix A over a sequence of frames.

[0035]

number

[0036]

number

[0037] [Forgetting factor] The effect of the forgetting factor is that as smoothing is applied over successive frames, the effect of previous frames becomes less and less influential on the smoothing of the frame being smoothed (adjusted).

[0038] When the forgetting factor in Equation 1 is 1 (λ=1), no smoothing occurs and it effectively acts as an all-pass filter. When 0<λ<1, the equation acts as a low-pass filter. Lower λ places more weight on older covariance data, while higher λ considers newer covariances more. A forgetting factor greater than 1 (e.g., 1<λ<2) performs as a high-pass filter. In some embodiments, the maximum allowable forgetting factor λ max is implemented. This maximum determines the behavior of the algorithm as the bin / band values ​​get larger. In some embodiments, λ is the maximum regardless of what the calculated forgetting factor is. max < 1 means that some smoothing is always performed in each band, λ max = 1 smooths the function to the desired N min It is applied only to bands with bins smaller than , leaving larger bands unsmoothed.

[0039] In some of these embodiments, the specific band λ b The forgetting factor for is the maximum allowable forgetting factor λ max and the effective number of bins in the band, N, determined to give good statistical estimates based on the window size. b and the minimum number of bins N min It is calculated as the minimum of the ratios of

[0040]

number

[0041] In one example, an analysis filter bank is used that has a narrower (i.e., fewer bins, more frames required for good statistical analysis) low frequency band and a wider (i.e., more bins, fewer frames required for good statistical analysis) high frequency band, which increases the amount of smoothing in the low frequency band and decreases the amount of smoothing in the high frequency band (or reduces λ max = 1, there is no smoothing at all).

[0042] An exemplary process flow for this method is shown in FIG. 2. An input signal FFT (Fast Fourier Transform) 201 provides, for a given band in the input signal, a corresponding covariance matrix over a window. The effective count of bins for that band is obtained 202. This can be calculated, for example, by the filter bank response value of the band. A desired bin count is determined 203, for example, by a subjective analysis of the number of bins needed to provide a good statistical analysis for the window. A forgetting factor is calculated 204 by taking the ratio of the calculated number of bins to the desired bin count. For a given frame (except the first frame), new covariance matrix values ​​are calculated 205 based on the new covariance values ​​calculated for the previous frame, the original values ​​for the current frame, and the forgetting factor. The new (smoothed) matrix formed by these new values ​​is used in further signal processing 206.

[0043] Figure 3 shows an example modification to the flow for maximum allowed forgetting factor. As in Figure 2, forgetting factors are calculated for the bands 301. Furthermore, a maximum allowed forgetting factor is determined 302. The values ​​are compared 303 and depending on whether the calculated coefficient is less than the maximum allowed coefficient, the calculated coefficient is used in the smoothing 305. If the calculated coefficient is greater than the maximum allowed coefficient, the maximum allowed coefficient is used in the smoothing 305. Although this example shows that if the coefficients are equal (but not greater), an equivalent flow can be envisaged in which if they are equal, the minimum value is used.

[0044] [Reset Smoothing] In some embodiments, it may be desirable to avoid smoothing on transients (sudden changes in signal magnitude) as this can create undesirable signal distortions / artifacts in the output. In these embodiments, the smoothing can be "reset" at the point where a transient is detected in the signal.

[0045] The estimation matrix of the previous time frame can be stored to facilitate computing the smoothed value for the current frame. If a transient is detected in the input signal during that frame, the smoothing function can be set to reinitialize.

[0046] If a transient is detected, the past matrix estimate is reset to the current estimate, so that the output of the smoothing filter after the transient is the estimate itself (no change is applied). In other words, for the reset frame

[0047]

number

[0048] 4 shows an example modification to the flow for transient detection. A determination is made 401 whether a transient has been detected for a given frame. If so 403, the new matrix values ​​remain the same as the input values. If not 402, the normal smoothing algorithm is used for that frame. The combination (matrix) of smoothed and unsmoothed (transient) frame values ​​is used for signal processing 404.

[0049] In some embodiments, if a transient is detected on any channel, the smoothing is reset. For example, if there are N channels, then N transient detectors (one per channel) can be used, and if any of these detect a transient, the smoothing is reset or the end of the signal or the end of smoothing (smoothing is turned off).

[0050] In the example of stereo input, it may be determined that the channels are sufficiently separate (or perhaps separate) such that considering only the transient in the left channel can mean that important transients in the right channel are inappropriately smoothed (and vice versa). Thus, two transient detectors (left and right) are used, and either of these can trigger a smoothing reset of the overall 2×2 matrix.

[0051] In some embodiments, the smoothing is reset only at the transient for a particular channel. For example, if there are N channels, only M (<N, possibly 1) detectors are used.

[0052] In the example of first order ambisonics (FOA) input, it can be determined that the first (W) channel is the most important compared to the other three (X, Y, Z), and given the spatial relationship between the FOA signals, transients in the latter three channels are likely to be reflected in some way in the W channel. Thus, the system can be configured to use a transient detector only on the W channel, and when a transient is detected on the W channel, trigger a reset of the overall 4×4 covariance matrix.

[0053] In some embodiments, the reset resets only the covariance elements that received the transient. This means that a transient in the nth channel resets only the values in the nth row and nth column of the covariance matrix (the entire row and the entire column). This can be done by having separate monitoring of transients on each channel, and a transient detected on any given channel triggers a reset for the matrix positions corresponding to that channel's covariance with the other channels (and vice versa, which is obvious but also the same for itself).

[0054] In some embodiments, a reset occurs only upon a majority / threshold number of channels detecting a transient, e.g., in a four channel system, a threshold may be set to trigger a reset only if at least two channels report a transient in the same frame.

[0055] [Example of transient detection] One example of a transient detector is the use of a ducking gain characteristic of a ducking decorrelator, which can be used to determine whether a transient has occurred in a single input channel. See, for example, U.S. Patent No. 9,747,909, the contents of which are incorporated by reference. During a transient, a sudden decrease in ducking gain occurs. The change in ducking gain can be compared to a threshold to determine whether a transient has occurred. The threshold can be determined by subjective assessment of what provides accurate transient detection.

[0056] An exemplary transient detector operates as follows.

[0057] 1. The input signal is high-pass filtered to enhance transients.

[0058] 2. The absolute value of the output signal of the high-pass filter is then low-pass filtered to determine the signal envelope.

[0059] 3. Slightly different cutoff frequency ω f >ω s Using two different low-pass filters with f ]n] and slow e s Two envelopes of [n] are generated.

[0060] 4. For each time-domain sample n of the signal envelope in a particular frame, where 1≦n≦N, 1.1e s [n]< g d [n-1]e f [n] Formula 4 If , the ducking gain g d teeth,

[0061]

number

[0062]

number

[0063] 5. A transient is considered to have occurred if there is a large change in ducking gain over the current frame, i.e. d g [1]-d g [N]>Threshold Equation 7 In some embodiments, a method for determining the occurrence of a transient from the ducking gain is to count the number / fraction of time domain samples in each frame for which the above condition is met. If this value passes a certain threshold, the frame is determined to contain a transient.

[0064] The ducking gain can be used for transient detection for the smoothing algorithm, however, any method of transient detection known in the art can be used.

[0065] [Resampling (Banding Transition)] In some embodiments, if the band configuration of the channel changes, the bands undergoing smoothing can be transformed into new bands for smoothing. If the input signal bands change in number and / or bandwidth from one frame to the next (e.g., due to a filter bank switch in the encoder), the stored (previous) matrix estimates can be resampled to the new band structure in order to continue uninterrupted operation of the smoothing mechanism. The transformation matrix between band structures can be calculated by taking a set of window functions in the frequency domain, i.e., the ideal filter bank magnitude response for the previous band scheme (i.e., the previous filter bank) as a matrix.

[0066] As an example, Bank1 is a (B1 x F) array containing the rows of the previous ideal filter bank response FR1, and Bank2 is a (B2 x F) array containing the rows of the new filter bank ideal response FR2, where F is the number of frequency bins.

[0067]

number

[0068]

number

[0069] [Example of smoothing by resetting] 1×1 covariance matrix A b Consider a single band b in [n]. The band consists of three bins (N b =3), but at least eight bins are needed for good statistical analysis (N min = 8), which is determined (subjectively). b = 3 / 8 band forgetting factor.

[0070] If the initial band covariance matrix is ​​(first frame and transient shown in bold):

[0071]

number

[0072]

number

[0073]

number

[0074]

number

[0075]

number

[0076] The second frame is computed using its own value (0.05), the "smoothed" value from the previous frame (0.1), and a forgetting factor (3 / 8).

[0077]

number

[0078]

number

[0079]

number

[0080] In some embodiments, the covariance matrix smoothing system can be integrated into an encoder. An exemplary system is provided in Figure 5. An exemplary encoder 501 generates 502 one or more covariance matrices for a signal 504 that a smoothing algorithm 503 modifies before the signal 504 is sent to a decoder 505.

[0081] In some embodiments, the covariance smoothing system can be integrated into the decoder. An example of a decoder with a smoothing algorithm is shown in Figure 6. An encoder 601 sends a signal 602 (e.g., audio channels) to a decoder 603. The decoder 603 applies a smoothing algorithm 604 before further processing 605 is performed (there may be processing before the smoothing).

[0082] In some embodiments, covariance smoothing may be used in the encoder to stabilize the spatial comfort noise parameterization in case of discontinuous transmission (DTX) frames. In this case, there is no reset of the covariance smoothing due to transients during DTX frames, but instead it is reset when the codec enters and exits DTX mode. An example of an encoder with a smoothing algorithm is shown in Figure 7, where an encoder 701 has a DTX module 702 incorporating a smoothing algorithm 703.

[0083] Although a number of embodiments of the present disclosure have been described, it will be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the following claims.

[0084] A computing device implementing techniques for covariance smoothing may have the following exemplary architecture. Other architectures are possible, including architectures with more or fewer components. In some implementations, the exemplary architecture includes one or more processors (e.g., a dual-core Intel® Xeon® processor), one or more output devices (e.g., LCD), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components may communicate and exchange data over one or more communication channels (e.g., buses) that may utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0085] The term "computer-readable medium" refers to any medium that participates in providing instructions to a processor for execution, including, but not limited to, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory), and transmission media, including, but not limited to, coaxial cables, copper wire, and fiber optics.

[0086] The computer readable medium may further include an operating system (e.g., a Linux operating system), a network communication module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but not limited to, recognizing input from and providing output to the network interface 706 and / or device 708, tracking and managing files and directories on the computer readable medium (e.g., memory or storage device), controlling peripheral devices, and managing traffic on one or more communication channels. The network communication module includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols such as TCP / IP, HTTP, etc.).

[0087] The architecture may be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device having one or more processors. The software may comprise multiple software components or may be a single body of code.

[0088] The described features may be advantageously implemented in one or more computer programs executable on a programmable system including at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a particular action or bring about a particular result. A computer program may be written in any form of programming language, including compiled or interpreted languages ​​(e.g., Objective-C, Java), and may be deployed in any form, as a stand-alone program or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0089] Processors suitable for executing a program of instructions include, for example, both general purpose and special purpose microprocessors, as well as the sole processor or one of multiple processors or cores of any type of computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer also includes, or is operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all types of non-volatile memory, including, for example, semiconductor memory devices, such as EPROMs, EEPROMs, and flash memory devices, magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, ASICs (Application Specific Integrated Circuits).

[0090] To provide for interaction with a user, the features can be implemented in a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or a retina display device, for displaying information to a user. The computer can have a touch surface input device (e.g., a touch screen) or a keyboard and a pointing device, such as a mouse or trackball, by which a user can provide input to the computer. The computer can have a voice input device for receiving voice commands from a user.

[0091] Features may be implemented in a computer system including back-end components such as a data server, or a computer system including middleware components such as an application server or an Internet server, or a computer system including a front-end component such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The components of the system may be connected by any form or medium of digital data communication, such as a communications network. Examples of communications networks include, for example, a LAN, a WAN, and the computers and networks forming the Internet.

[0092] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server sends data (e.g., HTML pages) to the client devices (e.g., for the purpose of displaying the data and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., results of user interaction) can be received at the server from the client device.

[0093] One or more computer systems can be configured to perform particular operations by having software, firmware, hardware, or a combination of these installed on the system that, when in operation, causes the system to perform the operations. One or more computer programs can be configured to perform particular operations by containing instructions that, when executed by a data processing device, cause the device to perform the operations.

[0094] Although the present specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or the scope that may be described in the claims, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described herein with respect to separate embodiments can be implemented in combination in a single embodiment. Conversely, various features described with respect to a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features are described above as operating in a particular combination and may even initially be claimed in this manner, one or more features from the combination of claims may in some cases be carved out of the combination, and the combination of claims may be directed to a subcombination or a variation of the subcombination.

[0095] Similarly, although the figures depict acts in a particular order, this should not be understood as requiring such acts to be performed in the particular order or sequence depicted, or that all of the depicted acts be performed, to achieve desired results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated within a single software product or packaged into multiple software products.

[0096] Accordingly, certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Moreover, the processes depicted in the accompanying figures do not necessarily require the particular order or sequence depicted to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

[0097] Although a number of implementations of the invention have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0098] The following is disclosed regarding embodiments of the present invention.

[0099] (1) A method for smoothing an input covariance matrix for a sequence of signal frames in a frequency band, comprising the steps of: comparing, for the signal frame, a valid count of bins within the frequency band with a desired bin count for the frequency band; calculating a forgetting factor for the frequency band as a ratio of the useful count to the desired count; generating a smoothing matrix value for the current frame using the forgetting factor and a previously generated value of a smoothing matrix for a previous frame relative to the current frame in response to the valid count of bins in the frequency band being less than the desired count; Including, The method, wherein the comparing, calculating and generating steps are performed by a system including one or more computer processors.

[0100] (2) The method according to (1), wherein the effective number of bins is calculated as the sum of filter bank response values ​​for the frequency band.

[0101] (3) The method according to (1) or (2), wherein the generating step uses a first-order filter.

[0102] (4) The method of (3), wherein the primary filter includes a difference between the values ​​of the input covariance matrix for the current frame and the previously generated values ​​of the smoothing matrix for the previous frame, the difference being weighted by the forgetting factor.

[0103] (5) comparing said forgetting factor with a maximum allowable forgetting factor; setting the forgetting factor to a minimum value of the forgetting factor and the maximum allowable forgetting factor; Further comprising: 5. The method of claim 1, wherein the comparing and setting steps are performed before generating the value of the smoothing matrix for the current frame.

[0104] (6) detecting whether a transient has occurred within the monitored frame; in response to detecting that the transient has occurred, resetting the smoothing by setting the smoothing matrix values ​​for the monitored frame to original values ​​of the input matrix values ​​for the monitored frame without using the forgetting factor; The method according to any one of (1) to (5), further comprising:

[0105] (7) The method of (6), wherein the detecting step is performed by using an aspect of a ducking decorrelator.

[0106] (8) A method according to any one of (1) to (7), wherein the detecting step is performed on all channels.

[0107] (9) The method according to (8), wherein the detecting step detects that the transient has occurred if any transient is detected on any channel for the monitored frame.

[0108] (10) The method according to (8), wherein the detecting step detects that a transient has occurred only if the transient has occurred on a particular channel.

[0109] (11) storing the smoothing matrix for the current frame; resampling the stored smoothing matrix across frequency bands; The method according to any one of (1) to (10), further comprising:

[0110] (12) one or more computer processors; and a non-transitory computer-readable medium storing instructions that, when executed by the one or more computer processors, cause the one or more computer processors to perform the operations of the method according to any one of (1) to (11); A system including:

[0111] (13) A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the operations of the methods described in any one of (1) to (11).

[0112] (14) An encoder including the system described in (12).

[0113] (15) A decoder including the system according to (12).

Claims

1. 1. A method for providing an audio service, comprising: comparing, for a time domain sequence of signal frames, an effective count of bins in a frequency band with a desired count of bins for said frequency band; calculating a forgetting factor for the frequency band as a ratio of the valid count to the desired count, the forgetting factor being limited to a maximum allowed forgetting factor; generating, for each frequency band of a plurality of frequency bands, a respective smoothed band-specific covariance matrix for the current signal frame using the forgetting factor and a previously generated value of the respective smoothed band-specific covariance matrix for a previous signal frame relative to the current signal frame; generating an audio signal for the audio service using a frequency domain representation of the time domain sequence of signal frames and further using the corresponding set of each smoothed band-specific covariance matrices; Including, The method, wherein the comparing, calculating and generating steps are performed by a system including one or more computer processors.

2. The method of claim 1 , wherein the effective number of bins for each frequency band of the plurality of frequency bands is calculated as a sum of filter bank response values ​​for the frequency band.

3. The method of claim 1 , wherein generating each smoothed band-specific covariance matrix uses a first-order autoregressive low-pass filter.

4. 4. The method of claim 3, wherein using the first-order autoregressive low-pass filter includes calculating a difference between an input covariance matrix for the current frame and the previously generated values ​​of the respective smoothed band-specific covariance matrices for the previous frames, the difference being weighted by the forgetting factor.

5. detecting whether a transient has occurred within the monitored frame; resetting the low pass filter in response to the detected transient. The method of claim 3 further comprising:

6. The method of claim 5 , wherein the detecting step is performed by using aspects of a ducking decorrelator.

7. The method of claim 6 , wherein the detecting step is performed on all channels.

8. 8. The method of claim 7, wherein the detecting step detects that the transient has occurred if any transient is detected on any channel for the monitored frame.

9. 9. The method of claim 8, wherein the detecting step detects that a transient has occurred only if the transient occurs on a particular channel.

10. storing each of the smoothed band-specific covariance matrices; resampling the stored respective smoothed band-specific covariance matrices across the plurality of frequency bands; The method of claim 1 further comprising:

11. one or more computer processors; a non-transitory computer-readable medium storing instructions that, when executed by the one or more computer processors, cause the one or more computer processors to perform the operations of the method of claim 1; A system including:

12. A non-transitory computer readable medium having stored thereon instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the operations of the method of claim 1.

13. 1. A method for processing an audio signal, comprising: receiving an input signal representing N audio channels, where N>1; dividing the input signal into a time domain sequence of frames; transforming each frame of the time domain sequence of frames into a respective frequency domain representation having a plurality of frequency bands, each frequency band including a respective plurality of frequency bins, with lower frequency bands having a smaller number of bins per band than higher frequency bands; calculating, for each frame of the time domain sequence of frames, a band covariance matrix based on the frequency domain representation of each frame of the time domain sequence of frames, the band covariance matrix being an estimate matrix of covariances of the N audio channels within the frequency bands, and multiple adjacent frequency bins of per-bin covariance matrices being combined for each of the frequency bands; calculating a smoothed band covariance matrix for the current frame of the sequence of frames by adding a smoothed band covariance matrix for a previous frame of the sequence of frames and a difference between the band covariance matrix for the current frame and a smoothed band covariance matrix for the previous frame of the sequence of frames; The difference is weighted by a forgetting factor; the forgetting factor for a frequency band is calculated as a ratio of an effective number of bins in the frequency band to a minimum number of bins for the frequency band; the forgetting factor is limited to a maximum allowable forgetting factor; the minimum number of bins for the frequency band corresponds to a minimum number of bins determined to provide a good statistical estimate of the band covariance matrix for the current frame; processing the frequency domain representation of each frame of the time domain sequence of frames based on the smoothed band covariance matrix to determine a processed audio signal; outputting the processed audio signal; The method includes:

14. The method of claim 13 , wherein the effective number of bins for each frequency band of the plurality of frequency bands is calculated as a sum of filter bank response values ​​for the frequency band.

15. The method of claim 13 , wherein the step of calculating the smoothed band covariance matrix uses a first order autoregressive low pass filter.

16. 16. The method of claim 15, wherein using the first-order autoregressive low-pass filter includes calculating a difference between an input covariance matrix for the current frame and the previously generated values ​​of respective smoothed band-specific covariance matrices for the previous frames, the difference being weighted by the forgetting factor.

17. detecting whether a transient has occurred within the monitored frame; resetting the autoregressive low-pass filter in response to the detected transient; The method of claim 15 further comprising:

18. The method of claim 17 , wherein the detecting step is performed by using aspects of a ducking decorrelator.

19. 20. The method of claim 17, wherein the detecting step detects that the transient has occurred if any transient is detected on any channel for the monitored frame.

20. 20. The method of claim 17, wherein the detecting step detects that a transient has occurred only if the transient occurs on a particular channel.

Images