Systems and methods for covariance smoothing

Abstract

To provide a system and method for covariance smoothing.SOLUTION: A method and system improve signal processing by smoothing a covariance matrix of a multi-channel signal by setting a forgetting factor based on the bins of a band. The method and system for resetting smoothing based on transient detection are also disclosed. The method and system for resampling for smoothing between band transitions are also disclosed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] [Cross-references to related applications] This application claims priority by reference to U.S. Provisional Patent Application No. 62 / 881,825, filed on 1 August 2019, and U.S. Provisional Patent Application No. 63 / 057,533, filed on 28 July 2020.

[0002] [Technical field] This disclosure relates to improvements for signal processing. In particular, this disclosure relates to processing audio signals to improve covariance smoothing for improved processing. [Background technology]

[0003] One form of audio signal processing involves presenting multi-channel audio to a listener so that the listener can determine the virtual spatial location of the audio, thereby providing the listener with an immersive experience. Early implementations of this are stereo, where the listener can spatially determine the "direction" from which the sound source is coming.

[0004] Recent developments in this technology utilize inter-channel dependencies in multi-channel systems to present a more immersive sound experience. This may involve the use of audio channel covariance matrices. [Overview of the Initiative]

[0005] Various signal processing systems and methods are disclosed herein. Some of these systems and methods may include smoothing the covariance values ​​of bands over consecutive frames.

[0006] In some examples, a system and method for smoothing an estimate of a covariance matrix for a sequence of signal frames within a frequency band, the method comprising the steps of: comparing the effective count of a bin within a frequency band with the count of a desired bin for the frequency band; calculating a forgetting coefficient for the band as the ratio of the effective count to the desired count; and, if the effective count of a bin within a frequency band is less than the desired count, generating a current estimate of the covariance matrix value for the current frame using the forgetting coefficient and a previously generated estimate of the covariance matrix value for a previous frame relative to the current frame, wherein the comparing, calculating, and generating steps are performed by a system comprising one or more computer processors. The smoothed covariance matrix can be used to further improve signal processing by reducing artifacts caused by rough transitions in the matrix.

[0007] In some such examples, the system and method may include calculating the number of effective bins as the sum of the filter bank response values ​​for a frequency band.

[0008] In some of these examples, the generation step uses a primary filter.

[0009] In some of these examples, the first-order filter includes the difference between the value for the current frame and the previously generated estimate for previous frames, and this difference is weighted by a forgetting coefficient.

[0010] In some such examples, the system and method also include the steps of comparing the forgetting coefficient with the maximum forgetting coefficient and setting the forgetting coefficient to the minimum of the calculated forgetting coefficient and the maximum allowable forgetting coefficient, the comparison step and the setting step being performed before generating the current estimate of the covariance matrix value for the current frame.

[0011] In some such examples, the system and method also include detecting whether a transient event has occurred within the monitored frame, and in response to detecting that a transient event has occurred, resetting the smoothing by setting the current estimated value of the covariance matrix value for the monitored frame to the original value of the covariance matrix value for the monitored frame without using the forgetting factor for the monitored frame. The detecting step can be performed by using an aspect of a ducking decorrelator.

[0012] In some such examples, the system and method include storing the current estimated value of the covariance matrix of the current frame, and resampling the current estimated values stored across frequency bands and converting from one banding structure to another.

[0013] The system and method can be part of an audio signal decoder.

[0014] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non - transient media. Such non - transient media may include, but are not limited to, memory devices such as those described herein including random access memory (RAM) devices, read - only memory (ROM) devices, etc. Thus, various inventive aspects of the subject matter disclosed herein may be implemented on a non - transient medium storing software. The software may be executable, for example, by one or more components of a control system as disclosed herein. The software may include, for example, instructions for performing one or more of the methods disclosed herein.

[0015] At least some aspects of the present disclosure may be implemented via an apparatus or a plurality of apparatuses. For example, one or more devices may be configured to at least partially execute the methods disclosed herein. In some implementations, the apparatus may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between the control system and the memory system, one or more interfaces between the control system and other devices, and / or one or more external device interfaces. The control system may include at least one of a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gates or transistor logic, or discrete hardware components. Thus, in some implementations, the control system may include one or more processors and one or more non-transitory storage media operably coupled to the one or more processors.

[0016] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will be apparent from the description, the drawings, and the claims. Note that the relative dimensions in the following drawings may not be drawn to scale. Like reference numerals and designations in the various drawings generally denote like elements, although different reference numerals do not necessarily denote different elements between different drawings.

Brief Description of the Drawings

[0017] [Figure 1] An example of a signal channel separated into samples, frames, bins, and bands is shown. [Figure 2]A flowchart illustrating an exemplary method for smoothing a signal covariance matrix using a smoothing algorithm is shown. [Figure 3] A flowchart (partial) of an exemplary method for smoothing a signal covariance matrix using a smoothing algorithm with the maximum forgetting coefficient is shown. [Figure 4] A flowchart (partial) of an exemplary method for smoothing a signal covariance matrix using a smoothing algorithm with transient detection is shown. [Figure 5] An exemplary encoder / decoder system with a smoothing algorithm within the encoder is shown. [Figure 6] This illustrates an exemplary encoder / decoder system with a smoothing algorithm within the decoder. [Figure 7] This illustrates an exemplary encoder / decoder system that incorporates a smoothing algorithm as part of the DTX. [Modes for carrying out the invention]

[0018] This disclosure describes methods and systems for smoothing the values ​​of an estimated matrix (e.g., a covariance matrix) over time, including cases where the estimates are calculated from a small number of samples, in which unsmoothed variations in the values ​​of the estimated matrix may cause problems such as noticeable artifacts in the output (e.g., audio).

[0019] This disclosure also describes the use of transient detection to prevent (among other things) over-smoothing values ​​and thereby introducing undesirable artifacts from the smoothing.

[0020] As used herein, the term "smoothing" refers to reducing the variation in values ​​across consecutive frames of a signal. For example, reducing the variation in covariance values ​​across consecutive frames.

[0021] As used herein, the term "covariance" refers to a measure of the simultaneous variation of two signals, such as the covariance of the energies of the signals.

[0022] As used herein, a “covariance matrix” represents a matrix of covariances of multiple signals. A “banded covariance matrix” represents a covariance matrix in which several adjacent frequency bins of a bin-by-bin covariance matrix are grouped into bands, thereby allowing the covariance to be expressed in a significantly smaller number (equal to the number of bands). For example, an input with 960 frequency bins can become 12 frequency bands, each with 960 bins uniformly or heterogeneously distributed among 12 bands. For the examples provided herein, a “covariance matrix” is considered to be a banded covariance matrix.

[0023] As used herein, the "estimated matrix" refers to the sample covariance matrix. As used herein, the "covariance matrix" and "smoothed matrix" include the estimated matrix.

[0024] As used herein, the term "complex advanced coupling (CACPL)" refers to a stereo coding method in which the left and right signals are coupled into mid(L+R) / 2 and side(LR) / 2 signals, and then the side signal is predicted from the mid signal.

[0025] As used herein, the term “smoothing algorithm” refers to smoothing a covariance matrix in accordance with this disclosure, in particular, based on the effective bin count of the bands. As used herein, the term “smoothed matrix” refers to the covariance matrix modified by the smoothing algorithm.

[0026] As used herein, the term “effective bin count” refers to either the actual count of bins in a band or an estimate of the effective bin count in a band. An example of an estimate of the effective bin count in a band involves summing the filter bank response values ​​for that band.

[0027] As used herein, the term “forgetting factor” refers to a weighted value that modifies the importance of certain parts of a function.

[0028] As used herein, "DTX" refers to an encoder module that handles discontinuous transmission events.

[0029] This specification describes a solution to the problem of smoothing the covariance of input channels. This improves signal processing techniques, for example, for voice and audio services, and for providing users with immersive experiences (e.g., virtual reality, augmented reality).

[0030] In digital signal processing as shown in Figure 1, the input signal (channel) is divided into frames 101 in the time domain. A sequence of consecutive frames can be grouped together into a "frame sequence" 102. In the frequency domain, the signal can be divided into frequency intervals called bins 103. A given bin in a given frame is called a sample 106. These bins can be grouped in frequency bands 104 / 105. This allows for a statistical average across a given band for a given window frame / frame sequence (where "window" is a weighting function applied to a frame or frame sequence). One application of this averaging is to estimate the covariance between audio channels.

[0031] Not all bands necessarily contain the same number of bins. For example, a lower frequency band 104 may have fewer bins per band than a higher frequency band 105. This means that, for statistical analysis, a lower frequency band requires a larger window (more frames) per band than a higher frequency band requires to produce meaningful statistical results. A given bin in a given frame can have its associated signal energy, and some bin values ​​can be averaged across bands and / or windows.

[0032] [Example of a covariance matrix] Taking complex advanced coupling (CACPL) as an example, the parameters α (complex prediction coefficient) and β (real inverse correlation coefficient) are the (2×2) mid-side covariance matrix Σ MS This matrix may be calculated from its elements. This matrix is ​​calculated over a series of analysis bands 1 ≤ b ≤ B.

[0033]

number

[0034] [Smoothing function and forgetting coefficient] Generally, a smoothing matrix can be computed using a low-pass filter designed to satisfy the smoothing requirements. For smoothing algorithms, the smoothing requirements are such that previous estimates are used to artificially increase the number of frequency samples (bins) used to generate the current estimate of the covariance matrix. In some embodiments, the smoothing matrix is ​​calculated from the input covariance matrix A across a frame sequence.

[0035]

number

[0036]

number

[0037] [Forgetting Factor] The effect of the forgetting coefficient is that when smoothing is applied across consecutive frames, the effect of previous frames becomes progressively smaller in relation to the smoothing of the frame being smoothed (adjusted).

[0038] When the forgetting factor in Equation 1 is 1 (λ = 1), no smoothing occurs and it effectively acts as an all-pass filter. When 0 < λ < 1, the equation acts as a low-pass filter. A lower λ places emphasis on older covariance data, while a higher λ gives more consideration to new covariance. A forgetting factor greater than 1 (e.g., 1 < λ < 2) is implemented as a high-pass filter. In some embodiments, the maximum allowable forgetting factor λ max is implemented. This maximum value determines the behavior of the algorithm when the bin / band value becomes large. In some embodiments, regardless of what the calculated forgetting factor is, λ max < 1 always performs some smoothing in each band, and λ max = 1 applies the smoothing function only to bands with bins smaller than the desired N min and leaves the larger bands unsmoothed.

[0039] In some of these embodiments, the forgetting factor for a particular band λ b is calculated as the minimum of the ratio of the maximum allowable forgetting factor λ max to the effective number of bins N b within the band and the minimum number of bins N min determined to give a good statistical estimate based on the window size.

[0040]

Equation

[0041] For example, by using an analysis filter bank with a narrower (i.e., fewer bins, more frames required for good statistical analysis) low-frequency band and a wider (i.e., more bins, fewer frames required for good statistical analysis) high-frequency band, this increases the amount of smoothing in the low-frequency band and decreases the amount of smoothing in the high-frequency band (or, λ max (If = 1, it has the effect of not smoothing at all.)

[0042] An exemplary process flow for this method is shown in Figure 2. An input signal FFT (Fast Fourier Transform) 201 provides a corresponding covariance matrix on the window for a given band in the input signal. The effective bin count for that band is obtained 202. This can be calculated, for example, by the filter bank response value of the band. The desired bin count is determined, for example, by a subjective analysis of the number of bins required to provide a good statistical analysis of the window 203. The forgetting coefficient is calculated by taking the ratio of the calculated number of bins to the desired bin count 204. For a given frame (except the first frame), a new covariance matrix value is calculated based on the new covariance value calculated for the previous frame, the original value for the current frame, and the forgetting coefficient 205. The new (smoothed) matrix formed by these new values ​​is used in further signal processing 206.

[0043] Figure 3 shows an exemplary modification to the flow for the maximum allowable forgetting coefficient. Similar to Figure 2, the forgetting coefficient is calculated for the bands 301. Furthermore, the maximum allowable forgetting coefficient is determined 302. The values ​​are compared 303, and depending on whether the calculated coefficient is less than the maximum allowable coefficient, the calculated coefficient is used in smoothing 305. If the calculated coefficient is greater than the maximum allowable coefficient, the maximum allowable coefficient is used in smoothing 305. This example shows that if the coefficients are equal (not greater than or equal to), the calculated coefficient is used, but an equivalent flow can be assumed where the minimum value is used when they are equal.

[0044] [Reset smoothing] In some embodiments, it may be desirable to avoid smoothing during transients (sudden changes in signal magnitude) because this could generate undesirable signal distortion / artifacts in the output. In these embodiments, smoothing can be "reset" at the point where a transient is detected in the signal.

[0045] The estimation matrix for previous time frames can be stored to facilitate the calculation of smoothing values ​​for the current frame. If transients are detected in the input signal during that frame, the smoothing function can be set to reinitialize.

[0046] If a transient is detected, past matrix estimates are reset to the current estimate, and as a result, the output of the smoothing filter after the transient becomes the estimate itself (no change is applied). In other words, for the reset frame...

[0047]

number

[0048] Figure 4 shows an exemplary modification to the flow for transient detection. A determination is made 401 as to whether or not a transient has been detected for a given frame. If so 403, the values ​​of the new matrix remain the same as the input values. Otherwise 402, the normal smoothing algorithm is used for that frame. The combination (matrix) of the smoothed frame values ​​and the unsmoothed (transient) frame values ​​is used for signal processing 404.

[0049] In some embodiments, smoothing is reset if a transient is detected in any channel. For example, if there are N channels, N transient detectors (one per channel) can be used, and if any of these detect a transient, smoothing is reset, or the signal is terminated or smoothing is terminated (smoothing is turned off).

[0050] In the example of stereo input, it may be determined that the channels are sufficiently separate (or perhaps separate) such that considering only the transient in the left channel could mean that important transients in the right channel are inappropriately smoothed (and vice versa). Thus, two transient detectors (left and right) are used, and either of these can trigger a smoothing reset of the overall 2×2 matrix.

[0051] In some embodiments, the smoothing is reset only at the transient for a particular channel. For example, if there are N channels, only M (< N, and in some cases 1) detectors are used.

[0052] In the example of first order ambisonics (FOA) input, it can be determined that the first (W) channel is the most important compared to the other three (X, Y, Z), and given the spatial relationship between the FOA signals, transients in the latter three channels are likely to be reflected in some way in the W channel. Thus, the system can be configured to use a transient detector only on the W channel, and when a transient is detected on the W channel, it triggers a reset of the overall 4×4 covariance matrix.

[0053] In some embodiments, the reset resets only the covariance elements that received the transient. This means that a transient in the nth channel resets only the values in the nth row and nth column of the covariance matrix (the entire row and the entire column). This can be done by having separate monitoring of transients on each channel, and a transient detected on any given channel triggers a reset for the matrix positions corresponding to that channel's covariance with the other channels (and vice versa, which is obvious but also the same for itself).

[0054] In some embodiments, a reset occurs only when a majority / threshold number of channels detect a transient. For example, in a four-channel system, the threshold may be set to trigger a reset only if at least two channels report a transient in the same frame.

[0055] [Example of transient phenomenon detection] One example of a transient detection method is its use against the ducking gain characteristics of a ducking inverse correlator, which can be used to determine whether a transient has occurred in a single input channel. See, for example, U.S. Patent No. 9,747,909, which is incorporated by reference. In a transient, a sharp decrease occurs in the ducking gain. The change in the ducking gain can be compared to a threshold to determine whether a transient has occurred. The threshold can be determined by a subjective assessment of what provides accurate transient detection.

[0056] An exemplary transient detector operates as follows:

[0057] 1. The input signal is high-pass filtered to enhance transient response.

[0058] 2. Next, in order to determine the signal envelope, the absolute value of the output signal of the high-pass filter is subjected to low-pass filtering.

[0059] 3. Slightly different cutoff frequencies ω f >ω s Using two different low-pass filters with the same properties, a fast e f ]n] and low speed e s Two envelopes [n] are generated.

[0060] 4. For each time-domain sample n (1 ≤ n ≤ N) of the signal envelope within a specific frame, 1.1e s [n]< g d [n-1]e f [n] Formula 4 If so, the ducking gain g d teeth,

[0061]

number

[0062]

number

[0063] 5. If there is a large change in the ducking gain across the current frame, it is considered that a transient has occurred. That is, as follows: d g [1]-d g [N]>Threshold Equation 7 In some embodiments, a method for determining the occurrence of transients from the ducking gain is to count the number / part of time-domain samples within each frame that satisfy the above conditions. If this value exceeds a certain threshold, the frame is determined to contain transients.

[0064] The ducking gain can be used for transient detection for smoothing algorithms, but any transient detection method known in the art can be used.

[0065] [Resampling (Banding Transition)] In some embodiments, if the channel band configuration changes, the bands to be smoothed can be transformed into new bands for smoothing. If the input signal bands change in number and / or bandwidth from one frame to the next (for example, due to filter bank switching in an encoder), the stored (previous) matrix estimates can be resampled to the new band structure to continue the smoothing mechanism's uninterrupted operation. The transformation matrix between band structures can be computed by taking a set of window functions in the frequency domain, i.e., the ideal filter bank amplitude response for the previous band configuration (i.e., the previous filter bank), as a matrix.

[0066] For example, Bank1 is an array (B1 × F) containing rows of the previous ideal filter bank response FR1, and Bank2 is an array (B2 × F) containing the ideal response FR2 of the new filter bank, where F is the number of frequency bins.

[0067]

number

[0068]

number

[0069] [Example of smoothing by resetting] 1x1 covariance matrix A b Let's take a single band b of [n] as an example. The band consists of three bins (N b =3), but at least 8 bins are needed for good statistical analysis (N min =8) is (subjectively) determined. This is λ b This gives a band forgetting coefficient of 3 / 8.

[0070] If the initial band covariance matrix is ​​as follows (the first frame and transient phenomena are shown in bold),

[0071]

number

[0072]

number

[0073]

number

[0074]

number

[0075]

number

[0076] The second frame uses its own value (0.05), the "smoothed" value of the previous frame (0.1), and the forgetting factor (3 / 8),

[0077]

number

[0078]

number

[0079]

number

[0080] In some embodiments, the covariance matrix smoothing system can be integrated into the encoder. An exemplary system is provided in Figure 5. The exemplary encoder 501 generates one or more covariance matrices for the signal 504 that are modified by a smoothing algorithm 503 before the signal 504 is transmitted to the decoder 505 502.

[0081] In some embodiments, a covariance smoothing system can be integrated into the decoder. An example of a decoder having a smoothing algorithm is shown in Figure 6. Encoder 601 transmits a signal 602 (e.g., an audio channel) to decoder 603. Decoder 603 applies the smoothing algorithm 604 before any further processing 605 is performed (processing may exist before smoothing).

[0082] In some embodiments, covariance smoothing may be used in an encoder to stabilize the parameterization of spatial comfort noise in the case of discontinuous transmission (DTX) frames. In this case, there is no transient reset of covariance smoothing between DTX frames; instead, it is reset when the codec enters and exits DTX mode. An example of an encoder having a smoothing algorithm is shown in Figure 7, where encoder 701 has a DTX module 702 incorporating a smoothing algorithm 703.

[0083] Numerous embodiments of this disclosure have been described. Nevertheless, it is understood that various modifications may be made without departing from the true intent and scope of this disclosure. Accordingly, other embodiments are also within the scope of the claims.

[0084] Computing devices implementing covariance smoothing techniques may have the following exemplary architectures. Other architectures are also possible, including architectures with more or fewer components. In some implementations, the exemplary architecture includes one or more processors (e.g., dual-core Intel® Xeon® processors), one or more output devices (e.g., LCDs), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disc, flash memory, etc.). These components can communicate and exchange data over one or more communication channels (e.g., buses) that utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0085] The term “computer-readable medium” refers to a medium involved in providing instructions to a processor for execution, including, but is not limited to, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory), and transmission media. Transmission media include, but are not limited to, coaxial cables, copper wires, and optical fibers.

[0086] The computer-readable medium may further include an operating system (e.g., the Linux® operating system), a network communication module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but is not limited to, recognizing inputs from network interface 706 and / or device 708 and providing outputs to them, tracking and managing files and directories on the computer-readable medium (e.g., memory or storage device), controlling peripheral devices, and managing traffic on one or more communication channels. The network communication module includes various components for establishing and maintaining network connectivity (e.g., software for implementing communication protocols such as TCP / IP, HTTP, etc.).

[0087] The architecture can be implemented using parallel processing or a peer-to-peer infrastructure, or on a single device with one or more processors. The software can include multiple software components or it can be a single code system.

[0088] The described features can be advantageously implemented in one or more computer programs executable on a programmable system, which includes a data storage system, at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to at least one input device and at least one output device. A computer program is a set of instructions that can be used directly or indirectly within a computer to perform a particular action or to produce a particular result. A computer program can be written in any form of programming language, including compiled or interpreted languages ​​(e.g., Objective-C, Java), and can be deployed in either form as a standalone program or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0089] Processors suitable for executing instruction programs include, for example, both general-purpose and dedicated microprocessors, and one of a single processor or multiple processors or cores in either type of computer. Generally, a processor receives instructions and data from read-only memory, random-access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer also includes or is operably coupled to one or more mass storage devices for storing data files, such devices including magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for materially embodying computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and all forms of non-volatile memory, including CD-ROM and DVD-ROM disks. Processors and memory can be supplemented or incorporated into ASICs (Application-Specific Integrated Circuits).

[0090] To provide user interaction, the feature can be implemented in a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or retinal display device for displaying information to the user. The computer may have a touch surface input device (e.g., a touchscreen) or a keyboard and a pointing device such as a mouse or trackball from which the user can provide input to the computer. The computer may have a voice input device for receiving voice commands from the user.

[0091] The system can be implemented as a computer system including backend components such as data servers, or middleware components such as application servers or internet servers, or frontend components such as client computers with a graphical user interface or internet browser, or any combination thereof. The system components can be connected by digital data communication in any form or medium, such as a communication network. Examples of communication networks include, for example, computers and networks forming LANs, WANs, and the Internet.

[0092] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact through a communication network. The client-server relationship arises from computer programs running on each computer and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., an HTML page) to the client device (for example, to display data and receive user input from a user interacting with the client device). Data generated on the client device (e.g., the results of user interaction) can be received from the client device by the server.

[0093] One or more computer systems can be configured to perform a specific operation by installing software, firmware, hardware, or a combination thereof on the system so that the system performs the operation while it is running. One or more computer programs can be configured to perform a specific operation by including instructions that cause the data processing device to perform the operation when executed by the device.

[0094] This specification includes many specific implementation details, which should not be interpreted as limitations on the scope of any invention or claims, but rather as descriptions of features specific to particular embodiments of a particular invention. Specific features described herein in relation to separate embodiments can be implemented in combination in a single embodiment. Conversely, various features described in relation to a single embodiment can also be implemented separately or in any appropriate subcombination in multiple embodiments. Furthermore, features are described above as operating in a particular combination, and may even be initially described in the claims in this manner, but in some cases, one or more features from a combination of claims can be extracted from the combination, and the combination of claims may cover subcombinations or variations of subcombinations.

[0095] Similarly, although the drawings show operations in a specific order, this should not be understood as requiring that such operations be performed in a specific order or sequence shown in the drawings, or that all illustrated operations be performed, in order to achieve the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0096] Therefore, specific embodiments of the subject matter are described. Other embodiments are also within the scope of the following claims. In some cases, the operations described in the claims may be performed in a different order and still achieve the desired results. Furthermore, the processes shown in the accompanying drawings do not necessarily require a specific illustrated order or sequence to achieve the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

[0097] Numerous implementations of the present invention have been described. Nevertheless, it is understood that various modifications can be made without departing from the spirit and scope of the present invention.

[0098] The following details are disclosed regarding embodiments of the present invention.

[0099] (1) A method for smoothing an input covariance matrix for a sequence of signal frames within a frequency band, With respect to the signal frame, the steps include comparing the effective count of bins within the frequency band with the count of a desired bin for the frequency band, A step of calculating a forgetting coefficient for the frequency band as the ratio of the effective count to the desired count, Depending on whether the effective count of the bins in the frequency band is less than the desired count, the steps include generating a value for the smoothing matrix for the current frame using the forgetting coefficient and a previously generated value of the smoothing matrix for the previous frame relative to the current frame; Includes, A method wherein the comparison step, the calculation step, and the generation step are performed by a system including one or more computer processors.

[0100] (2) The method according to (1), wherein the number of effective bins is calculated as the sum of the filter bank response values ​​for the frequency band.

[0101] (3) The method according to (1) or (2), wherein the generating step is performed using a primary filter.

[0102] (4) The method according to (3), wherein the first-order filter includes the difference between the value of the input covariance matrix for the current frame and the previously generated value of the smoothing matrix for the previous frame, the difference being weighted by the forgetting coefficient.

[0103] (5) A step of comparing the forgetting coefficient with the maximum allowable forgetting coefficient, The steps include setting the forgetting coefficient to the minimum value among the forgetting coefficient and the maximum allowable forgetting coefficient, It further includes, The method according to any one of (1) to (4), wherein the comparison step and the setting step are performed before generating the value of the smoothing matrix for the current frame.

[0104] (6) A step of detecting whether or not a transient event occurred within the monitored frame, In response to detecting that the transient phenomenon has occurred, the smoothing is reset by setting the value of the smoothing matrix for the monitored frame to the original value of the input matrix for the monitored frame, without using the forgetting coefficient. The method described in any one of (1) through (5), further including the method described in (1) through (5).

[0105] (7) The method according to (6), wherein the detection step is performed by using an embodiment of a ducking inverse correlator.

[0106] (8) The detection step is performed on all channels, according to the method of any one of (1) to (7).

[0107] (9) The method according to (8), wherein the detection step is to detect that a transient has occurred if any transient is detected in any channel for the monitored frame.

[0108] (10) The method according to (8), wherein the detection step is to detect that the transient phenomenon has occurred only when the transient phenomenon occurs in a specific channel.

[0109] (11) A step of storing the smoothing matrix of the current frame, The steps include: resampling the stored smoothing matrix across frequency bands; The method described in any one of (1) through (10), further including the method described in (1) through (10).

[0110] (12) One or more computer processors, When executed by the one or more computer processors, a non-temporary computer-readable medium storing instructions causing the one or more computer processors to perform the operation described in any one of (1) to (11) is used. A system that includes this.

[0111] (13) A non-temporary computer-readable medium that, when executed by one or more computer processors, stores instructions causing the one or more computer processors to perform an operation as described in any one of (1) to (11).

[0112] An encoder including the system described in (14)(12).

[0113] A decoder including the system described in (15)(12).

Claims

1. A method of providing audio services, The steps include comparing the effective count of bins within a frequency band with the minimum count of bins for the frequency band for the time-domain sequence of the signal frame, The steps include: calculating a forgetting coefficient for the frequency band as the ratio of the effective count to the minimum count, wherein the forgetting coefficient is limited to the maximum allowable forgetting coefficient; For each frequency band of a plurality of frequency bands, the steps of generating each smoothed band-specific covariance matrix for the current signal frame across a window, using the forgetting coefficient and previously generated values ​​of the respective smoothed band-specific covariance matrices for previous signal frames relative to the current signal frame; The steps of generating an audio signal for the audio service using the frequency domain representation of the time domain sequence of the signal frame, and further using the corresponding set of the respective smoothed band-specific covariance matrices; Includes, The minimum count of bins for the frequency band corresponds to the minimum number of bins determined to give a good statistical estimate for the window, A method wherein the comparison step, the calculation step, and the generation step are performed by a system including one or more computer processors.

2. The method according to claim 1, wherein the effective number of bins for each frequency band of the plurality of frequency bands is calculated as the sum of the filter bank response values ​​for the frequency bands.

3. The method according to claim 1, wherein the step of generating the respective smoothed band-specific covariance matrix is ​​performed using a first-order autoregressive low-pass filter.

4. The method according to claim 3, wherein the use of the first-order autoregressive low-pass filter comprises calculating the difference between the input covariance matrix for the current frame and the previously generated values ​​of the respective smoothed band-specific covariance matrices for the previous frames, the difference being weighted by the forgetting coefficient.

5. A step to detect whether or not a transient event occurred within the monitored frame, The steps include resetting the low-pass filter in response to the detected transient phenomenon and The method according to claim 3, further comprising:

6. The method according to claim 5, wherein the detection step is performed by using an embodiment of a ducking inverse correlator.

7. The method according to claim 6, wherein the detection step is performed on all channels.

8. The method according to claim 7, wherein the detection step is to detect that a transient phenomenon has occurred if any of the transient phenomena are detected in any channel for the monitored frame.

9. The method according to claim 8, wherein the detection step is to detect that the transient phenomenon has occurred only when the transient phenomenon occurs on a specific channel.

10. The steps include storing the respective smoothed band-specific covariance matrices, The steps include: resampling the respective smoothed band-specific covariance matrices stored across the plurality of frequency bands; The method according to claim 1, further comprising:

11. One or more computer processors, When executed by the one or more computer processors, a non-temporary computer-readable medium storing instructions causing the one or more computer processors to perform the operation described in claim 1, and A system that includes this.

12. A non-temporary computer-readable medium that, when executed by one or more computer processors, stores instructions causing the one or more computer processors to perform the operation described in claim 1.

Images