Low latency low frequency effect codec

Abstract

The present disclosure relates to low-latency low-frequency effect codecs. In some embodiments, a method of encoding a low-frequency effect (LFE) channel includes receiving a time-domain LFE channel signal, filtering the time-domain LFE channel signal using a low-pass filter, converting the filtered time-domain LFE channel signal to a frequency-domain representation of the LFE channel signal including a number of coefficients representing a spectrum of the LFE channel signal, arranging coefficients into a number of sub-band groups corresponding to different frequency bands of the LFE channel signal, quantizing coefficients in each sub-band group according to a frequency response curve of the low-pass filter, encoding the quantized coefficients in each sub-band group using an entropy coder tuned for the sub-band group, and generating a bitstream including the encoded quantized coefficients, and storing the bitstream on a storage device or streaming the bitstream to a downstream device.

Description

[0001] Information related to divisional application This case is a divisional application. The parent application of this divisional application is the invention patent application filed on September 1, 2020, with application number 202080061951.3 and invention title "Low-Latency Low-Frequency Effect Encoder and Decoder". Cross-reference of related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 895,049, filed September 3, 2019, and U.S. Provisional Patent Application No. 63 / 069,420, filed August 24, 2020, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] This invention generally relates to audio signal processing, and more specifically, to processing low-frequency effect (LFE) channels. Background Technology

[0004] For example, standardization efforts for immersive services include developing Immersive Sound and Audio Services (IVAS) codecs for audio, multi-stream teleconferencing, virtual reality (VR), and streaming of user-generated live and non-live content. The goal of the IVAS standard is to develop a single codec with excellent audio quality, low latency, support for spatial audio decoding, an appropriate bit rate range, high-quality error recovery, and low practical implementation complexity. To achieve this, it is desirable to develop IVAS codecs that can handle low-latency LFE operation based on devices capable of performing IVAS or any other device capable of processing LFE signals. LFE channels are used for deep bass-heavy sound ranging from 20 Hz to 120 Hz and are typically sent to speakers designed to reproduce low-frequency audio content. Summary of the Invention

[0005] This paper discloses an implementation scheme for a configurable low-latency LFE codec.

[0006] In some implementations, a method for encoding a low-frequency effect (LFE) channel includes: receiving a time-domain LFE channel signal using one or more processors; filtering the time-domain LFE channel signal using a low-pass filter; converting the filtered time-domain LFE channel signal into a frequency-domain representation of the LFE channel signal containing a number of coefficients representing the spectrum of the LFE channel signal using the one or more processors; arranging the coefficients into a number of subband groups corresponding to different frequency bands of the LFE channel signal using the one or more processors; quantizing the coefficients in each subband group according to the frequency response curve of the low-pass filter using the one or more processors; encoding the quantized coefficients in the subband groups using an entropy decoder tuned for each subband group using the one or more processors; generating a bit stream containing the encoded quantized coefficients using the one or more processors; and storing the bit stream on a storage device or streaming the bit stream to a downstream device using the one or more processors.

[0007] In some implementations, quantizing the coefficients in each frequency band group further includes: generating a scaling shift factor based on the maximum number of available quantization points and the sum of the absolute values ​​of the coefficients; and quantizing the coefficients using the scaling shift factor.

[0008] In some implementations, if the quantized coefficients exceed the maximum number of quantization points, then the scaling shift factor is reduced and the coefficients are quantized again.

[0009] In some implementations, the quantization point is different for each frequency band group.

[0010] In some implementations, the coefficients in each frequency band group are quantized according to a fine quantization scheme or a coarse quantization scheme, wherein the fine quantization scheme allocates more quantization points to the corresponding sub-frequency band group compared to the quantization points assigned to one or more sub-frequency band groups according to the coarse quantization scheme.

[0011] In some implementations, the sign bit of the coefficient is decoded separately from the coefficient itself.

[0012] In some implementations, there are four sub-band groups, with the first sub-band group corresponding to a first frequency range of 0 Hz to 100 Hz, the second sub-band group corresponding to a second frequency range of 100 Hz to 200 Hz, the third sub-band group corresponding to a third frequency range of 200 Hz to 300 Hz, and the fourth sub-band group corresponding to a fourth frequency range of 300 Hz to 400 Hz.

[0013] In some implementations, the entropy decoder is an arithmetic entropy decoder.

[0014] In some implementations, converting the filtered time-domain LFE channel signal into a frequency-domain representation of the LFE channel signal containing a certain number of coefficients representing the spectrum of the LFE channel signal further includes: determining a first step length of the LFE channel signal; specifying a first window size of a window function based on the first step length; applying the first window size to one or more frames of the time-domain LFE channel signal; and applying a modified discrete cosine transform (MDCT) to the windowed frames to generate the coefficients.

[0015] In some implementations, the method further includes: determining a second step size of the LFE channel signal; specifying a second window size of the window function based on the second step size; and applying the second window size to the one or more frames of the time-domain LFE channel signal.

[0016] In some implementations, the first step length is N milliseconds (ms), where N is greater than or equal to 5 ms and less than or equal to 60 ms, the first window size is greater than or equal to 10 ms, the second step length is 5 ms, and the second window size is 10 ms.

[0017] In some implementations, the first step is 20 milliseconds (ms), the first window size is 10 ms, 20 ms, or 40 ms, the second step is 10 ms, and the second window size is 10 ms or 20 ms.

[0018] In some implementations, the first step is 10 milliseconds (ms), the first window size is 10 ms or 20 ms, the second step is 5 ms, and the second window size is 10 ms.

[0019] In some implementations, the first step is 20 milliseconds (ms), the first window size is 10 ms, 20 ms, or 40 ms, the second step is 5 ms, and the second window size is 10 ms.

[0020] In some implementations, the window function is a Kaiser-Bezos-derived (KBD) window function with a configurable fade-out length.

[0021] In some implementations, the low-pass filter is a fourth-order Butterworth low-pass filter with a cutoff frequency of about 130 Hz or less.

[0022] In some implementations, the method further includes: using the one or more processors to determine whether the energy level of a frame of the LFE channel signal is below a threshold; generating a silent frame indicator to indicate the decoder based on the energy level being below the threshold; inserting the silent frame indicator into the metadata of the LFE channel bitstream; and reducing the LFE channel bit rate when a silent frame is detected.

[0023] In some implementations, a method for decoding a low-frequency effect (LFE) channel bitstream includes: receiving an LFE channel bitstream using one or more processors, the LFE channel bitstream containing entropy decoding coefficients representing the spectrum of a time-domain LFE channel signal; decoding the quantized coefficients using the one or more processors with an entropy decoder; dequantizing the inverse-quantized coefficients using the one or more processors, wherein the coefficients have been quantized in a sub-band group corresponding to a frequency band according to the frequency response curve of a low-pass filter used to filter the time-domain LFE channel signal in an encoder; converting the inverse-quantized coefficients into a time-domain LFE channel signal using the one or more processors; adjusting the delay of the time-domain LFE channel signal using the one or more processors; and filtering the delay-adjusted LFE channel signal using a low-pass filter.

[0024] In some implementations, the order of the low-pass filter is configured to ensure that the first total algorithm delay caused by encoding and decoding the LFE channel in a multi-channel audio signal containing the LFE channel signal is less than or equal to the second total algorithm delay caused by encoding and decoding the other channels.

[0025] In some implementations, the method further includes: determining whether the second total algorithm delay exceeds a threshold; configuring the low-pass filter as an N-order low-pass filter based on the second total algorithm delay exceeding the threshold, where N is an integer greater than or equal to 2; and configuring the order of the low-pass filter to be less than N based on the second total algorithm delay not exceeding the threshold.

[0026] Other embodiments disclosed herein relate to systems, devices, and computer-readable media. Details of one or more of the disclosed embodiments are set forth in the accompanying drawings and the following description. Other features, objectives, and advantages are apparent from the description, drawings, and claims.

[0027] The specific embodiments disclosed herein offer one or more of the following advantages. The disclosed low-latency LFE codec: 1) is primarily designed for LFE channels; 2) is primarily designed for the frequency range of 20 Hz to 120 Hz, but carries audio up to 300 Hz in low / medium bitrate scenarios and up to 400 Hz in high bitrate scenarios; 3) achieves low bitrate by applying a quantization scheme based on the frequency response curve of the input low-pass filter; 4) has low algorithm latency and is designed to operate with a 20 ms step size and a total algorithm latency of 33 msec (including framing); 5) can be configured to smaller step sizes and lower algorithm latency to support other scenarios, including configurations as low as 5 msec step sizes and 13... 6) Total algorithm latency (including framing) of msec; 7) Automatic selection of low-pass filter at decoder output based on the latency available to the LFE codec; 8) Quiet mode with a low bit rate of 50 bits / second (bps) during quiet periods; and 9) Bit rate fluctuating between 2 kilobits / second (kbps) and 4 kbps during active frames, depending on the quantization level used, and 50 bps during quiet frames. Attached Figure Description

[0028] In the drawings, for ease of explanation, a specific arrangement or order of illustrative elements (e.g., elements representing devices, units, instruction blocks, and data elements) is shown. However, those skilled in the art will understand that the specific order or arrangement of the illustrative elements in the drawings is not intended to imply a requirement for a particular processing order or sequence or process separation. Furthermore, the inclusion of illustrative elements in the drawings is not intended to imply that such elements are required in all embodiments, or that the features represented by these elements may not be included in some embodiments or combined with other elements in some embodiments.

[0029] Furthermore, in the drawings, connecting elements (e.g., solid or dashed lines or arrows) are used to illustrate connections, relationships, or associations between or among two or more other schematic elements. The absence of any of these connecting elements is not intended to imply that a connection, relationship, or association cannot exist. In other words, some connections, relationships, or associations between elements are not shown in the drawings to avoid obscuring the invention. Additionally, for ease of illustration, a single connecting element is used to represent multiple connections, relationships, or associations between elements. For example, where connecting elements represent communication of signals, data, or instructions, those skilled in the art will understand that these elements, as needed, represent one or more signal paths affecting the communication.

[0030] Figure 1 The diagram illustrates an IVAS encoder / decoder for encoding and decoding IVAS and LFE bitstreams according to one or more implementation schemes.

[0031] Figure 2A It is a block diagram illustrating LFE coding according to one or more implementation schemes.

[0032] Figure 2B It is a block diagram illustrating LFE decoding according to one or more implementation schemes.

[0033] Figure 3 This is a graph illustrating the frequency response of a fourth-order Butterworth low-pass filter with a corner cutoff point of 130 Hz according to one or more implementation schemes.

[0034] Figure 4 It is a graph illustrating the Fielder window according to one or more implementation schemes.

[0035] Figure 5 The diagram illustrates the variation of points with frequency based on one or more implementation schemes.

[0036] Figure 6 The diagram illustrates the rough quantification of the variation of points with frequency based on one or more implementation schemes.

[0037] Figure 7 The diagram illustrates the probability distribution of quantized MDCT coefficients during fine quantization according to one or more implementation schemes.

[0038] Figure 8 The diagram illustrates the probability distribution of quantized MDCT coefficients during coarse quantization, based on one or more implementation schemes.

[0039] Figure 9 It is a flowchart of the process of encoding the coefficients of the Modified Discrete Cosine Transform (MDCT) according to one or more implementation schemes.

[0040] Figure 10 It is a flowchart of the process of decoding the coefficients of the Modified Discrete Cosine Transform (MDCT) according to one or more implementation schemes.

[0041] Figure 11 It is an implementation reference based on one or more implementation schemes. Figures 1 to 10 A block diagram of the system describing the features and processes.

[0042] The same reference symbols are used in various diagrams to indicate similar elements. Detailed Implementation

[0043] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the various described embodiments. Those skilled in the art will appreciate that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid unnecessarily obscuring aspects of the embodiments. Several features are described below that can be used independently of each other or in any combination with other features.

[0044] Nomenclature

[0045] As used herein, the term "comprising" and its variations are interpreted as open-ended terms meaning "including but not limited to". The term "or" is interpreted as "and / or" unless the context clearly indicates otherwise. The term "based on" is interpreted as "at least partially based on". The terms "one exemplary embodiment" and "exemplary embodiment" are interpreted as "at least one exemplary embodiment". The term "another embodiment" is interpreted as "at least one other embodiment". The terms "determined" or "determined" are interpreted as obtaining, receiving, operating, calculating, estimating, predicting, or deriving. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains.

[0046] System Overview

[0047] Figure 1 The diagram illustrates an IVAS codec 100 for encoding and decoding an IVAS bitstream (including an LFE channel bitstream) according to one or more embodiments. During encoding, the IVAS codec 100 receives N+1 channels of audio data 101, wherein N channels of the audio data 101 are input to a spatial analysis and downmixing unit 102, and one LFE channel is input to an LFE channel encoding unit 105. The audio data 101 includes, but is not limited to: mono signals, stereo signals, binaural signals, spatial audio signals (e.g., multichannel spatial audio objects), high-order high-fidelity stereo reproduction (FoA), high-order high-fidelity stereo reproduction (HoA), and any other audio data.

[0048] In some implementations, the spatial analysis and downmixing unit 102 is configured to implement Complex Advanced Coupling (CACPL) for analyzing / downmixing stereo audio data, and / or to implement Spatial Reconstruction (SPAR) for analyzing / downmixing FoA audio data. In other implementations, the spatial analysis and downmixing unit 102 implements other formats. The output of the spatial analysis and downmixing unit 102 includes spatial metadata and one to N channels of audio data. The spatial metadata is input to a spatial metadata encoding unit 104, which is configured to quantize and entropy decode the spatial metadata. In some implementations, quantization may include fine, moderate, coarse, and additionally coarse quantization strategies, and entropy decoding may include Huffman or arithmetic decoding.

[0049] One to N channels of audio data are input to a main audio channel encoding unit 103, which is configured to encode the one to N channels of audio data into one or more Enhanced Voice Service (EVS) bitstreams. In some implementations, the main audio channel encoding unit 103 conforms to 3GPP TS 26.445 and provides a variety of functionalities, such as enhancing the quality and decoding efficiency of narrowband (EVS-NB) and wideband (EVS-WB) voice services, enhancing the quality of ultra-wideband (EVS-SWB) voice, enhancing the quality of mixed content and music in conversational applications, robustness to packet loss and latency jitter, and backward compatibility with AMR-WB codecs.

[0050] In some embodiments, the main audio channel coding unit 103 includes a preprocessing and mode selection unit that selects between a speech decoder for encoding the speech signal and a perceptual decoder for encoding the audio signal at a predetermined bit rate based on mode / bit rate control. In some embodiments, the speech encoder is an improved variation of Algebraic Excited Linear Prediction (ACELP), which extends to specific LP-based modes for different speech categories.

[0051] In some implementations, the audio encoder is a modified discrete cosine transform (MDCT) encoder, which is more efficient at low latency / low bit rate and designed to perform seamless and reliable switching between the speech encoder and the audio encoder.

[0052] As previously described, the LFE channel signal is used for deep bass-heavy sound ranging from 20 Hz to 120 Hz and is typically sent to speakers (e.g., subwoofers) designed to reproduce low-frequency audio content. The LFE channel signal is input to an LFE channel signal encoding unit 105, which is configured to follow a reference... Figure 2AThe description describes the encoding of LFE channel signals.

[0053] In some implementations, the IVAS decoder includes: a spatial metadata decoding unit 106 configured to recover spatial metadata; and a main audio channel decoding unit 107 configured to recover one to N channel audio signals. The recovered spatial metadata and the recovered one to N channel audio signals are input to a spatial synthesis / upmixing / reproduction unit 109, which is configured to use the spatial metadata to synthesize and reproduce the one to N channel audio signals into N or more channel output audio signals for playback on speakers in various audio systems, including but not limited to: home theater systems, video conferencing systems, virtual reality (VR) equipment, and any other audio systems capable of reproducing audio. An LFE channel decoding unit 108 receives and is configured to decode the LFE bitstream, as described in reference [reference missing]. Figure 2B As described.

[0054] Although the exemplary implementation of LFE encoding / decoding described above is performed by an IVAS codec, the low-latency LFE codec described below can be a standalone LFE codec, or it can be included in any dedicated or standardized audio codec for encoding and decoding low-frequency signals in audio applications that require or expect low latency and configurability.

[0055] Figure 2A This is a diagram illustrating one or more embodiments. Figure 1 The block diagram of the functional components of the LFE channel encoding unit 105 shown in the figure. Figure 2B This is a diagram illustrating one or more embodiments. Figure 1 The block diagram of the functional components of the LFE channel decoder 108 shown is illustrated. The LFE channel decoder 108 includes an entropy decoding and inverse quantization unit 204, an inverse MDCT and windowing unit 205, a delay adjustment unit 206, and an output LPF 207. The delay adjustment unit 206 may be located before or after the LPF 207 and performs delay adjustment (e.g., by buffering the decoded LFE channel signal) to match the decoded LFE channel signal with the decoded output of the main codec. References are made later. Figure 2B The LFE channel encoding unit 105 and LFE channel decoding unit 108 described herein are collectively referred to as LFE codecs.

[0056] LFE channel coding unit 105 includes an input low-pass filter (LPF) 201, a windowing and MDCT unit 202, and a quantization and entropy decoding unit 203. In an embodiment, the input audio signal is a pulse code modulation (PCM) audio signal, and the LFE channel coding unit 105 expects an input audio signal with a stride of 5 milliseconds, 10 milliseconds, or 20 milliseconds. Intrinsically, the LFE channel coding unit 105 operates on 5-millisecond or 10-millisecond subframes and performs windowing and MDCT on combinations of these subframes. In an embodiment, the LFE channel coding unit 105 operates with a 20-millisecond input stride and intrinsically divides this input into two subframes of equal length. The last subframe of the previous input frame leading to the LFE is concatenated with the first subframe of the current input frame leading to the LFE and is windowed. The first subframe of the current input frame leading to the LFE is concatenated with the second subframe of the current input frame leading to the LFE and is windowed. MDCT is performed twice, once on each windowed block.

[0057] In this embodiment, the algorithm latency (excluding framing latency) is equal to 8 milliseconds plus the latency caused by the input LPF 103 plus the latency caused by the output LPF 207. With a fourth-order input LPF 201 and a fourth-order output LPF 207, the total system latency is approximately 15 milliseconds. With a fourth-order input LPF 201 and a second-order output LPF 207, the total LFE codec latency is approximately 13 milliseconds.

[0058] Figure 3 This is a graph illustrating the frequency response of an exemplary input LPF 201 according to one or more embodiments. In the example shown, LPF 201 is a fourth-order Butterworth filter with a cutoff frequency of 130 Hz. Other embodiments may use different types of LPFs (e.g., Chebyshev, Bessel) with the same or different orders and the same or different cutoff frequencies.

[0059] Figure 4 This is a graph illustrating the Field window according to one or more embodiments. In this embodiment, the window function applied by windowing and MDCT unit 202 is a Field window function with a fade-out length of 8 milliseconds. The Field window is a Kaiser-Bezos-derived (KBD) window with alpha=5, which is constructed to satisfy the Princen-Bradley condition of MDCT and is therefore used in the Advanced Audio Decoding (AAC) digital audio format. Other window functions may also be used.

[0060] Quantization and Entropy Decoding

[0061] In this embodiment, the quantization and entropy decoding unit 203 implements a quantization strategy conforming to the frequency response curve of the input LPF 201 to quantize the MDCT coefficients more efficiently. In this embodiment, the frequency range is divided into four sub-band groups representing four frequency bands: 0 Hz to 100 Hz, 100 Hz to 200 Hz, 200 Hz to 300 Hz, and 300 Hz to 400 Hz. These bands are examples, and more or fewer bands may be used with the same or different frequency ranges. More precisely, the MDCT coefficients are quantized using a scaling-shift factor dynamically calculated based on the MDCT coefficient values ​​in a specific frame, and the quantization point is selected according to the LPF frequency response curve, such as... Figures 5 to 8 As shown in the figure. This quantization strategy helps reduce the quantization points of MDCT coefficients belonging to the 100 Hz to 200 Hz, 200 Hz to 300 Hz and 300 Hz to 400 Hz frequency bands, while reserving the optimal quantization point in the main LFE frequency band 0 Hz to 100 Hz, where the lowest frequency effect energy (e.g., rumble) will exist.

[0062] In the embodiment, the path to the LFE channel encoding unit 105 is described below. F len Quantization strategy for millisecond (ms) input PCM stride (input frame length), where frame length F len 5 can be selected f For any value given by ms, where 1 <= f <=12.

[0063] First, divide the input PCM stride into equal lengths. N There are 10 subframes, and the width of each subframe is (S). w ) = F len / N ms. N It should be selected so that each S w It is a multiple of 5 ms (for example, if F len= 20 ms, then N It can be 1, 2, or 4; if F len = 10 ms, then N can be 1 or 2; and if F len = 5 ms, then N (Equal to 1). Make S i It is the first in any given frame i Subframe, here i The range is 0 <= i<= N Integers, where S 0 Corresponding to the last subframe in the previous input frame leading to LFE coding unit 105, and S 1 arrive S N yes In the current frame N Subframes.

[0064] Next, each S i and S i+1 Subframes are cascaded and the Field window is used (see...) Figure 4 The samples are windowed, and then MDCT is performed on these windowed samples. This results in a total of N MDCTs for each frame. The number of MDCT coefficients from each MDCT is ( num_coeffs ) = Sampling frequency S w / 1000. Frequency resolution per MDCT (width of each MDCT coefficient) ( W mdct It is approximately 1000 / (2) S w ) Hz. Given that woofers typically have an LPF cutoff of approximately 100 Hz to 120 Hz, and that the post-LPF energy after 400 Hz is generally very low, MDCT coefficients up to 400 Hz are quantized and sent to the LFE decoding unit 108, while the remainder of the MDCT coefficients are quantized to 0. Sending MDCT coefficients up to 400 Hz ensures high-quality reconstruction up to 120 Hz at the LFE decoding unit 108. Therefore, the quantization and decoding ( N quant The total number of MDCT coefficients is equal to N. 400 / W mdct .

[0065] Next, the MDCT coefficients will be arranged in... M In each sub-band group, the width of each sub-band group is W mdct The sum of the widths of all sub-band groups is equal to 400 Hz, and the width of each sub-band is a multiple of SBW. m Hz, of which m The range is 1 <= m <= M An integer. Within this width, the... mThe number of coefficients in the subband group = SN quant = N SBW m / W mdct (i.e., from each MDCT) SBW m / W mdct (coefficients). Then, the MDCT coefficients in each band group are scaled according to a shift scaling factor (shift) described below, which is determined by all... N quant The sum or maximum value of the absolute values ​​of the MDCT coefficients is determined. Then, at the encoder input, the scaled MDCT coefficients in each frequency band group are individually quantized and decoded using a quantization scheme conforming to the LPF curve. The quantized MDCT coefficients are decoded using an entropy decoder (e.g., an arithmetic or Huffman decoder). Different entropy decoders are used to decode each frequency band group, and each entropy decoder uses an appropriate probability distribution model to efficiently decode the corresponding sub-frequency band group.

[0066] Now we will describe a stride with a duration of 20 milliseconds (ms). F len = 20 ms), 2 subframes ( N = 2) An instance-based quantization strategy with a sampling frequency of 48000. Under this instance-based input configuration, the subframe width S w = 10 ms and the number of MDCTs = N =2. Perform the first MDCT on the 20 ms block. This block is formed by concatenating the 10 ms to 20 ms subframes from the previous 20 ms input with the 0 ms to 10 ms subframes from the current 20 ms input, and then windowing it into a 20 ms long Field window (see...). Figure 4 ).exist N = 1 And N With a value of 4, the Field window is scaled accordingly and the fade-out length is changed to 16 / Nms. A second MDCT is performed on the 20ms block formed by windowing the current 20ms input frame using a 20ms long Field window. The number of MDCT coefficients for each MDCT ( num_coeffs = 480, the width of each MDCT coefficient W mdct = 50 Hz, the total number of quantization and decoding coefficients N quant= 16, and the total number of quantization and decoding coefficients / MDCT = 16 / N = 8.

[0067] Next, the MDCT coefficients are arranged in four sub-band groups (M=4), where each sub-band group corresponds to a 100 Hz frequency band (0 to 100, 100 to 200, 200 to 300, 300 to 400, ...). SBW m =100 Hz, the number of coefficients in each frequency band group = SN quant = N SBW m / W mdct = 4). Let a1, a2, a3, a4, a5, a6, a7, and a8 be the first 8 MDCT coefficients to be quantized from the first MDCT, and b1, b2, b3, b4, b5, b6, b7, and b8 be the first 8 MDCT coefficients to be quantized from the second MDCT. The four subband groups are arranged to have the following coefficients:

[0068] Subband group 1 = {a1, a2, b1, b2}

[0069] Subband group 2 = {a3, a4, b3, b4}

[0070] Subband group 3 = {a5, a6, b5, b6}

[0071] Subband group 4 = {a7, a8, b7, b8}

[0072] Each frequency band group corresponds to a 100 Hz frequency band.

[0073] Frames with a gain of approximately -30 dB (or less than -30 dB) can have values ​​greater than 10. -2 Or 10 -1 MDCT coefficients may be lower, while frames with full-scale gain may have MDCT coefficients with values ​​of 20 or higher. To satisfy this wide range of values, the maximum available quantization point ( max_value ) and MDCT coefficients ( lfe_dct_new The scaling shift factor is calculated by summing the absolute values ​​of the values ​​of the components. shift ),as follows:

[0074] shift =floor( shifts_per_double log2( max_value / sum(abs( lfe_dct_new)))).

[0075] In the implementation plan, lfe_dct_new It is an array of 16 MDCT coefficients. shifts_per_double It is a constant (e.g., 4). max_value It is an integer chosen for fine quantization (e.g., 63 quantization values) and coarse quantization (e.g., 31 quantization values), with shifts limited to 5 bits from 4 to 35 in fine quantization and 5 bits from 2 to 33 in coarse quantization.

[0076] Then, the MDCT coefficients are quantized as follows:

[0077] vals=round( lfe_dct_new (2^( shift / shifts_per_double The round() operation rounds the result to the nearest integer value.

[0078] If the quantified value ( vals Exceeding the maximum allowed number of available quantization points ( max_val ), then reduce the scaling shift factor ( shift And recalculate the quantized value ( vals In other implementations, the sum function is used instead. abs (lfe_dct_new The maximum value function `max(abs()` can be used. lfe_dct_new To calculate the scaling and shift factors ())) shift However, using the max() function disperses the quantized values ​​more, making it more difficult to design an efficient entropy decoder.

[0079] In the quantization steps described above, the quantized values ​​of each frequency band group are calculated together in a loop, but the quantization point of each frequency band group is different. If the first frequency band group exceeds the allowable range, the scaling shift factor is reduced. If any of the other sub-frequency band groups exceeds the allowable range, the sub-frequency band group is reduced to... max_value For each frequency band group, the sign bits of all MDCT coefficients and the absolute values ​​of the quantized MDCT coefficients are decoded separately.

[0080] Figure 5The diagram illustrates the variation of fine quantization points with frequency according to one or more implementation schemes. In fine quantization, subband group 1 (0 Hz to 100 Hz) has 64 quantization points, subband group 2 (100 Hz to 200 Hz) has 32 quantization points, subband group 3 (200 Hz to 300 Hz) has 8 quantization points, and subband group 4 (300 Hz to 400 Hz) has 2 quantization points. In the embodiment, each subband group is entropy decoded using an entropy decoder (e.g., an arithmetic or Huffman entropy decoder), where each entropy decoder uses a different probability distribution. Therefore, the main 0 Hz to 100 Hz range is allocated the most quantization points.

[0081] Note that the quantization points assigned to subband groups 1 through 4 follow the shape of the LPF frequency response curve, which contains more information at lower frequencies than at higher frequencies and no information outside the cutoff frequency. To correctly reconstruct frequencies up to 130 Hz, the MDCT coefficients corresponding to frequencies above 130 Hz are also encoded to avoid or minimize aliasing. In some embodiments, MDCT coefficients up to 400 Hz are encoded so that frequencies up to 130 Hz can be properly reconstructed at the decoding unit.

[0082] Figure 6 The diagram illustrates the variation of coarse quantization points with frequency according to one or more implementation schemes. In coarse quantization, subband group 1 (0 Hz to 100 Hz) has 32 quantization points, subband group 2 (100 Hz to 200 Hz) has 16 quantization points, subband group 3 (200 Hz to 300 Hz) has 4 quantization points, and subband group 4 (300 Hz to 400 Hz) is unquantized and undecoded by entropy. In the embodiment, entropy decoding is performed on each subband group using separate entropy decoders with different probability distributions.

[0083] Figure 7 The diagram illustrates the probability distribution of quantized MDCT coefficients during fine quantization according to one or more implementation schemes. The y-axis represents the frequency of occurrence, and the x-axis represents the number of quantization points. Sg1 is subband group 1 corresponding to the quantized MDCT coefficients in the 0 Hz to 100 Hz band; Sg2 is subband group 2 corresponding to the quantized MDCT coefficients in the 100 Hz to 200 Hz band; Sg3 is subband group 3 corresponding to the quantized MDCT coefficients in the 200 Hz to 300 Hz band; and Sg4 is subband group 4 corresponding to the quantized MDCT coefficients in the 300 Hz to 400 Hz band.

[0084] Figure 8The diagram illustrates the probability distribution of quantized MDCT coefficients during coarse quantization according to one or more implementation schemes. The y-axis represents the frequency of occurrence, and the x-axis represents the number of quantization points. Sg1 is subband group 1 corresponding to the quantized MDCT coefficients in the 0 Hz to 100 Hz band; Sg2 is subband group 2 corresponding to the quantized MDCT coefficients in the 100 Hz to 200 Hz band; Sg3 is subband group 3 corresponding to the quantized MDCT coefficients in the 200 Hz to 300 Hz band; and Sg4 is subband group 4 corresponding to the quantized MDCT coefficients in the 300 Hz to 400 Hz band.

[0085] Note that the dominant frequency band (0 Hz to 100 Hz) is where the LFE effect is most observed and therefore receives more quantization points to achieve greater resolution. However, fewer bits are allocated to the dominant frequency band in coarse quantization compared to fine quantization. In this embodiment, whether a frame for MDCT coefficients uses fine or coarse quantization depends on the desired target bit rate set by the dominant audio channel encoder 103. The dominant audio channel encoder 103 sets this value once during initialization or dynamically frame-by-frame based on the number of bits required or used to encode the dominant audio channel in each frame.

[0086] Silent Frame

[0087] In some implementations, a signal is added to the LFE channel bitstream to indicate a silent frame. A silent frame is a frame with energy below a specified threshold. In some implementations, 1 bit is included in the LFE channel bitstream transmitted to the decoder (e.g., inserted in the frame header) to indicate a silent frame, and all MDCT coefficients in the LFE channel bitstream are set to 0. This technique can reduce the bit rate to 50 bps during a silent frame.

[0088] Decoder LPF

[0089] An implementation of LPF 207 (see LFE channel decoding unit 108) is provided at the output of the LFE channel decoding unit 108. Figure 2B Two options are available. LPF 207 is selected based on the available latency (total latency of other audio channels minus LFE fading latency minus input LPF latency). Note that it is expected that the other channels will be encoded / decoded by the main audio channel encoding unit 103 / main audio channel decoding unit 107, and the latency of the channels depends on the algorithm latency of the main audio channel encoding unit 103 / main audio channel decoding unit 107.

[0090] In the implementation, if the available delay is less than 3.5 ms, a second-order Butterworth LPF with a cutoff of 130 Hz is used; otherwise, a fourth-order Butterworth LPF with a cutoff of 130 Hz is used. Therefore, at the LFE channel decoding unit 108, a trade-off must be made between removing aliasing energy outside the cutoff frequency and algorithm delay. In some implementations, the LPF 207 can be completely removed because the woofer typically has an LPF. The LPF 207 helps reduce aliasing energy outside the cutoff of the LFE decoder output itself and can contribute to efficient post-processing.

[0091] Instance procedure

[0092] Figure 9 This is a flowchart of the process of encoding MDCT coefficients according to one or more implementation schemes 900. References can be used, for example... Figure 11 The system 1100 described is used to implement process 900.

[0093] Process 900 includes the following steps: receiving a time-domain LFE channel signal (901); filtering the time-domain LFE channel signal using a low-pass filter (902); converting the filtered time-domain LFE channel signal into a frequency domain representation of the LFE channel signal containing a certain number of coefficients representing the spectrum of the LFE channel signal (903); arranging the coefficients into a certain number of sub-band groups corresponding to different frequency bands of the LFE channel signal (904); quantizing the coefficients in each sub-band group using a scaling shift factor according to the frequency response curve of the low-pass filter (905); encoding the quantized coefficients in each sub-band group using an entropy decoder configured for the sub-band group (906); generating a bit stream containing the encoded quantized coefficients (907); and storing the bit stream on a storage device or streaming the bit stream to a downstream device (908).

[0094] Figure 10 This is a flowchart of the process 1000 for decoding MDCT coefficients according to one or more implementation schemes. For example, refer to [reference needed]. Figure 11 The system 1100 described is used to implement process 1000.

[0095] Process 1000 includes the following steps: receiving an LFE channel bitstream (1001), wherein the LFE channel bitstream contains entropy-decoded coefficients representing the spectrum of the time-domain LFE channel signal; decoding and inverse-quantizing the coefficients (1002), wherein the coefficients are quantized using a scaling shift factor according to the frequency response curve of a low-pass filter in sub-band groups corresponding to different frequency bands; converting the decoded and inverse-quantized coefficients into a time-domain LFE channel signal (1003); adjusting the delay of the time-domain LFE channel signal (1004); and filtering the delay-adjusted LFE channel signal using a low-pass filter (1005). In embodiments, the order of the low-pass filter can be configured based on the total algorithm delay obtained from a master codec that can encode / decode a multi-channel audio signal containing a time-domain LFE channel signal across the full bandwidth of the channels. In some implementations, the decoding unit only needs to know whether the encoding unit encodes the MDCT coefficients using fine quantization or coarse quantization. The quantization type can be indicated using bits in the LFE bitstream header or any other suitable signaling mechanism.

[0096] In some implementations, decoding of the inverse-quantized coefficients to time-domain PCM samples is performed as follows: The inverse-quantized coefficients in each frequency band group are rearranged into N groups (N is the number of MDCTs operated at the coding unit), where each group has coefficients corresponding to the corresponding MDCT. According to the exemplary implementation described above, the coding unit encodes the following four sub-frequency band groups:

[0097] Subband group 1 = {a1, a2, b1, b2}

[0098] Subband group 2 = {a3, a4, b3, b4}

[0099] Subband group 3 = {a5, a6, b5, b6}

[0100] Subband group 4 = {a7, a8, b7, b8}.

[0101] The decoding unit decodes the four subband groups and rearranges them back to {a1, a2, a3, a4, a5, a6, a7, a8} and {b1, b2, b3, b4, b5, b6, b7, b8}, then zero-padding is applied to the groups to obtain the desired inverse MDCT (iMDCT) input length. N iMDCT operations are performed to inversely transform the MDCT coefficients in each group into a time-domain block. In this example, each block is 2... Sw is the width of the subframe, where Sw is the subframe width defined above. Next, by... Figure 4The LFE coding unit shown uses the same Field window to window this block. Each subframe S is reconstructed by appropriately overlaying the windowed data of the previous iMDCT output with the current iMDCT output. i (i is 1<= i <= N (Integers between N). Finally, the output of (1003) is reconstructed by concatenating all N subframes.

[0102] Instantiated system architecture

[0103] Figure 11 It is an implementation reference based on one or more implementation schemes. Figures 1 to 10 A block diagram of system 1100 describing the features and processes. System 1100 includes one or more server computers or any client devices, including but not limited to: application servers, user equipment, conference room systems, home theater systems, virtual reality (VR) equipment, and immersive content acquisition devices. System 1100 includes any consumer devices, including but not limited to: smartphones, tablet computers, wearable computers, vehicle computers, game consoles, surround sound systems, information stations, etc.

[0104] As shown, system 1100 includes a central processing unit (CPU) 1101, which is capable of executing various processes based on a program stored in, for example, read-only memory (ROM) 1102 or a program loaded into random access memory (RAM) 1103 from, for example, storage unit 1108. Data required by the CPU 1101 during the execution of various processes is also stored as needed in RAM 1103. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0105] The following components are connected to I / O interface 1105: input unit 1106, which may include a keyboard, mouse, etc.; output unit 1107, which may include a display, such as a liquid crystal display (LCD) and one or more speakers; storage unit 1108, which includes a hard disk or other suitable storage device; and communication unit 1109, which includes a network interface card, such as a network card (e.g., wired or wireless).

[0106] In some implementations, the input unit 1106 includes one or more microphones located in different positions (depending on the host device) and capable of capturing audio signals in various formats (e.g., mono, stereo, spatial, immersive and other suitable formats).

[0107] In some implementations, output unit 1107 includes a system with a variety of numbers of speakers. Output unit 1107 (depending on the capabilities of the host device) can reproduce audio signals in various formats (e.g., mono, stereo, immersive, two-channel, and other suitable formats).

[0108] Communication unit 1109 is configured to communicate with other devices (e.g., via a network). Drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111 (e.g., disk, optical disk, magneto-optical disk, flash drive, or other suitable removable media) is mounted on drive 1110, such that computer programs read from removable media 1111 are loaded into storage unit 1108 as needed. Those skilled in the art will understand that although system 1100 is described as including the components described above, in practice, some of these components may be added, removed, and / or replaced, and all such modifications or alterations are within the scope of this invention.

[0109] According to exemplary embodiments of the present invention, the processes described above can be implemented as computer software programs or implemented on computer-readable storage media. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing methods. In these embodiments, the computer program can be downloaded and installed from a network via communication unit 1309, and / or installed from removable media 1111.

[0110] Generally, various exemplary embodiments of the present invention can be implemented as hardware or special-purpose circuitry (e.g., a control circuitry system), software, logic, or any combination thereof. For example, the units discussed above can be comprised of a control circuitry system (e.g., with...) Figure 11 The CPU, along with other components, executes the control circuitry, thus enabling the control circuitry system to perform the actions described herein. Some aspects may be implemented as hardware, while others may be implemented as firmware or software executable by a controller, microprocessor, or other computing device (e.g., the control circuitry system). While various aspects of exemplary embodiments of the invention may be illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that these blocks, devices, systems, techniques, or methods described herein may be implemented (by way of non-limiting example) in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers, or other computing devices or combinations thereof.

[0111] Furthermore, the various blocks shown in the flowchart can be viewed as method steps and / or operations implemented by computer program code, and / or as multiple coupled logic circuit elements constructed to perform associated functions. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code configured to perform the methods described above.

[0112] In the context of this invention, a machine / computer-readable medium can be any tangible medium that may contain or store a program used by an instruction execution system, device, or apparatus, or in combination with an instruction execution system, device, or apparatus. A machine / computer-readable medium can be a machine / computer-readable signal medium or a machine / computer-readable storage medium. A machine / computer-readable medium can be non-transitory and may contain, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing. More specific examples of machine / computer-readable storage media will include electrical connections having one or more wirings, portable computer disks, hard disks, RAM, ROM, erasable read-only memory (EPROM or flash memory), optical fibers, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] Computer program code that carries out the methods of this invention can be written in any combination of one or more programming languages. This computer program code can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device having a control circuitry system, such that when executed by the processor of the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be performed. The program code can be executed as a standalone software package entirely on a computer, partially on a computer, partially on a computer and partially on a remote computer, or entirely on a remote computer or server, or distributed across one or more remote computers and / or servers.

[0114] While this document contains numerous details of specific embodiments, such details should not be construed as limiting the scope of what can be claimed, but rather as descriptions of features specific to particular embodiments. Specific features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination and even initially claimed, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may be for sub-combinations or variations thereof. The logical flow depicted in the figures does not require the specific order or sequence shown to achieve the desired result. Additionally, other steps may be provided, or steps may be removed from the described flow, and other components may be added to or removed from the described system. Therefore, other embodiments are within the scope of the appended claims.

Claims

1. A method for encoding low-frequency effect (LFE) channels, comprising: Use one or more processors to receive time-domain LFE channel signals; The time-domain LFE channel signal is filtered using a low-pass filter to produce a filtered time-domain LFE channel signal, wherein the low-pass filter has a cutoff frequency. The one or more processors are used to convert the filtered time-domain LFE channel signal into a frequency domain representation of the time-domain LFE channel signal, which includes a certain number of coefficients representing the spectrum of the time-domain LFE channel signal. The coefficients are arranged into two or more subband groups corresponding to different frequency bands of the time-domain LFE channel signal using one or more processors, wherein the different frequency bands include a main LFE band below the cutoff frequency of the LFE loudspeaker and at least one other LFE band above the cutoff frequency of the LFE loudspeaker, wherein each subband group has a width, and the sum of the widths of the subband groups includes the main LFE band and the at least one other LFE band; The one or more processors are used to quantize the coefficients in each band group according to the frequency response curve of the low-pass filter to produce quantized coefficients, wherein the quantization point is different for each band group. The quantized coefficients in the sub-band group are encoded using one or more processors with an entropy decoder for each band group tuning; and The one or more processors are used to generate a bitstream containing the encoded quantized coefficients.

2. A low-latency, low-frequency-effect (LFE) decoder, comprising: One or more processors; as well as A non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the operation of claim 1 of the method.

3. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the operation of claim 1.

Images