Audio coding improvements

Abstract

A method and apparatus for improving the encoding and decoding of audio signals is disclosed. The method improves upon previously proposed precision-reduced, lossless encoding audio coding, improves the relationship between decoded sound quality and data rate, and addresses practical problems caused by unreliable data channels.

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for improving the encoding and decoding of audio signals. [Background technology]

[0002] Audio codecs typically reduce data rates by taking advantage of the following characteristics of audio: Spectrum: Power density usually decreases with increasing frequency. Tonality: Signal power is often concentrated in a narrow band. Dynamic range: The volume fluctuates over time, and there are quiet periods. ·Channel similarity

[0003] Additionally, the data rate may be reduced by approximation. A certain approximation error is allowed, the magnitude of which varies with time, frequency, and the desired quality level. A codec is said to be lossless if it does not use approximations and the decoded audio is an exact reproduction of the audio fed to the encoder. Linear predictive coding may be used to exploit the audio spectrum. Using a model of the audio spectrum, each audio sample is predicted from its previous values ​​and the prediction error (which is usually small) is transmitted.

[0004] In adaptive pulse code modulation (ADPCM), this level of prediction error is modeled and used to normalize the prediction error. The normalized prediction errors are assumed to have a relatively stable distribution and may be entropy coded. FLAC, an open-source lossless audio codec, implements this method using a fixed level of modeling for each audio block. Furthermore, the normalized prediction error may be reduced in precision by quantization, thereby stabilizing the data rate. This quantization may also be noise-shaped to distribute the approximation error over the spectrum, thereby reducing its audibility.

[0005] The parameters may be modeled in the encoder and the model may be transmitted to the decoder in the bitstream (forward adaptation). Alternatively, both the encoder and decoder may apply the same method to synchronously adapt their respective models to the audio (backward adaptation scheme).

[0006] Another approach in audio codecs is to separate the approximation stages. The first stage, prequantization, combines a process of coarsely quantizing the audio with noise shaping to reduce the audibility of the quantization while also reducing the data rate required to encode the audio. This precision-reduced audio is then transmitted using a lossless codec. This scheme can be applied in stages without further degradation of quality. Furthermore, the separation of the precision reduction process from the subsequent efficient audio encoding process makes it easier to properly implement both processes.

[0007] Generally, codecs that operate on each sample are called "time domain codecs." Time domain codecs are used in speech, telecommunications and other applications where low latency is required. Time domain techniques are also effective for lossless audio codecs such as FLAC.

[0008] However, for general applications involving wideband audio, the dominant approach is to start with a time-frequency transform. Instead of each sample representing a short time span and a wide band (eg, approximately 21 μs x 24 kHz), the transformed samples represent a long time span and a narrow band (eg, approximately 21 ms x 24 Hz with a 1024 point transform). The rationale for using such transforms is that most of the signal energy is often concentrated in a very small number of transformed samples, so it is possible to get a rough impression of the audio from these few values, and to design coding that takes advantage of this sparsity.

[0009] This approach works well within the range of data rates for which it is designed. However, as data rates increase and quality requirements increase, forcing values ​​to zero to create sparsity becomes overly coarse. When sparsity is lost, many of the advantages of working in the transform domain are lost, leaving several disadvantages: Large transforms for good frequency resolution require large block sizes. The overall encoding and decoding delay is usually several blocks, resulting in a large minimum delay that makes the codec unsuitable for many real-time applications. The codec is based on a specific fixed-size transform, meaning the block size cannot be adjusted to suit the application, resulting in a lack of customizability. -Conversion processing requires implementation costs. The audible effect of block-based processing spreads naturally over the entire window in which that block is decoded. This can cause energy from a transient event to travel backward in time, heralding its arrival to the listener. If the noise floor varies with frequency, a scale factor must be transmitted to the decoder. This process consumes data rate and imposes constraints on the signal shape to fit a predefined model, for example, one scale factor per critical band.

[0010] Higher data rates are becoming more widely used, and therefore there is a need for improved time-domain audio codec techniques that can be used at high sampling rates (44.1 kHz and above) and high data rates (256 kbps and above for two channels). This allows for superior sound quality compared to conventional frequency domain codecs, while offering the advantages of low latency and reduced computational effort.

[0011] Real-world data communication channels often have varying capacities. For example, a wireless channel may be subject to intermittent interference. Therefore, audio codecs are required to seamlessly adapt to reduced channel capacity, degrading sound quality when necessary, while still operating without dropouts, clicks, or malfunctions. As can be seen, therefore, there is a need for improved encoding and decoding methods for audio signals, and related encoders, decoders, and codecs. [Prior art documents] [Non-patent literature]

[0012] [Non-Patent Document 1] MAGerzon and PGCraven “Lossless coding method for waveform data”, WO1996037048A2. [Non-patent document 2] PGCraven and JRStuart, “Cascadable Lossy Data Compression Using a Lossless Kernel”, preprint 4416 102nd AES convention 1997. [Non-patent document 3] LG Roberts “Picture Coding Using Pseudo-Random Noise,” IRE Trans. Inform. Theory, vol. IT-8, pp. 145-154 1962. [Non-patent document 4] MAGerzon and PGCraven, “Optimal noise shaping and dither of digital signals” preprint 2822 87th AES convention 1989. [Non-Patent Document 5] MAGerzon and PGCraven, “Compatible Improvement of 16-Bit Systems Using Subtractive Dither” preprint 3356 93rd AES convention 1992. [Non-patent document 6] JR Stuart, “Noise: Methods for Estimating Detectability and Threshold” JAES Volume 42 Issue 3 pp. 124-140; March 1994. Summary of the Invention [Problem to be solved by the invention]

[0013] definition As used in this specification and the appended claims, the following terms shall have the following meanings:

[0014] JPEG2025540705000002.jpg48170

[0015] <signal domain> Refers to the class of input audio signal. For example, 24-bit audio forms the signal domain. In this case, it refers to audio where the sample values ​​for each channel are represented as 24-bit signed integers. A signal region is said to be "smaller" than another signal region when there are fewer possible signals per sample. For example, 16-bit audio has a smaller signal range than 24-bit audio. Note that the signal domain does not necessarily apply to the entire signal. Different audio blocks within a signal may belong to different signal domains, and audio channels within the same audio block may belong to different signal domains.

[0016] <signal domain family> A set of signal regions defined by certain parameters. For example, "n-bit audio" refers to a family of signal domains in which each sample value is represented by an n-bit signed integer. Or equivalently, each sample value is Δ=2 24-n It refers to a group of signal regions expressed as 24-bit signed integers that are divisible by a stepsize expressed as

[0017] <Lossless codec> A codec that operates in the signal domain and consists of an encoder and a decoder connected to it. It has the property that if any input audio from the signal domain is fed to the encoder, the decoder will output a reproduction of the input audio.

[0018] <Lossless Encoder> An encoder that operates in the signal domain. It has the property that any data output can only be generated by at most one input audio signal within its signal domain. This property means that the encoder preserves information about the signal it encodes in an indestructible manner, and the process can be reversed by the decoder.

[0019] <Lossless decoder> A decoder that operates in the signal domain. It has the property that any audio output within the signal domain can be generated by any data input. This property guarantees the existence of an encoder that corresponds to the decoder and constitutes a lossless codec in the signal domain.

[0020] Exploits for compression A lossless codec is said to use a signal domain group parameter for compression if, when the same audio signal is quantized into different signal domains within the signal domain group, the coded data size varies depending on the parameter, with smaller signal domains resulting in smaller data sizes. [Means for solving the problem]

[0021] According to a first aspect of the present invention, there is provided a method for encoding input blocks of audio into data packets, each input block containing one or more channels of audio samples. The method for encoding an input block of audio into data packets comprises: receiving an input block of said audio; determining a quantization step size Δ for each audio channel within each block based on a rate control mechanism; determining a pseudo-random offset for each sample in the input block, the pseudo-random offset for each channel forming a pseudo-random sequence with a seed; quantizing each sample in the input block with noise shaping to obtain a pre-quantized block, wherein each sample value in the pre-quantized block is congruent modulo Δ with a corresponding pseudo-random offset; losslessly encoding the pre-quantized block based on Δ using a lossless encoder to obtain a lossless encoded data block, wherein the dependence on Δ is such that smaller values ​​of Δ result in larger sizes of the lossless encoded data block, and the lossless encoding is an injection mapping such that for any pre-quantized block, losslessly encoding another pre-quantized block that is also congruent modulo Δ with the corresponding pseudo-random offset necessarily results in a different block of lossless encoded data; a buffering step of buffering the losslessly encoded data block in a buffer; generating data packets for subsequent transmission based on the buffered data, at least some of the data packets including data representing the seed of the pseudo-random number sequence.

[0022] In this way, the rate control mechanism can adjust the level of approximation error throughout the stream and selectively direct the approximation error into areas where it is less acoustically noticeable. Pseudorandom offsets have the advantage of avoiding quantization distortion while also avoiding the increased approximation error associated with additive dither.

[0023] Furthermore, a lossless encoder needs to exploit Δ to improve compression efficiency. That is, the loss of signal precision due to the quantizer is reflected as an appropriately lower data rate. Lossless encoders that are not adapted to pseudo-random offsets cannot utilize Δ to improve compression efficiency. This is because, for such an encoder, the input signal always appears to be highly accurate, regardless of the value of Δ.

[0024] Finally, the encoding method is configured so that the decoder can reproduce the same pseudo-random sequence as the pre-quantizer. The data representing this seed may be in a simple form, for example a block count index (modulo a power of two), which is sufficient for a decoder to quickly skip to a particular position within a standardized pseudo-random number sequence.

[0025] Preferably, the rate control mechanism receives information about the buffer and the quantization step size Δ is determined based on the fullness of the buffer. In this way, the encoded data rate is servo-controlled to stabilize the buffer occupancy and adjust the lossless encoded data rate to match the channel data rate.

[0026] The encoding method further comprises a step of separating the lossless encoded data in each lossless encoded data block into a first portion and a second portion and storing the first and second portions separately in the buffer storing step; the first portion includes base layer data and the second portion includes enhancement data, and decoding the base layer data without the enhancement data results in an approximation of the pre-quantized block; The data packets are generated such that each packet contains an integer number of base layer data blocks, with any free space being filled with enhancement data.

[0027] In this way, if the decoder has trouble recovering the buffered data, it is still able to produce an approximation of the audio rather than no output at all. Meanwhile, buffering is still available to decouple the variable data rate in lossless coding from the characteristics of the data channel.

[0028] Preferably, the enhanced data is stored in a first-in-first-out (FIFO) buffer and the data packets are generated using base layer data blocks from one end and the FIFO buffered enhanced data from the other end.

[0029] In this way, the decoder can access the enhancement data and decode the first block in a packet before it has finished parsing the base layer data for all blocks in the packet. This can be achieved without consuming data rate for a length field indicating the length of the entire base layer data.

[0030] The encoding method further comprises an analysis step of analyzing samples in the input block, and the quantization step size Δ is further determined based on the analysis of the samples. Preferably, if it is determined in the analysis step that there is a risk of buffer overflow, the quantization step size Δ is increased. This allows the encoder to predict and avoid buffer overflows, allowing the codec to safely operate with less buffering and shorter codec delays.

[0031] According to a second aspect of the present invention, there is provided an encoder configured to encode input blocks of audio into data packets using the encoding method of the first aspect.

[0032] According to a third aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the first aspect.

[0033] According to a fourth aspect of the present invention, there is provided a method for decoding data packets into output blocks of audio comprising one or more channels of output audio samples. The method for decoding audio into output blocks comprises: receiving the data packet; extracting, for each channel and block based on the data of the data packet, information indicative of a quantization step size Δ and a seed; determining an offset for each sample in a block, the offset for each channel being a pseudo-random sequence based on the corresponding seed; decoding the data to obtain an innovation sample (prediction error) for each sample in a block based on the data; filtering the innovation sample with a quantization process to generate a filtered sample corresponding to each sample in the block, the filtered sample being based on a corresponding innovation sample and also congruent modulo Δ with a corresponding offset; generating an output block of audio based on the filtered samples.

[0034] In this way, by extracting the Δ and the seed, the decoder can establish the quantization characteristics of the audio input to the lossless encoder and adjust its output to conform thereto. Additionally, the decoder expands the quantization characteristics of the audio input to the lossless encoder into specifications for each sample by generating a pseudo-random sequence (this may not apply to all channels in all blocks, and some streams may specify that some channels in a particular block do not use pseudo-random offsets). Finally, the decoder ensures that each filtered sample conforms to its quantization specification. In presenting such a lossless decoder architecture, the filtering step is neither the first nor the last step. Therefore, a step of decoding the innovation samples is provided before filtering, and the filtering step is coupled to the output.

[0035] A first portion of each data packet is decoded without delay, and a second portion of each data packet is buffered and delayed before decoding. In this way, the decoder applies a complementary delay to that applied by a corresponding encoder embodiment, allowing it to decode an approximation of the audio rather than no output at all if there is a problem recovering the buffered data.

[0036] According to a fifth aspect of the present invention there is provided a decoder arranged to decode data packets into blocks of audio using the method for decoding into output blocks of audio of the fourth aspect.

[0037] According to a sixth aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the fourth aspect.

[0038] According to a seventh aspect of the present invention, there is provided a codec comprising a combination of an encoder according to the second aspect and a decoder according to the fifth aspect.

[0039] According to an eighth aspect of the present invention, there is provided a method of encoding audio into data. The method for encoding audio into data comprises: receiving input blocks of audio, each input block containing one or more channels of audio samples quantized at the input audio precision; determining a pre-quantization precision for each channel of each block, where a block has one or more channels, and the pre-quantization precision is coarser than the input audio precision; generating a pre-quantized block, and if the pre-quantization resolution is coarser than the input audio resolution, generating the pre-quantized block by quantizing each sample in the input block to the pre-quantization resolution with noise shaping having a noise transfer function that follows a noise equal loudness curve in a corner frequency range of 1 kHz to at least 13 kHz; Losslessly encoding the pre-quantized block to generate a losslessly encoded data block.

[0040] In this way, the noise introduced by the quantization process is shaped into an unobtrusive, gentle curve below the threshold frequency, without perceptually emphasizing certain frequency regions more than others. Above the threshold frequency, the equal loudness contours rise sharply, but it is not useful to follow this rise too closely. On the other hand, although the equal loudness contours rise below the threshold frequency, strictly following this rise does not provide much noise shaping effect.

[0041] Preferably, the corner frequency is at least 15 kHz.

[0042] In the frequency band exceeding the corner frequency, the gain of the noise transfer function forms a flat portion. In this way, the power of the overall approximation error is reduced.

[0043] In another embodiment, the gain of the noise transfer function peaks and then decays in the frequency band above the corner frequency. Typically, attenuating the gain in the high frequency band where the signal level is low reduces the likelihood that the signal will be drowned out by noise.

[0044] In the frequency band above the corner frequency, the gain of the noise transfer function varies depending on the input block. This makes it possible to adjust the approach to high frequencies depending on the level of signal power of the high frequency components that actually exist. The gain of the noise transfer function preferably follows the smoothed spectrum of the input audio. Following the smoothed spectrum of the input audio allows operation at a desired signal to approximation error ratio, which allows a selected bit rate to be allocated in the frequency domain.

[0045] According to a ninth aspect of the present invention there is provided an encoder arranged to encode audio into data using the method of the eighth aspect.

[0046] According to a tenth aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the eighth aspect.

[0047] According to an eleventh aspect of the present invention, there is provided a method for reducing audible transients occurring when noise shaping on an audio signal is stopped. The method includes a quantization sample value changing step of changing next n quantization sample values, the quantization sample value changing step comprising: multiplying the state variables of the noise shaper and / or the difference between one or more previous outputs and corresponding inputs of the noise shaper by a pre-computed matrix to obtain an intermediate representation comprising up to n values; quantizing the n or fewer values ​​in the intermediate representation using direct or backsubstitution to generate n or fewer quantized intermediate values; multiplying the n or fewer quantized intermediate values ​​by a matrix of pre-computed integer values ​​to generate n modifications to the quantized sample values; applying the n modification values ​​to the quantized sample values.

[0048] In this way, an elegant solution to the difficult simultaneous rounding problem is achieved that reduces potentially audible artifacts. The difficult linear algebraic aspects of this problem for a particular frequency weighting are pre-computed. This allows real-time solution for a particular instance when the noise shaper is stopped to be performed by simple matrix operations.

[0049] According to a twelfth aspect of the present invention there is provided an apparatus configured to reduce audible transients occurring when noise shaping of an audio signal is stopped using the method for reducing audible transients occurring when noise shaping of an audio signal of the eleventh aspect.

[0050] According to a thirteenth aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the eleventh aspect.

[0051] According to a fourteenth aspect of the present invention, there is provided a method for generating a compressed bitstream by losslessly compressing an audio signal comprising one or more channels. The method comprises, for each channel: receiving a sequence of audio samples, each audio sample having a quantized value that is an integer multiple of a corresponding step size Δ plus a corresponding pseudo-random offset; predicting the value of each audio sample by filtering previous audio sample values; generating a sequence of innovation samples by subtracting a corresponding predicted value from the value of each audio sample; generating a sequence of integer innovation samples by performing a rounding division on each innovation sample by a corresponding step size Δ; and generating a symbol based on the integer innovation sample, the method further comprising: generating base layer data by entropy encoding the symbols from all channels; and generating the compressed bitstream based on the base layer data.

[0052] In this way, the audio quantized to the pseudo-random offsets is compressed using a step size Δ, allowing efficient lossless coding. Such an encoder is desirable. This is because quantizing a pseudo-random offset avoids the distortion concerns that arise from quantizing to a fixed number of bits, while also not increasing quantization noise due to dithering.

[0053] The sequence of audio samples is received as blocks of audio samples, with the audio samples within a block being quantized using a different value of step size Δ than the audio samples within one or more other blocks. In this way, a lossless encoder can efficiently handle audio with varying degrees of quantization across blocks, which is desirable when encoding over fixed-rate data links.

[0054] And the method further comprises embedding information in the compressed bitstream specifying the corresponding step size Δ and the corresponding pseudo-random offset for the audio samples. In this way, the lossless encoder can communicate this important configuration information to the decoder in-band, rather than via a side channel.

[0055] In some embodiments, there is more than one channel. The audio samples of one channel may be quantized with a different pseudo-random offset than the audio samples of another channel. In this way, the pseudo-random offsets in different channels can be made independent of each other. If they are identical, there will be substantially no offset in the quantized difference signal between the two channels. The step size Δ used in one channel may be different from the step size Δ used in another channel. In this way, higher quantization accuracy can be applied to full-band channels such as Left and Right. On the other hand, for channels such as LFE (low frequency effects), a lower quantization precision can be applied because the playback system has a low-pass characteristic and can tolerate a larger average approximation error.

[0056] The symbol generating step includes performing a further rounding division on each integer innovation sample, and the compressed bitstream generating step is also based on the remainder of the further rounding division. In this way, the data representing the symbols can be processed without being buffered, while the extension data can be buffered. The symbol generating step may include adding the remainder of the further rounded division to a subsequent integer innovation sample. In this way, the sound effects of the extension data can have a high-pass characteristic, improving the fidelity of the audio represented by the symbols alone. This improves the reconstruction quality even when the enhancement data cannot be recovered by the decoder.

[0057] According to a fifteenth aspect of the present invention, there is provided an encoder configured to losslessly compress an audio signal comprising one or more channels to generate a compressed bitstream using the method of the fourteenth aspect. In this way, a lossless encoder can be constructed that takes advantage of the above method.

[0058] According to a fifteenth aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the fourteenth aspect. In this way, lossless coding that takes advantage of the above advantages can be performed on a computer.

[0059] According to a sixteenth aspect of the present invention, there is provided a method for decoding a bitstream into an audio signal having one or more channels. The method comprises: receiving a compressed bitstream together with a specification of a step size Δ and a specification of a pseudo-random offset; entropy decoding a portion of the compressed bitstream to generate a sequence of decoded symbols for each channel; an integer innovation sample sequence generation step for generating, for each channel, a sequence of integer innovation samples based on the decoded symbols for that channel; generating, for each channel, a sequence of predicted samples; generating, for each channel, a sequence of pseudo-random offsets based on said pseudo-random offset specification; and calculating, for each channel, a sequence of audio samples, said calculating step comprising: multiplying each integer innovation sample in the sequence of integer innovation samples by a corresponding step size Δ to obtain a multiplication result; adding the corresponding predicted sample to the multiplication result to obtain an addition result; quantizing the sum to a value congruent to the corresponding pseudo-random offset modulo the corresponding step size Δ; Each prediction sample in the sequence of prediction samples is generated by filtering a previously calculated audio sample.

[0060] In this way, lossless decoding can be performed as part of a lossless codec that operates efficiently by compressing the pseudo-random offset quantized audio using Δ. Such a codec, and therefore a decoder, is desirable because quantizing to a pseudo-random offset avoids the distortion concerns that arise from quantizing to a fixed number of bits with the least significant bits being zero.

[0061] Preferably, one or more of said specifications are decoded from said compressed bitstream. In this way, these decoding parameters can be obtained from the bitstream rather than receiving configuration information from a side channel.

[0062] And the specification of the step size Δ allows Δ to take one or more different values. In this way, lossless codecs can efficiently accommodate audio with varying degrees of quantization from block to block, which is desirable when transmitting data over fixed-rate data links.

[0063] In some embodiments, one or more channels may be designated. For different channels, the sequence of pseudo-random offsets may be different. In this way, the pseudo-random offsets in different channels can be made independent of each other. If the offsets are identical, there will be substantially no offset in the quantized difference signal between the two channels. The step size Δ used in one channel may be different from the step size Δ used in another channel. In this way, higher quantization accuracy can be applied to full-band channels such as the left and right. On the other hand, for channels such as LFE, a lower quantization precision can be applied since the playback system has a low-pass characteristic and is less sensitive to approximation errors.

[0064] The integer innovation sample sequence generation step is also based on extension data decoded from another portion of the bitstream. In this way, the symbol data can be handled without buffering, while the extension data can be buffered. The dependence on the extended data may include adding and subtracting values ​​to successive samples. In this way, the sound effects of the extension data are given a high-pass characteristic, improving the fidelity of the audio represented by the symbols alone. This improves the reconstruction quality even when the extended data cannot be restored.

[0065] According to a seventeenth aspect of the present invention there is provided a decoder arranged to decode a bitstream into an audio signal having one or more channels using the method of the sixteenth aspect. In this way, a decoder can be constructed that takes advantage of the present method.

[0066] According to an eighteenth aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the sixteenth aspect. In this way, the method making use of the above advantages can be executed on a computer.

[0067] According to a nineteenth aspect of the present invention, there is provided a codec comprising a combination of the encoder of the thirteenth aspect and the decoder of the seventeenth aspect.

[0068] According to a twentieth aspect of the present invention, there is provided a method for losslessly compressing a sequence of audio samples from an audio signal having one or more channels and converting them into data packets. The method for converting into data packets includes: dividing the sequence of audio samples into a sequence of audio blocks, each audio block comprising a plurality of audio samples; encoding each audio block into a data block and an enhancement block; generating a sequence of data packets, each data packet containing data from an integer number of data blocks and extension blocks; the data blocks contain information that allows an approximate reconstruction of the audio signal, the combination of the data blocks and the extension blocks contains information that allows a complete reconstruction of the audio signal; For all block indices t, Data block t is not included in a data packet after data block t+1, Any data contained in extension block t+1 is not contained in a data packet that precedes any data contained in extension block t; Any data contained in extension block t is not included in any data packets after data block t.

[0069] In this way, block-based coding is decoupled from packetization, making it possible to apply a single lossless coding scheme to a variety of data transmission schemes with different characteristics. Layered coding of base layer data blocks and enhancement data ensures that each packet is associated with a specific data block, while the enhancement data is buffered, allowing for isolation of the inherently rate-varying lossless coding and channel characteristics. Also, by ensuring that the extension data is not sent after the corresponding data block, the packet can be fully decoded as soon as it is received.

[0070] The integer number of data blocks contained in a data packet is not constant for all data packets. In this way, the repetition period of the packets can be decoupled from the duration of the blocks.

[0071] And, in one or more data packets, the integer number of data blocks is zero. In this way, packet repetition periods shorter than the block duration can also be accommodated.

[0072] According to a 21st aspect of the present invention, there is provided an encoder configured to losslessly compress and convert a sequence of audio samples from an audio signal having one or more channels into data packets using the method of the 20th aspect of the present invention. In this way, an encoder can be constructed that takes advantage of this method.

[0073] According to a 22nd aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the 20th aspect of the present invention. In this way, the method having the above advantages can be executed on a computer.

[0074] According to a twenty-third aspect of the present invention, there is provided a method for decoding a sequence of data packets into audio samples in one or more channels. The method comprises: receiving a data packet in the sequence of data packets and parsing an integer number of data blocks and bufferable data from the data packet; placing the bufferable data into a first-in, first-out (FIFO) buffer; and decoding each data block in turn into audio samples using the enhanced data obtained from the FIFO buffer.

[0075] In this way, data blocks can be decoded as soon as the packets are received, while the FIFO buffering of the extended data isolates the channel from the variable rate characteristics inherent in lossless coding.

[0076] And, the integer number of data blocks parsed from a data packet is not constant for all data packets in the sequence of data packets. In this way, the repetition period of the packets can be decoupled from the duration of the blocks.

[0077] In one or more data packets in the sequence of data packets, the integer number of data blocks parsed from the data packet is zero. In this way, packet repetition periods shorter than the block duration can also be accommodated.

[0078] According to a 24th aspect of the present invention there is provided a decoder arranged to decode a sequence of data packets into audio samples in one or more channels using the method of the 23rd aspect. In this way, a decoder can be constructed that takes advantage of the present method.

[0079] According to a 25th aspect of the present invention there is provided a computer readable medium storing instructions which, when executed by one or more processors, cause said one or more processors to perform the method of the 23rd aspect. In this way, the method having the above advantages can be executed on a computer.

[0080] According to a 26th aspect of the present invention, there is provided a codec comprising a combination of the encoder of the 21st aspect and the decoder of the 24th aspect.

[0081] As will be appreciated by those skilled in the art, the present invention may take on a variety of embodiments depending on the application, as will become apparent from the following description. [Brief explanation of the drawings]

[0082] Embodiments of the present invention will now be described with reference to the following drawings. [Figure 1] 1 is a diagram illustrating the main components of an audio encoder 101 according to the invention and how these components may be connected. [Figure 2] 1 is a flow chart illustrating the operation of an audio encoder according to the present invention. The data packets generated by the audio encoder are not constrained to contain a fixed number of audio blocks. Thus, the presentation of audio blocks 150 occurs asynchronously with the extraction of data packets 160, and these processes are coordinated by data buffering. [Figure 3] 1 shows an overview of the main components of an audio decoder according to the invention; [Figure 4] Two equivalent architectures for noise-shaping quantization using pseudo-random offsets for integer multiples of step size Δ. In (a), the offset 402 is added or subtracted just before and just after the main quantizer 413, while in (b), the offset is added or subtracted before and after the entire noise-shaping quantizer. These two architectures (and further rearrangements) are computationally equivalent. [Figure 5] (a) Diagram showing an example of a modification of the prior art proposal to pre-quantize (500) audio and then encode it with a lossless codec (501) using subtractive dither. Pseudo-random dither (510) is added before quantization, and its synchronized representation (511) is subtracted at the decoder side. This added signal energy reduces the efficiency of the lossless codec, but noise shaping (520) can be used to mitigate this inefficiency, although the noise shaping (521) must also be reproduced at the decoder side. [Figure 5] (b) A diagram showing an example of an improvement over the prior art proposal of "pre-quantizing audio and then encoding it with a lossless codec" using a pseudorandom offset. The pseudorandom dither 510 is added and subtracted before and after the pre-quantizer 500 so that its energy does not impair the efficiency of the lossless codec. However, the lossless codec 502 must be adjusted to reflect that the samples being coded have a non-zero modulo Δ offset. [Figure 6] A diagram showing various noise-shaping transfer functions useful for pre-quantization, plotted against frequency (Hz) and amplitude (dB) on the vertical axis. Between the vertical lines (1 kHz and 15 kHz), all functions have similar shapes, following the shape of equal loudness contours appropriately adjusted for noise. [Figure 7] 7 shows the concept of constructing a least-squares model that minimizes the auditory perceptibility of artifacts when the noise shaping process is turned off. The original audio 700 is replaced with selected quantized audio 701. The difference dt between the two is passed through filter 702 to produce a frequency-weighted error signal e, which is measured by power meter 703. [Figure 8] 8 is a flowchart showing a set of processing steps for minimizing the auditory perceptibility of artifacts when the noise shaping process is stopped. At run time, a specific instance of the problem needs to be solved (810). This simply involves a simple matrix operation using pre-computed matrices to generate an appropriate set of modifications to the last modifiable audio value that will affect the filter state and minimize the perceptibility at that point. At design time, these pre-computed matrices are designed (800) based on a specification of the relative weighting of the errors at each frequency (801). [Figure 9] 10 is a flowchart illustrating a method for analyzing an audio block to estimate how the coding bit rate varies depending on the pre-quantization configuration. [Figure 10] 1 is a diagram showing main signal processing steps in a lossless encoder according to the present invention and the flow of data between the steps. [Figure 11] FIG. 2 is a diagram showing the main signal processing steps in a lossless decoder according to the present invention and the data flow between the steps. [Figure 12] 1 illustrates an example packet format for communication between an encoder and a decoder according to the present invention. The packet contains base layer data describing an integer number of audio blocks, and the remainder of the packet is filled backwards with buffered enhancement data. The enhancement data is packetized without regard to block boundaries, resulting in partial fragments at each end. [Figure 13] FIG. 10 illustrates how the synchronization field in the packet header is synchronized with the decoder's FIFO buffer. [Figure 14] Diagram showing how to handle FIFO buffer underflow. (a) Base layer blocks are shown flowing into the delay line from a lossless encoder, and enhancement data is shown flowing into the FIFO buffer. Two data packets 1400 and 1402 are output. Packet 1400 contains a hole 1450 caused by an underflow in the encoder-side FIFO buffer. (b) The data for these packets is shown flowing into the decoder's FIFO buffer, showing how the decoder estimates where the hole 1450 is in the data. [Figure 15] 10 is a flow chart illustrating how a rate control servo incorporates desirable acoustic considerations. DETAILED DESCRIPTION OF THE INVENTION

[0083] In Non-Patent Document 1 (pp. 67-71), Gerzon and Craven propose the construction of a lossy audio codec consisting of an initial pre-quantization stage to reduce audio precision, followed by a lossless audio codec. Craven and Stuart also propose a similar method in Non-Patent Document 2.

[0084] When we implemented this concept, we found that by making the following improvements, we could achieve sound quality that exceeds that of state-of-the-art audio codecs at high sampling rates (44.1 kHz and above) and high data rates (256 kbps and above). Furthermore, such high audio quality can be achieved with low delay and low computational load on the encoder and decoder. The resulting codec also has the ability to seamlessly switch between lossy operation at these data rates and lossless operation at higher data rates.

[0085] The main advantage of separating the lossy encoder into a pre-quantizer and a lossless encoder is the separation of concerns. A pre-quantizer can focus on reducing the precision (and therefore entropy) of the audio, allowing signal processing to be done with care to achieve high-quality results, while a lossless codec does not alter the audio during normal operation, eliminating concerns about sound quality. Therefore, we can focus on encoding the audio with the minimum amount of data while maintaining computational efficiency.

[0086] The second advantage is cascadability. The decoded audio is an exact reproduction of the audio input to the lossless encoder. Therefore, the decoded audio can be recompressed again at the same data rate without pre-quantization again and without introducing new approximation errors.

[0087] An interesting application of cascade connectivity is streaming from a smartphone with wireless retransmission to earphones. Streaming typically occurs at the data rate the wireless channel can accommodate, but if the wireless environment deteriorates, the smartphone can requantize to a lower-quality representation with a coarser resolution, and then revert to lossless retransmission when the wireless environment improves.

[0088] Although it is preferable to separate the pre-quantizer and lossless encoder, it is entirely possible to restructure the signal processing to integrate the data-reducing quantization process into the lossless encoder process, resulting in an all-in-one lossy encoder.

[0089] <Outline of a typical encoder configuration> The general structure of the encoder is shown schematically in FIG. 1 and in flow chart form in FIG. First, the configuration will be described with reference to FIG.

[0090] Input digital audio representing one or more channels is input to the encoder 101 in blocks 120 . The size of the blocks is configurable, but preferably represents approximately 1-2 milliseconds of audio. Smaller blocks provide more flexibility to dynamically adjust the degree of approximation error depending on the audio. On the other hand, the data overhead in a losslessly coded stream increases, and the computational cost also increases because the encoder makes more frequent decisions.

[0091] Each audio block is then pre-quantized (102) to produce pre-quantized audio 121. At this stage, the audio precision is reduced so that the encoded data rate matches the capabilities of the transmission channel. Lossless operation is also possible if the channel capacity is sufficient, in which case the pre-quantizer passes some or all of the channel components within an audio block unchanged. But typically the audio is quantized to a reasonable precision using pseudo-random offsets and noise shaping. The pseudorandom offset makes the approximation error noise-like (rather than distortion), and the noise shaping adjusts the spectral shape of the approximation error to minimize its auditory perceptibility.

[0092] The required pseudorandom offset is provided by pseudorandom offset unit 106 . This is standardized because the decoder also needs to reproduce these pseudo-random offsets.

[0093] The pre-quantizer preferably includes other signal processing functions to reduce the encoded data rate, such as reducing the sampling rate or converting multiple independent audio channels to mono. These features are useful for dealing with situations where the channel capacity drops suddenly. For example, if someone in the family starts watching a high-definition video, interference may occur in the wireless link.

[0094] The pre-quantized audio 121 is then sent to a lossless encoder 103 . A lossless encoder is responsible for converting each audio block into a data block such that a corresponding decoder can accurately reproduce the audio block. Lossless codecs exploit known audio properties to reduce the encoded data rate. In Non-Patent Document 1, Gerzon and Craven envisaged the use of a general-purpose lossless audio codec, the design of which was the subject of the document. However, in reality, conventional lossless codecs (currently dominated by FLAC) are not suitable because they lack various specialized features that are desirable for improving overall system performance. In particular, the lossless encoder must be configured to work in conjunction with the pseudo-random offset. Otherwise, the apparently high-precision audio input may cause the lossless encoder to operate at an undesirably high data rate.

[0095] The coded blocks are then sent to the packet generator 104 . The packet generator 104 is responsible for generating the actual packets 124 that are transmitted over the communication channel. The process of forming coded blocks into packets could reasonably be considered part of a lossless encoder, but is described here as a separate component because it plays a unique role within the overall encoder. The size of the coding blocks varies, especially when operating losslessly. For some channels, such as file storage, this variation is not an issue. However, many real-time communication channels require packets to be output from the encoder at a fixed or peak-limited data rate. For example, there are cases where the packet size is constant and a minimum interval is set between packets, or where packets must be output according to a predetermined schedule and a maximum packet size is set.

[0096] The packet generator preferably includes buffering 108 to accommodate mismatches between the inherently varying data rate from the lossless encoder and the fixed or peak-limited data rate of the channel. If the lossless encoder is producing blocks containing more data than the available data rate, the buffer will fill up, and conversely, if it is producing short encoded blocks, the buffer will empty.

[0097] In some embodiments, the peak rate of output may not be limited, for example, for codecs intended for file-to-file encoding. In that case, the buffer 108 may be omitted since there is no short-term capacity limit that requires buffer storage.

[0098] Although it is possible to buffer the entire data stream, it is preferable for the lossless encoder to output it in two parts. One (hereafter base layer data 122) can be decoded by itself into a coarse representation of the audio. The other (hereafter referred to as extension data 123) comprises additional data that can be combined with the base layer data to allow lossless reconstruction. The base layer data is subjected to a certain delay (hereinafter referred to as latency) in a delay line 110 within the buffer 108 . The extended data undergoes a variable delay in a first-in-first-out (FIFO) buffer 109 ranging from 0 to the latency. This variable delay allows the output data rate of the lossless encoder to be decoupled from the capacity of the communication channel. On the communication channel, the enhancement data precedes the base layer data by a variable amount of time ranging from 0 to the latency.

[0099] The packet generator is also preferably controlled by transmission information 132 which specifies the frequency and size of packets to be sent. As environmental conditions change, the available bandwidth may also fluctuate, so it is useful for the encoder 101 to be able to accommodate these changes. A user data input 133 is also shown, since from time to time there may be an opportunity to transmit externally supplied, non-time critical data in packets.

[0100] The buffer 108 is configured so that its degree of filling can be measured, and this degree of filling is called the buffer stress 130 . This measurement is sent to a rate control servo 105 . The rate control servomechanism is responsible for closing the feedback loop. Fine (or lossless) quantization of audio results in large coded blocks output from the lossless encoder, filling the buffer and increasing buffer stress. On the other hand, coarser quantization results in smaller coded blocks, which reduces buffer content and buffer stress. The rate control servo preferably sends instructions 131 to adjust the degree of quantization performed by the pre-quantizer, taking into account the audible effect of changes in quantization precision, so that buffer stress is kept within acceptable limits.

[0101] Sometimes, when the codec operates at low latency and there is little buffer storage available, the feedback mechanism alone may not be enough to prevent the buffer from overloading. Audio has a large dynamic range; periods of quiet, tranquil, finely quantized audio may be immediately followed by loud, high-entropy blocks such as cymbal crashes. If this block were to be quantized as finely as the previous audio block, the lossless encoder would output a very large amount of data, potentially overwhelming the buffer.

[0102] The incoming audio block is preferably analyzed (107) to estimate the relationship between the quantization step size and the number of bits in the coded block, and this information is also taken into account by the rate control servo 105. In this type of rate control, it is likely that many designers will choose to use the analysis results of the current block as the primary rate control mechanism, with feedback from buffer stress being at most a secondary factor. However, for reasons that will be explained later, the inventors believe that better results in terms of sound quality can be obtained by focusing on buffer stress when selecting the degree of quantization. The analysis results for the current block should preferably be ignored unless they indicate that the buffer is at risk of bursting if urgent action is not taken.

[0103] The flowchart in Figure 2 shows another view of the same general encoder configuration. Preferably, there is no fixed relationship between the audio blocks and the packets in which they are encoded and stored. This separates the encoding process from the characteristics of the transmission channel, which may have constraints on supported packet sizes and transmission timing.

[0104] Thus, in FIG. 2, the receipt of an audio block (step 200) and the receipt of a packet request (step 210) are treated as separate, asynchronous events that are linked by buffering.

[0105] Upon receiving an audio block, the encoder preferably performs an initial analysis of the block (step 201) to determine the relationship between the pre-quantization precision and the amount of data required to encode the block.

[0106] The encoder determines a step size Δ to use when pre-quantizing the audio to reduce the amount of coded data (step 202). Δ may be different for each channel. As will be explained below, the encoder preferably bases this selection primarily on the current stress level of the output buffering process. The initial analysis above may change this decision, especially if there is a risk of the buffer filling process being overrun. However, it should be noted that difficult audio begins midway through the block, increasing the pre-quantizer noise at the beginning of the block and resulting in a pre-response that cannot be hidden by auditory temporal masking.

[0107] The encoder uses a pseudo-random number generator to calculate pseudo-random offsets for the audio blocks (step 203).

[0108] Next, a pre-quantizer quantizes the audio to values ​​that are integer multiples of Δ offset by a pseudo-random offset (step 204). It is the pseudo-random nature of the offset that randomizes the quantization and avoids introducing quantization artifacts. This process is considered different in this specification from the subtractive dither described below, but is numerically equivalent and offers similar benefits to subtractive dither: avoiding quantization distortion without increasing quantization error.

[0109] The quantized audio is then input to a lossless encoder configured to operate with pseudo-random offsets (step 205).

[0110] Lossless codecs traditionally use the quantization step size at the input for compression. For example, FLAC scans audio blocks to detect when the least significant bits (LSBs) are consistently zero, and (subject to certain constraints) makes appropriate truncations in the encoded data rate. Gerzon (Non-Patent Document 1) considers the use of more general cases for compression, including step sizes that are not powers of two.

[0111] However, in this embodiment, the input to the lossless encoder is quantized to values ​​that are pseudo-randomly offset from integer multiples of the step size Δ. Each possible value of Δ defines a signal region, which together form a set of signal regions with Δ as a parameter. In this embodiment, Δ needs to be utilized for compression, since pre-quantization provides no gain to the system unless the lossless encoder utilizes Δ for compression. Such a method of utilizing Δ will be described later.

[0112] The output of the lossless encoder is preferably split into two components. Combined, these allow the decoder to accurately reproduce the pre-quantized audio fed to the lossless encoder in a lossless manner. One of these components, the base layer data, is capable of providing an approximate reproduction of the audio by itself. The other type improves the quality of playback and is called enhanced data.

[0113] The base layer data and enhancement data are then put into a buffering process (step 206). The storage process isolates the varying data rate output from the lossless encoder from the characteristics of the transmission channel. These data are preferably handled separately in the buffering process. The base layer data is kept as an indivisible unit and is therefore considered to be injected into the delay line. On the other hand, the extended data is treated as a bitstream, and is input into a FIFO buffer regardless of block boundaries, and is taken out sequentially from there.

[0114] Finally, the buffer stress measure used to select Δ for subsequent blocks is updated (step 207). A good indicator of buffer stress is the excess amount of coded data in the buffer relative to the average channel data rate integrated over one block period. This value is updated by adding the encoding size of the entire block and subtracting the expected channel capacity during the block.

[0115] An asynchronous packet request (step 210) is processed by retrieving an integer number of blocks of base layer data from the delay line (step 211). The number of blocks is determined according to the period of audio that the packet covers. This number is related to the repetition period of the packets on the channel, but may also be externally specified. The retrieved blocks are placed into packets, leaving a variable amount of free space within the packet. This remaining area is filled by retrieving the extension data as a bitstream from the FIFO buffer, regardless of block boundaries (step 212). This extension data preferably flows from the end of the packet towards the beginning. This arrangement allows the decoder to process the enhancement data in packets before the base layer data has been parsed and the base layer / enhancement boundary is known.

[0116] Finally, the buffer stress indicator is updated (step 213) to adjust for any discrepancies between the actual packet size and the expected size based on the configured average data rate and the number of blocks covered by the packet.

[0117] <General decoder configuration overview> FIG. 3 shows the corresponding decoder architecture. The received packet 324 is preferably split into two parts. One (base layer data 322) is sent directly to the lossless decoder 303 without being buffered. The other (extended data 326 ) is sent to the FIFO buffer 309 . In the buffer, the extended data 326 is subjected to a variable delay corresponding to the extension delay introduced at the encoder side, and is then supplied to the lossless decoder 303 as delayed extended data 323 . With this arrangement, all data is delayed by a fixed time between the lossless encoder and the lossless decoder, so that the base layer data provided to the lossless decoder is aligned with its corresponding enhancement data. For base layer data, this delay occurs entirely in the encoder-side buffer. On the other hand, for the extended data, part of the delay occurs in the encoder buffer and the rest in the decoder buffer.

[0118] The advantage of this arrangement is that it accommodates the case where the buffered data is not available for decoding. For example, if a decoder wants to immediately start decoding in the middle of a stream, data sent in previous packets will not be available. Alternatively, a packet loss may have caused the FIFO buffer in the decoder to lose synchronization. In such a situation, the decoder can still decode the base layer data to provide an approximate reproduction of the desired audio until the buffer regains synchronization and full lossless decoding becomes possible again.

[0119] The lossless decoder 303 is configured to decode the quantized data using pseudo-random offsets. To this end, a pseudo-random offset 306 is calculated that replicates the corresponding offset 106 generated by the pre-quantizer. These pseudo-random offsets are fed to the lossless decoder so that the output of the lossless decoder meets the same remainder conditions that the prequantizer followed when quantizing.

[0120] After lossless decoding, the audio is upsampled (302) if necessary. Upsampling is performed when the stream indicates that a pre-quantizer in the encoder has reduced the sampling rate, as described below. This upsampling allows the decoder to output a constant output sampling rate even when the prequantizer dynamically enables or disables decimation in response to varying transmission channel conditions. Decimation and upsampling are preferably designed to minimize audible artifacts when changing sampling rates through lossless codecs.

[0121] <Pre-quantization> The pre-quantizer is responsible for reducing the precision of the audio according to control commands. The primary technique for this processing is noise-shaping quantization with pseudo-random offsets, as shown in Figure 4. The operation is controlled by a parameter Δ, which controls the precision of the quantization. Noise-shaping quantization is well known in the prior art and is discussed as a pre-quantization technique in Non-Patent Document 1 (particularly FIG. 20b).

[0122] Here, it is assumed that the input audio signal is given as an integer value. For example, a 24-bit audio signal is -2 23 That's it, +2 23 Takes an integer value in the range less than. In Figure 4-(a), the quantizer Q Δ 413 quantizes its input to integer multiples of a step size Δ, which is an integer. However, this quantizer Q Δ A subtraction node and an addition node with the pseudorandom offset signal 402 are placed before and after 413, respectively. Through this three-stage process, quantization to a value offset by the pseudo-random number sequence 402 for an integer multiple of Δ is performed.

[0123] The error caused by this process is filtered by filter 415 (this transfer function A(z -1 ) has no delay term). The error in the overall process is filtered by filter 416 (this transfer function B(z -1 ) also has no delay term). The sum of the outputs of these filters constitutes the feedback signal 403, which is added to the audio input before quantization. This reduces the error caused by the quantization process to the transfer function (1+A(z -1 )) / (1+B(z -1 )) is spectrally shaped, which has the effect of reducing errors in frequency regions that are easily perceived by the ear, and instead increasing errors in frequency regions that are less easily perceived by the ear. A(z -1 ) or B(z -1 ) can be omitted, in which case the configuration is simplified.

[0124] The auxiliary quantizer box Q' 414 is included in the diagram for somewhat formal reasons. After applying the error feedback, a high precision signal is obtained, which the auxiliary quantizer box Q' 414 re-quantizes to a predetermined precision, such as integer values. This process is performed by the filter A(z -1 ) by limiting the precision of the signal fed to the filter A(z -1 ) can be implemented in fixed precision arithmetic. In addition, the filter A(z -1 ) is omitted, this processing is not necessary. The auxiliary quantizer box Q' 414 is advantageously implemented with a conventional additive dither.

[0125] Typically, audio quantization is done with a step size that is a power of two, resulting in an output with an integer number of zeros in the low-order bits. However, in the pre-quantizer, step sizes of powers of two are not suitable for use because they are too widely spaced and limit the noise level adjustment to 6 dB steps. Since the pre-quantizer needs to adjust the level of quantization noise with higher precision, Δ (step size) needs to be able to take values ​​other than powers of two. In a typical codec, integer values ​​allowed for Δ are defined in a table format, and are often set so that the values ​​increase at rates of, for example, 1.5 dB, 2 dB, and 3 dB.

[0126] The pseudorandom values ​​402 used for subtraction and addition are preferably integers uniformly distributed in the range 0 to less than Δ. As shown in Figure 4-(a), a pseudo-random value 402 is obtained by generating a value in the range of 0.0 to 1.0 using a pseudo-random number generator (PRNG) 410, multiplying it by Δ411, and then quantizing it to an integer value 412 (usually by truncating the decimal part). However, other derivations are possible. In particular, since pseudorandom values ​​are used for both subtraction and addition, only the remainder modulo Δ affects the actual operation. For example, one could directly use pseudorandom integers with a much larger range than Δ, which would still give a nearly uniform distribution modulo Δ.

[0127] The pseudo-random offset can be applied in a variety of ways. For example, as shown in Figure 4-(a), the quantizer Q Δ Instead of subtracting and adding just before and after , this offset is subtracted from the input signal to the overall noise shaping quantization and added back to its output, as shown in Figure 4-(b). Although they look very different, Figure 4-(a) and Figure 4-(b) are computationally identical.

[0128] <Example of pseudo-random offset> The concept of quantization to a value obtained by adding a pseudo-random offset to an integer multiple of the step size will be explained using a specific example. For convenience of calculation, Δ=100, and the quantization error is set to be in the range of −50 or more and less than 50. JPEG2025540705000003.jpg67169 Note that the values ​​of the quantized sequence are within a range of ±50 from the value of the signal sequence, but the last two digits match the offset sequence.

[0129] Then the transfer function is 1-z -1 Another example using noise shaping is shown below. JPEG2025540705000004.jpg60169Here, the "signal + feedback" sequence is subject to quantization, and the error caused by quantization is delayed and sign-inverted (-z -1 ), which forms the feedback 403 that is added to the signal.

[0130] <Relationship between pseudorandom offset and subtractive dither> The concept of adding a pseudorandom value before quantization and then subtracting a synchronous representation of it has been proposed before. So why use the descriptive term "pseudorandom offset" rather than the more widely accepted term "subtractive dither"? This is because it is a different concept from subtractive dithering, and the difference between the two is not in the arithmetic processing itself, but in the location of the processing.

[0131] In 1962, Roberts (Non-Patent Document 3) proposed a technique of adding noise to image pixels before quantizing them for transmission, and then subtracting the same noise at the receiving end. In 1989, Gerzon and Craven (Non-Patent Document 4) proposed the now widely accepted term "subtractive dither" for Roberts' technique, defining it as "a technique in which the dither applied to the quantizer is subtracted at the output of the digital transmission path" (p. 12).

[0132] The important thing here is that the addition and subtraction processes are separate (across the transmission path). The limited bit width of the transmission path creates the need for quantization and for synchronized noise sources at the transmitter and receiver. In Roberts' case, it was television broadcasting, and Gerzon and Craven subsequently proposed the application of subtractive dithering by quantizing high-precision audio signals to 16 bits, recording them on a CD, and then subtracting the dither on the CD player side (Non-Patent Document 5). If there are no restrictions on the capacity of the transmission line, there is no need to use a quantizer in the first place.

[0133] If subtractive dithering is added to the "pre-quantizer + lossless codec" configuration proposed by Gerzon & Craven and Craven & Stuart, the resulting system will have the configuration shown in Figure 5-(a). That is, on the encoder side, dither 510 is added before the quantization process 500 that reduces precision, and on the decoder side, dither 511 is subtracted after the lossless codec 501 in synchronization with it. In the absence of noise shaping (or in the case of fixed noise shaping), this scheme can be a useful improvement over the "pre-quantizer + lossless codec" proposals in [1] and [2]. This is because it is widely known that dithering is effective, and that subtractive dithering is even better.

[0134] However, adding dithering increases the entropy of the signal received by the lossless codec, reducing the efficiency of the lossless codec. This problem is particularly pronounced with spectrally white dithers, which fill in the spectrally quiet regions of the signal and prevent lossless codecs from taking advantage of these low-entropy regions. This loss of efficiency can be mitigated to some extent by filtering 520, but cannot be completely eliminated.

[0135] However, as shown in the conceptual diagram of FIG. 5(a), the dither subtracted at the receiver side also needs to be filtered (521) to match the noise shaping at the transmitter side. In Non-Patent Document 5, this processing was treated as inherent in the fixed standard noise shaping. However, in the "pre-quantizer + lossless codec" approach, this filtering needs to be adapted to the audio spectrum, which results in the need for synchronization between the encoder and decoder. This requirement negates a key advantage of the "pre-quantizer + lossless codec" approach: no need to communicate the spectral shape of the noise floor to the decoder, making the use of subtractive dither across codecs impractical.

[0136] In contrast, the improvement that the present invention makes over the prior art "pre-quantizer + lossless codec" proposal is shown in general form in FIG. 5-(b). Here, a pseudo-random offset 511 is added and subtracted just before and just after the quantizer 500 . This process produces a signal with a bit width greater than the quantization precision, which at first glance seems to make it impossible for a lossless audio codec to operate at the desired low bit rates. However, as disclosed herein, if the lossless codec 502 is properly configured to accommodate a known offset, it is possible to actually achieve the desired low bit rate.

[0137] At the decoder side of a lossless codec, the corresponding pseudo-random offsets still need to be synchronized, but the noise shaping does not need to be synchronized. Furthermore, the signal received by the lossless codec 502 does not have its entropy increased by the introduction of a pseudo-random offset.

[0138] <Spectral shape of prequantizer noise> The generally accepted view is that the auditory perceptibility of codec noise depends on the spectral content of the signal that masks it. For this reason, lossy audio codecs should concentrate errors in the spectral regions that are currently masked by the audio signal.

[0139] In Non-Patent Document 1, Gerzon explains in Figure 20a, pages 67-69 how this idea can be applied to the pre-quantizer of a lossless audio codec, estimating an auditory masking curve from which noise shaping coefficients can be derived.

[0140] In contrast to this approach, the inventors have found it preferable to design noise shaping filters based on equal loudness contours, in particular on hearing thresholds. An example of a suitable noise-shaping transfer function is shown in Figure 6. 600 and 601 correspond to a sampling rate of 48 kHz, while 602 and 603 correspond to 96 kHz.

[0141] In the range of approximately 1 kHz to 15 kHz, the noise-shaping transfer function is shaped based on the spectrum of a uniformly exciting noise at the threshold level. There is a dip around 3-4kHz, and another dip around 12kHz.

[0142] Above 15 kHz, the spectrum of uniform excitation noise rises sharply. In this region, increasing the pre-quantizer noise level above the signal noise level does not provide any noise-shaping benefit, so the curves fall below the equal loudness curves. For example, it may be flattened up to the Nyquist frequency (as shown by 600, 601, and 602, each with a different gain in the flat section), or it may fall off further at higher frequencies, as shown by 603, reflecting a lower spectral density of the signal. The vertical line at 15 kHz marks the rough boundary between these two regions.

[0143] The equal loudness contours also rise at low frequencies, but there is limited benefit to closely matching the pre-quantizer noise levels across the rise. Therefore, all four curves shown in Figure 6 are flat below 1 kHz. However, the choice of noise shaping curve in the four octave range from 1 kHz to 15 kHz has been found to have a significant impact on the quality of the sound added by the pre-quantizer.

[0144] On the other hand, in the range above 15 kHz, the practical aspect of achieving sufficient noise shaping effect without damaging the audio signal becomes more important. Therefore, it is reasonable to have a flat noise transfer function, and at high sampling rates, it is also reasonable to have the noise transfer function fall off again at frequencies above 20 kHz, as shown in curve 602. Another reasonable option is to make the noise transfer function follow the audio spectrum above 15 kHz.

[0145] The boundaries of 1kHz and 15kHz are merely guidelines and not strict specifications. For example, around 13 kHz, this is the region where the curve begins to rise sharply, so it is entirely reasonable to relax the tracking of the noise shaping curve at this frequency.

[0146] Data on equal loudness is readily available from standards such as ISO 226:2003 and ISO 389-7:2019. However, these data are based on equal loud sine waves and require adjustment to apply to noise. The reason is that the variable integral bandwidth of the ear varies with frequency band, so that to achieve the same loudness as a sine wave at a given sound pressure level, different noise spectral densities are required for different frequencies. Details are explained in Non-Patent Document 6.

[0147] When noise is shaped according to equal loudness contours tailored to the noise, it sounds smooth and no particular frequency band is emphasized to draw attention to it. Such noise stimulates sensory receptors evenly throughout the cochlea. Noise shaped to provide uniform excitation at threshold is the most powerful yet auditorily inaudible sound, making the quantizer noise less audible or inaudible when heard alone. The advantage of noise shaping the pre-quantizer error with this type of curve is that the added noise, if any, is perceptible, but is gentle and steady in nature, blends naturally into the background, and is easily ignored. In contrast, a noise spectrum based on masking theory may not be perceptible if the signal completely masks it, but if the added noise causes even a slight change that affects perception, the noise spectrum may be so closely coupled to the signal spectrum that it is perceived by the listener as a distortion of the signal rather than background noise.

[0148] Therefore, the inventors believe that it is preferable to minimize the auditory perceptibility and annoyance of the noise added in pre-quantization alone, rather than trying to exploit additional spectral regions that are known to be masked by the signal.

[0149] Since the shape of the curve for uniform excitation noise changes depending on the volume level, it may be preferable to use a curve that is appropriate for the loudness of the actual noise. However, this is not a major problem. This is because the curves of uniform excitation noise are roughly parallel. Furthermore, audio codecs typically do not know the acoustic gain of the playback system, and therefore the listener's actual SPL (sound pressure level), making it difficult to determine exactly which curve to use. Since the aim of high resolution codecs is to make the noise floor inaudible, we propose to use the curve of uniform excitation noise at the hearing threshold.

[0150] ISO 389-7:2019 provides thresholds for both free-field and diffuse-field listening conditions. Experiments have shown that noise shapers designed based on free-field thresholds sound more pleasing than those that attempt to incorporate diffuse-field thresholds, and Figure 6 details the resulting curve shapes. The benefit of noise shaping at low frequencies depends on the amount of noise amplification in the flattened region.

[0151] Dynamic Pre-Quantization Noise Shaping Whether the noise shaping follows a dynamically calculated auditory masking threshold, as shown by Gerzon, or follows a variable high frequency boost and shape as described above, the noise shaping transfer function (1 + A(z -1 )) / (1+B(z -1 )) should be changed from time to time.

[0152] In fact, the ability to dynamically change the noise shaping transfer function is a key advantage in prequantized codecs. The important thing is that the decoder does not need to know anything about the noise shaping that has been applied. In contrast, transform codecs implement a frequency-dependent noise floor via band scale factors, and must communicate this information to the decoder. This communication not only incurs a cost in data rate, but also imposes constraints on the format specification to standardize the exact set of available spectral noise shapes. On the other hand, pre-quantized codecs do not require such standardization, so encoders have greater freedom in how they control the audio quantization accuracy, and there is greater potential for technical improvement after the standard is established.

[0153] Therefore, (1+A(z -1 )) / (1+B(z -1 )) needs to be considered to gracefully modify it without problematic artifacts. One possibility is to gradually change the coefficients, but this is computationally expensive and requires providing a suitable coefficient change trajectory. Another option is to change it instantly at a block boundary (possibly in sync with the change of W), but care must be taken to avoid introducing artifacts when changing it. A(z -1 ) is kept constant, and B(z -1 ) is the only thing that should be changed. The reason is that A(z -1 ) without careful tuning of the filter history will introduce discontinuities in the delay-dependent impulse response. In contrast, B(z -1 ) changes do not cause such problems. B(z -1 ) remains valid even after the coefficient changes, and represents the total pre-quantizer changes that are actually perceived by the listener.

[0154] <Reducing the sampling rate> The pre-quantizer preferably can make decisions to dynamically reduce the sampling rate, typically by a factor of two, from about 96 kHz to about 48 kHz, although other ratios can also be implemented. To achieve this reduction by a factor of 2, a lossless codec must be able to accept blocks containing half the normal number of samples, and block sizes for normal sampling rates should be constrained to be divisible by two. It is further preferred that a corresponding upsampling is performed at the decoder output in response to the reduction in sampling rate.

[0155] This mode may be enabled or disabled mid-stream, so it is important to minimize audio artifacts caused by the switch. Decoder behavior during switching is preferably standardized, so that the encoder can understand the entire signal processing flow and perform processing to minimize artifacts.

[0156] Nevertheless, frequent changes in sampling rate are undesirable, and it is preferable to maintain a reduced sampling rate rather than temporarily increase it. Preferably, the sampling rate is reduced not in response to changes in audio characteristics, but rather in response to changes in transmission conditions that make it impossible to maintain the data rate required for satisfactory operation at higher sampling rates. More preferably, a delay and hysteresis is provided when deciding to return to the high sampling rate, so that a stable high communication capacity must be maintained for a certain period of time before operation at the original high sampling rate can be resumed. This prevents situations where a high sampling rate is only temporarily enabled.

[0157] Preferably, the lossless codec adjusts its internal state appropriately when the sampling rate changes. For example, the predictor may keep recent audio history across state boundaries to predict the initial samples of the next block. When the sampling rate is changed, these histories are preferably modified to represent valid values ​​as if the previous block had been coded at the new sampling rate. The details of this modification must be standardized so that both encoders and decoders can perform the same modification, so that no non-lossless behavior is introduced into a lossless codec.

[0158] <Conversion to mono> Preferably, the lossless encoder should be able to encode two identical channels at a data rate that is almost the same as encoding one channel alone. In this case, the lossless encoder subtracts the first channel from the second channel, resulting in a difference of exactly zero, so this transformed channel should be encoded with very little data.

[0159] This feature can be exploited by a pre-quantizer. For example, the data rate can be reduced by converting the two channels to have identical audio (e.g., averaging both channels). However, this is a change that is clearly perceptible to the ear, and is unlikely to be suitable for claiming high-resolution reproduction. Nevertheless, it can be a useful solution for extending codec operation to data rates lower than can be satisfactorily achieved with an independent channel.

[0160] This is an operating mode that, like a reduced sampling rate, should be activated in response to a reduction in the transmission channel capacity, rather than in response to the characteristics of the supplied audio. It should also preferably be enabled or disabled intentionally, rather than momentarily, and care should be taken to avoid artifacts associated with the disappearance or reappearance of the difference signal. In particular, because the differential signal is noise shaped (by noise shaping each channel separately), the techniques described in the "Going Lossless" section below can be used to prevent clicks caused by the cessation of noise shaping when the differential channel goes completely to zero.

[0161] Typically, the channel is quantized to an integer multiple of Δ with a pseudo-random offset. From this, the pseudo-random offset is a pseudo-random number sequence that differs for each channel. The case where the two channels are identical is a special case that deviates from this general policy, and a lossless codec should preferably be able to recognize and encode this special case.

[0162] <Transition to lossless> While we have looked at various ways to reduce the quantization accuracy of the audio using a pre-quantizer, another important possibility is to leave the audio completely unchanged. In this case, the entire codec is lossless.

[0163] By enabling this mode of operation, it is possible to operate normally in lossless mode, but smoothly transition to lossy mode when the channel capacity is reduced or when the audio contains difficult-to-process sections where the encoded data rate exceeds the channel capacity.

[0164] In lossless operation, the audio is not modified by the pre-quantizer, so a zero offset is applied to the audio passed to the lossless encoder rather than a pseudo-random offset. Therefore, a lossless codec must be flexible enough to handle audio with or without pseudo-random offsets.

[0165] It is also important to be able to switch to and from lossless mode without introducing auditory artifacts. The transition to lossy operation is relatively easy: just start noise-shaping quantization. However, the transition to lossless operation is problematic.

[0166] Noise shaping operates on the premise that the audible effect of errors introduced into the current sample can be reduced by modifying (spectrally shaping) subsequent samples. However, once you go lossless, no changes can be made to any samples after that. Therefore, the error occurring in the last lossy sample cannot be shaped at all, and the error of the lossy samples before that can only be shaped to a very limited extent. The result is a click when noise shaping stops.

[0167] If lossless means 16-bit or 24-bit audio, the clicks, while undesirable, may be relatively difficult to perceive. However, in the present invention, when re-encoding a stream that was previously transmitted at a pre-quantized low bitrate, the quantization levels are coarser, so the clicks that occur when transitioning from lossy to lossless become a more severe problem and the need to mitigate them becomes even greater.

[0168] To avoid clicks when going lossless, a technique is required that quantizes and noise-shapes a finite number of samples, then quantizes them together to minimize the spectrally weighted error. The transition from normal noise shaping to this technique should occur at the timing of the last n lossy samples. The larger the value of n, the better the quantization error shaping performance, but the higher the calculation cost. In practice, even moderate values ​​of n, such as 4 or 8, are effective enough in reducing click noise, and it is thought that increasing n above 32 is almost meaningless.

[0169] Bulk quantization can be performed using the least squares method, and the model used to set up the problem is shown in Figure 7. Given the original audio 700, we replace it with the desired quantized audio 701, which satisfies the quantization constraint of an integer multiple of Δ plus a pseudo-random offset. The difference between the quantized audio and the original audio, d t is the transfer function W(z -1 ), and the resulting signal e t The power of is appreciated (703). The least squares problem seeks to choose the quantized audio 701 so as to minimize this power 703 .

[0170] JPEG2025540705000005.jpg47170

[0171] JPEG2025540705000006.jpg40170

[0172] However, if the value of the latter d becomes zero because the quantizer operates losslessly, the assumption that the value of the latter d can be changed no longer holds. Therefore, d t When selecting, the value of the subsequent e must also be taken into consideration.

[0173] JPEG2025540705000007.jpg27169

[0174] JPEG2025540705000008.jpg90169

[0175] JPEG2025540705000009.jpg32170

[0176] JPEG2025540705000010.jpg41169

[0177] JPEG2025540705000011.jpg35170

[0178] JPEG2025540705000012.jpg51169

[0179] JPEG2025540705000013.jpg61170

[0180] Once you have a value of Y that you are happy with, you can convert it to the desired variable X by calculating X=VY. These offsets X are added to the last modifiable audio sample to arrive at the desired value that minimizes the clicks that occur when noise shaping stops. This is the "selected quantized audio" shown in FIG.

[0181] The key point is that most of this calculation can be done upfront at design time, and only a small amount of real-time work needs to be done when the last n samples need to be batch-quantized before actually going lossless. (Q1 T Q T W1), R1, and V depend only on W, which determines the weighting of the errors in different spectral regions, and can be prepared in advance and kept in tabular form for later use.

[0182] JPEG2025540705000014.jpg40170

[0183] A crude approach is to first take the most recent quantization error value and multiply it by a pre-calculated and stored matrix -(RV) -1 (Q T W1) to obtain a column vector. Next, generate integer vector Y by rounding each element of the column vector. The resulting Y is processed by pre-multiplying it by a second matrix V, which has been pre-computed and stored, as before.

[0184] QR decomposition is not the only method for solving least-squares problems, and alternative methods exist that organize some of the computations differently. The important point is that the problem is solved in a transformed form with respect to the set of variables Y, with the problem being better conditioned with respect to the variables, and finally transformed to the desired values ​​using a single factor matrix of integer values.

[0185] <Summary of the algorithm> The flowchart in FIG. 8 summarizes the processing steps involved in the above algorithm. During design (step 800), a desired frequency weighting filter (which may be the inverse function of the noise shaping transfer function) is used (step 801) to formulate a least squares problem for n variables (step 802).

[0186] The potentially large matrices involved in this least squares problem are first reduced to an n×n matrix that describes the same minimization problem (step 803). Since this problem is typically ill-conditioned, a lattice reduction algorithm, such as the LLL algorithm, is used to find another, better-conditioned basis that can be transformed into the original basis by an integer-valued single factor matrix (step 804).

[0187] A matrix that describes this well-conditioned problem in an easily solvable form is calculated (step 805) and stored for use at run time (step 806). Also stored is a matrix of integer-valued simple factors that transforms the solution to this well-conditioned problem back into the original variables. At run time (step 810), the state of the noise shaping filter holds all the relevant information about the noise that should be suppressed when noise shaping is turned off. This state is mapped to an n-dimensional minimization problem by pre-multiplying it by a previously stored matrix (step 811).

[0188] The problem is solved as an integer solution in the improved conditioned basis (step 812). This can be done quickly and roughly by rounding each coefficient, but more precisely requires backsubstitution using a pre-stored upper triangular matrix. The solution thus obtained is transformed into the basis of the original problem by multiplying it by a pre-stored integer-valued single factor matrix (step 813).

[0189] <Correction of Δ step size> If the step size Δ is not 1, this can be handled by dividing H by Δ. Then, find the integer value of X and multiply X by Δ to restore the scale. It should be noted that the multiplication process by Δ can be pre-installed in the pre-stored matrix V, if necessary. In this case, the determinant of the stored matrix is ​​Δ n This becomes:

[0190] <Fix pseudorandom offset> To add pseudo-random offsets to the n values, add n rows containing the offsets with their signs reversed to the beginning of the vector H, and similarly prepend a copy of W2 onto W1. After finding X, you can simply add these offsets back in.

[0191] <Modifying noise shaping functions to different forms> As mentioned above, taking into account the possibility that the impulse response of W may be infinite, the weighted error signal is expressed in the form W1H as a term based on the previous error. If the noise shaping filter is of the all-pole type, these previous errors directly correspond to the state variables of the filter. On the other hand, if the filter takes another form, previous errors may not be easily available or may require many. Therefore, it is practically more convenient to use the state variables of the filter as H rather than the previous errors. In such a case, W1 can be appropriately modified so that W1H is still the weighted error signal. The modified W1 may not be a perfect Toeplitz matrix, but this is not a problem because the calculation does not use this property.

[0192] <Correction for vectors with low impact> One or more of the diagonal elements of R1 may be very small. This means that there are coefficients in Y that have little effect on the evaluation index. For such coefficients, it may be preferable to set them to zero rather than allowing large values ​​to be chosen in backsubstitution to obtain a small improvement in the performance metric. Eliminating these coefficients reduces the size of the matrices that need to be pre-computed and stored.

[0193] <Calculation cost> The design-time computational cost of converting the noise specifications into the appropriate storage matrix is ​​significant, but the run-time cost of solving a particular instance of the problem is small. The initial multiply-and-accumulate operation (Q1 T Q T Calculating W1)H is similar to running a noise shaping filter for n more samples. The back-substitution solution for Y is to, for each value, apply the dot product of the already calculated Y value with the pre-computed vector to the pre-computed (Q1 T Q T W1) Includes the process of subtracting from H and calculating the result. This process requires less than n multiply-and-accumulate operations per coefficient, which is lighter than the quantization process that would be required if the noise shaper were still running. Furthermore, the process of pre-multiplying Y by R1 is also composed of n multiplication and accumulation operations for each coefficient. Therefore, the additional computational cost of this technology is only a very small difference of 2n product-sum operations per n samples, compared to when noise shaping processing is performed continuously for n samples.

[0194] <Standardizing step sizes between channels> Various decisions can be made as to whether the step size Δ should be commonly restricted for all channels or whether different step sizes should be allowed for different channels. As an intermediate option, for example in a 5.1 multichannel signal, it may be reasonable to assign one step size for {L,R,C}, another step size for {Ls,Rs}, and yet another step size for {Lfe}. Allowing different step sizes allows flexibility in the prequantizer, but the benefits are likely to be small for channels that are closely correlated, such as {L,R,C}. The step size must be notified to the decoder, so an increase in the number of values ​​to be notified results in a corresponding increase in the data rate. Additionally, if there is a possibility that a strong correlation exists between channels, it is useful to set a common step size for all channels, as this will allow the lossless encoder to take advantage of that correlation, thereby contributing to improved data compression efficiency. Furthermore, if the pre-quantizer may perform a sampling rate reduction, then channels constrained to a common step size are preferably also constrained identically in sampling rate.

[0195] <Current Block Analysis> Preferably, an analysis is performed to estimate the amount of data that will be losslessly coded for the currently provided block. FIG. 9 is a diagram showing an example of an appropriate analysis method. When an audio block is received (step 900), the audio for each channel is windowed (step 901) and the autocorrelation function (ACF) of the windowed audio is calculated (step 902). The window's support may extend back in time to previous blocks. This autocorrelation function preferably has one more term than the order of the prediction filter used in the lossless encoder.

[0196] For each of the multiple combinations of Δ and noise shaping (step 903), the following processing can be performed on the autocorrelation function of each channel. Calculate the autocorrelation function of the quantization noise introduced by the quantizer (step 904). This process involves calculating and storing the autocorrelation function of the noise introduced by the unit quantization in advance, and then applying Δ 2 This can be easily done by multiplying The autocorrelation function of this quantization noise is added to the autocorrelation function of the signal to obtain an estimate of the autocorrelation function before quantization (step 905). · Filtering is performed with an appropriate FIR filter with the first tap set to 1 to calculate the innovation sample (prediction error), and then the Levinson-Durbin algorithm is applied to estimate the power P (step 906). Finally, the coded data rate per sample is estimated (step 907) by the following formula: log4(P / blockSize)-log2Δ+K where blockSize is the number of samples in a block and K is a constant. Although the value of K can be derived from the entropy of a normal distribution and a window function, it is preferable to measure it empirically to take into account factors such as inefficiencies in lossless coding and non-normal distributions of innovation samples.

[0197] The estimate for losslessly encoding the entire block is then the sum of the per-channel estimates plus an allowance for bitstream overhead. Optionally, to evaluate the effectiveness of exploiting the correlation between channels, a similar process may be performed on the channel difference signals, and the channel and its difference signal that can be represented using fewer bits may be selected.

[0198] This analysis process requires a certain amount of calculation, but it is a calculation that is preferably performed in any case in the lossless coding process in order to design a prediction filter. Preferably, the analysis results, including the autocorrelation function of the noise for the actually applied pre-quantizer configuration, are provided to the lossless encoder so that the analysis process does not have to be duplicated at the lossless encoder side.

[0199] The analysis results that are discarded are evaluations for pre-quantizer configurations that are not ultimately used. If desired, this wasted computation can be minimized by using only the initial term of the autocorrelation function, at the expense of a small degree of accuracy. This is because in practice, most of the spectral variation can be captured by a low-order (e.g., second-order) prediction filter, and the benefits of increasing the order of the prediction filter tend to diminish.

[0200] The measured autocorrelation function is also preferably used to guide the selection of a noise shaping filter based on the general spectral characteristics of the audio. In this regard, as mentioned above, the choice of such a filter preferably only affects the shape of the transfer function above a certain threshold frequency.

[0201] Optionally, the selection of such a filter may be based on the autocorrelation function of the previous block rather than the current block. This makes it possible to prevent an audio event occurring in the middle of a block from changing the spectral characteristics of the noise at the beginning of the block. However, because the pre-quantization architecture allows noise shaping to be varied within a block, more advanced signal analysis can be applied to detect changes in audio characteristics and more precisely target noise shaping changes.

[0202] <Signal processing in lossless encoders> FIG. 10 is a diagram showing an outline of signal processing in a lossless encoder. Audio blocks 1020, which may be multi-channel, are matrixed 1000 to exploit redundancies between channels. The data rate reduction effect here is not as great as one might hope, so we do not recommend going beyond conditionally subtracting one channel from another to produce a difference channel. This works well when the two channels are carrying mono or near mono signals.

[0203] Preferably, the ability to perform such matrixing imposes constraints on the pre-quantizer. That is, it is preferable that channels that are allowed to be matrixed have a common step size Δ (and actually a common sampling rate). This ensures that the difference channel maintains a known remainder modulo Δ, avoiding problems in the further quantization process. Each channel is then processed independently.

[0204] The process begins with the predictor 1001 exploiting the spectral shape. Filter 1010, P(z -1 ) to calculate the predicted value of each sample based on the previous sample values. Subtracting the predicted value gives a signal that is usually called "innovation." Another aspect that is useful in designing a suitable prediction filter is that the encoder uses a filter 1-P(z -1 ) can be thought of as filtering the audio signal. In this case, P(z -1 ) are chosen to whiten the spectrum of the resulting innovation.

[0205] The innovations are then quantized 1011 to integer multiples of Δ (pre-quantization step size). This quantization process does not lose information because there is only one quantized value for each set of consecutive values ​​that fall within Δ for each sample value of the input audio 1020 . Surprisingly, no special handling for the pseudo-random offset is required at this stage.

[0206] The quantized innovations 1024 thus obtained can be divided 1002 by Δ to obtain integer values ​​for further processing. In this specification, in order to clearly explain how the processing in the decoder corresponds to the processing in the encoder, the quantization processing to an integer multiple of Δ and the division processing by Δ are described separately. However, in implementation, it is possible to integrate these into one rounding division process. This rounding division process has a certain degree of flexibility, and if the definitions in the definitions section above are followed, even if two values ​​that differ by an integer multiple of Δ are input to the prediction unit 1001, the same rounding division result will not be output. However, regardless of the processing method adopted, it is necessary to accurately implement the inverse processing on the decoder side, so it is desirable that this be standardized as a specification.

[0207] Each sample value is then divided 1003 into two parts, with the adjustment block 1005 being discussed later. Innovations have been shown to have a relatively constant distribution shape (similar to a normal distribution with thick tails), but the standard deviation varies. Therefore, it is generally desirable to divide the innovation by a scale factor (hereafter referred to as level) to obtain a deviate with a stable distribution suitable for entropy coding.

[0208] A typical approach in lossless coding is to scale this scale factor to a power of 2, i.e., 2 k The k-bit fractional part obtained after division is truncated, and the remaining most significant bits (MSB) (the most significant part of the binary word) are Huffman coded. In this case, the truncated k-bit fractions are approximately uniformly distributed, so there is no gain in entropy coding them, and they are simply added verbatim to the Huffman code to form the composite codeword.

[0209] Similarly, the division unit 1003 outputs the decimal part after division by the level as output 1022 and outputs the upper bits (integer part) to the entropy encoding process 1004 to obtain data 1021. Here, the upper bits (scaled by level) approximately represent the innovation, and by generating this approximate innovation signal, the decoder is able to approximately decode the audio. According to scalable codec terminology, the coded upper bits 1021 are called "base layer data" and the fractional bits 1022, which complement the upper bits and enable the input signal to the splitter 1003 to be accurately reconstructed, are called "extension data".

[0210] The enhancement data is preferably packaged separately from the base layer data. Variable delay FIFO buffering is a key component in pre-quantized codecs, but the buffered data is at risk. For example, in situations such as mid-stream startup or packet loss, buffered data may be unavailable for several blocks, while unbuffered data is immediately accessible. For this reason, it is preferable to have a configuration in which only the enhancement data is stored in the buffer, and the base layer data is not stored. This allows approximate decoding to be performed from base layer data even when buffered data is unavailable.

[0211] Since there are situations in which approximate decoding is actually heard, it is important to minimize the auditory perceptibility of this approximation. For this purpose, a regulation block 1005 is provided. In this block, the previous value of the extension data 1022 is added to the current innovation to noise-shape the division process with a transfer function that has zero DC (direct current) components, thereby reducing the auditory perception of approximation errors.

[0212] When crossing block boundaries, additional considerations are required. When Δ changes from Δ1 to Δ2, the value of the delayed extension data needs to be adjusted by multiplying by Δ1 / Δ2 to accommodate the scale change of the quantization innovation.

[0213] There are other arithmetic rearrangements that achieve the same effect. For example, the adjustment value may be multiplied by Δ and then added before being divided by Δ. This reconfiguration eliminates the need to adjust the delay value when Δ changes. Such an adjustment 1005 to the division process 1003 has a slight adverse effect on the compression efficiency of the lossless encoder. This is because the entropy of the higher bits increases, resulting in an increase in the amount of data in the base layer data 1021. However, the sound quality benefits of approximate decoding more than justify this small increase in data rate.

[0214] This approach is not limited to single zero configurations. While more complex adjustments can be made to implement any noise-shaping transfer function, a single DC zero appears to be the most practical compromise.

[0215] <Buffering and signal processing in lossless decoders> FIG. 11(a) is a diagram showing an outline of signal processing in a lossless decoder. Each operation essentially corresponds to the operation in the encoder, but has the opposite effect and is performed in the reverse order.

[0216] Base layer data 1121 and extension data 1123 are read from the received packet. The base layer data is decoded by an entropy decoding process 1104, which is the inverse of the entropy encoding process 1004 of the encoder. Meanwhile, the extended data is put into a FIFO buffer 1106 .

[0217] For each sample, the enhanced data 1122 is taken from the FIFO buffer and combined 1103 with the entropy decoded base layer data. This combining process 1103 reverses the splitting process 1003 in the encoder, scaling the entropy decoded base layer data by level and then complementing the detail components with enhancement data 1122 . The decoder adjustment process 1105 operates to invert the encoder adjustment process 1005 . This adjustment is performed by subtracting the value of the previous extension data. A multiplication by Δ operation 1102 is then applied to obtain a reproduction 1124 of the quantization innovations 1024 in the encoder.

[0218] <Decoder prediction> Here, how the decoder prediction unit 1101 reverses the processing of the encoder prediction unit 1001 will be described. An a priori assumption is that the previous output value from the decoder's predictor is consistent with the previous input value to the encoder's predictor. In this case, the output of the predictive filter 1110 of the decoder reproduces the output of the predictive filter 1010 of the encoder. Call this common value p. Also, the current input and output of the predictor part of the encoder are called x and y, respectively.

[0219] A lossless encoder encodes audio so that for each sample, the remainder modulo Δ is congruent to some value d. A lossless decoder must reproduce this value d. In FIG. 11, a similar scheme to the dither generator shown in FIG. 4 is used for the pseudorandom offset generator 1107, however, some special cases require different processing. For example, if the prequantizer operates in lossless mode and does not modify the signal, then d ≡ 0 modulo Δ and does not come from a pseudorandom offset generator. Also, if the channels are matrixed and the difference between two pre-quantized channels is transmitted, the remainder d modulo Δ is congruent to the difference of the pseudorandom values ​​corresponding to the respective channels.

[0220] Invertibility is evident from the following points: That is, in the encoder, y=x-p+ε E where ε E is the error introduced by the quantizer of the encoder. And xy ≡ d modulo Δ also holds. The input to the encoder's quantizer is xp, and the input to the decoder's quantizer is dp. Since x≡d modulo Δ, if both quantizers 1011 and 1111 are standardized to have the same rounding behavior, then they will both add the same error ε to their input.

[0221] Therefore, the output of the encoder's predictor is y=x-p+ε, and the output of the decoder's predictor is x ’ =y+d-(ε+dp). This is equal to x as desired, resulting in lossless playback. It will be appreciated that there are many different configurations of this computation that are equally effective. FIG. 11-(b) shows an example of such an alternative layout, where y+p is quantized instead of −p. Since y is divisible by Δ, the quantization process commutes with the addition of y, but the signal passing through the quantizer is sign-inverted, so the operation of the quantizer must be adjusted accordingly. For this reason, in this embodiment, this quantizer is given the reference number 1112 . While quantizer 1111 replicates the operation of quantizer 1011 of the encoder, quantizer 1112 must perform a complementary operation, as described in Non-Patent Document 1.

[0222] If the recent outputs of the decoder's predictor do not accurately represent the inputs to the encoder's predictor, then the above a posteriori assumption does not hold, and there is no reason to expect that subsequent decoder predicted outputs will exactly match the corresponding encoder predicted inputs. However, in practice they may coincide, and in rare cases two samples may reproduce the encoder's predicted input. Even more rarely, enough output samples happen to have the correct value to guarantee that all future outputs will have the correct value. If the order of the prediction filter is small (for example, 4), such a probabilistic mechanism is sufficient to achieve lossless operation within an acceptable time.

[0223] Preferably, both the quantizers in the encoder and decoder have noise shaping functions (not shown). As long as the noise shaping is identical in both the encoder and decoder, exact recovery still holds. Noise shaping helps reduce the auditory perception of noise during the period until the decoder reaches a state that matches the encoder. Additionally, noise shaping may speed up this state matching process if it is chosen to reduce noise variability in the decoder predictor output.

[0224] <Decoder matrix processing> Preferably, a lossless codec has the ability to encode the difference between two channels rather than encoding each channel separately. This allows the data rate to be reduced if correlation exists between channels.

[0225] If one channel is matrixed, then at the decoder, after processing by the predictor, the matrixing must be undone by adding the other decoded channel to the difference channel. However, the matrixing also affects the pseudo-random offset applied to the difference channel. The pseudo-random sequence defines the offsets used at the output of the pre-quantizer, and these offsets must be losslessly reproduced at the output of the decoder. However, in the decoder, the pseudo-random offset is applied in the predictor placed within the matrix processing. Therefore, the pseudorandom offset applied to the differential channel in the predictor must be the difference of the pseudorandom sequence for each channel. This restores the correct pseudo-random offset when the other channel is added. This difference does not affect the output of the predictor, so there is no need to take the remainder modulo Δ.

[0226] <Extended Error> If the FIFO buffer cannot output the correct extension data, the extension signal will be incorrect. However, the decoder is adjusted so that each erroneous enhancement value is added to one sample and subtracted from the next. Therefore, the expansion error is first (1-z- 1 ) and then filtered by the predictor in the decoder. The frequency response of the predictor is a rough approximation of the current spectrum of the audio. The transfer function is (1-z- 1 ) is included to reduce the auditory impact of errors. If the decoder knows that the FIFO buffer cannot currently output the correct extended data, it is desirable to minimize the extended error by supplying a constant value to the combiner rather than extracting erroneous data from the FIFO buffer.

[0227] <Packet Structure> FIG. 12 shows the structure of an example encoded packet 1200. The packet in this example contains three blocks and two channels of audio. The packet starts with a packet header, after which three audio blocks 1220, 1221, 1222 are described at base layer precision. Each block includes a block header and base layer data corresponding to each channel. This series of information is called "forward coded data."

[0228] On the other hand, the enhanced data is treated separately, reflecting the variable delay FIFO buffering in the encoder and decoder. The remainder of the packet is filled with extension data 1230, whether large or small, taken from the encoder's FIFO buffer.

[0229] In this example, it is assumed that the extension data corresponding to block 1220 and part of block 1221 has already been transmitted. Therefore, the extension data corresponding to this block is extension data 1241B of the latter half of block 1221 and extension data 1242 of block 1222. Furthermore, in this example, there is room in the packet for two more blocks of extended data, so 1243, 1244, and 1245A are stored in succession. Here, the symbol A indicates that only the first part of the extension data for that block fits into the packet.

[0230] The decoder typically attempts to fully decode each block before moving on to the next, which requires reading the extended data from the FIFO buffer. When a packet arrives, the decoder's FIFO buffer may be nearly empty. In the example shown in FIG. 12, the FIFO buffer contains only the extended data corresponding to block 1220 and a portion of the extended data corresponding to block 1221. Therefore, to decode blocks 1221 and 1222, the extended data contained in this packet must be read, and therefore the extended data must be put into the FIFO buffer when the packet is received.

[0231] As shown in FIG. 12, the extension data is preferably formatted to fill the packet in reverse order from the end of the packet towards the end of the forward encoded data. The advantage of this layout is that the forward encoded data is of variable size. Therefore, the decoder cannot determine the end position of the forward-encoded data until it has completed entropy decoding for all blocks described by the forward-encoded data. Explicitly indicating where the forward-encoded data ends and the extension data begins would unnecessarily consume space within the packet. However, a decoder receiving a data packet must somehow know the length of the entire packet. If the extension data starts at the end of the packet, there is no need to provide a length information field in the packet.

[0232] If the extension data is stored in a format in which it is stored backward from the end of the packet, the decoder cannot determine the end position of the extension data until it has completed decoding all of the forward-encoded data. But this is not a problem. This is because when a packet is received, the entire packet is placed into the FIFO buffer, and after the forward-encoded data has been decoded, only the forward-encoded data can be removed from the FIFO buffer before the next packet is received.

[0233] Although we typically think of packets as streams of bits, in real computer systems, bits are stored in larger units such as bytes or words, so it is useful to have consistent endianness. For example, if the endian convention is low order bit first, then forward encoded data should be written and read low order bit first. However, because the extension data is stored backwards from the end of the packet, words of the extension data should be written and read in the opposite direction, i.e., most significant bit first.

[0234] Flexible Packetization Data is typically transmitted in packets. When an audio codec encodes audio blocks, there is typically a one-to-one relationship between coding blocks and packets. However, if the resulting packets are not suitable for the transmission channel, a packet splitting and reassembly layer, such as L2CAP on Bluetooth, may be required.

[0235] Such a packet splitting and recombining layer also has drawbacks. This creates data overhead and consumes bandwidth that could be used to improve sound quality. It also increases overall latency, and if an error occurs that cannot be retransmitted in time, for example a packet loss at the transmission layer, it may result in two corrupted audio codec packets instead of one.

[0236] The length of the block is preferably relatively short, about 1 to 2 milliseconds. This reduces the delay in the encoder's servo control loop, allowing it to respond more quickly to changes in data rate after lossless encoding, and allowing the noise floor to more accurately track the audio events that contribute to it. Each packet contains an integer number of blocks, and the packet in FIG. 12 contains three blocks. This integer value may be different for each packet.

[0237] To support such configurations, the packet header includes a field that specifies how many blocks are in the packet (or each block header may include a flag indicating whether it is the last block in the packet). Each block is preferably assigned a consecutive index, and the packet header also preferably includes a field that specifies the least significant bits of the block index corresponding to the first block in the packet. This allows the decoder to estimate how many blocks were described in the lost packet from the block index field of the next received packet, even if the packet is corrupted or otherwise not received. Then, after an appropriate amount of error concealment processing has been performed, the packet can be decoded at the appropriate time.

[0238] The advantage of including a variable integer number of blocks in each packet is that it decouples the block coding process from the packet characteristics required by the transmission channel, without incurring the drawbacks associated with the packet segmentation and recombination layer. The buffer storage process for extended data as described above is extremely important in realizing this flexible configuration. This is because by adjusting the amount of extension data included in a packet appropriately, it is possible to maintain an overall balance even if the number of blocks included in each packet varies slightly from the long-term average.

[0239] JPEG2025540705000015.jpg52169

[0240] The format preferably supports changing all parameters affecting decoding (e.g., prediction coefficients, pre-quantized step size or sampling rate changes, entropy coding table changes, etc.) at any block boundary. These parameter changes should not be restricted to only being allowed at packet boundaries. This means that the values ​​of such parameters (if they have changed since the previous block) are conveyed in the block header, rather than specifying values ​​for the entire packet in the packet header.

[0241] This configuration is advantageous in terms of delays due to buffer storage processing within the encoder. That is, when a block is input to the encoder and pre-quantized and losslessly encoded, coding decisions can be made without needing to determine the location of packet boundaries. Determining packet boundaries can be deferred until the coded blocks are output from the buffer and are actually transmitted. If the bandwidth capacity of the transmission channel suddenly drops, the original packet boundary plan may be altered. In this case, if there are sufficient computational resources, it is possible to retroactively correct the pre-quantization and encoding processes for the buffered blocks, resulting in improved sound quality. However, such computational resources are not always available, especially in real-time environments. In such cases, the ability to quickly change only the packetization strategy without touching already coded blocks becomes extremely important.

[0242] Another advantage is that if the data needs to be transmitted over a different transmission channel with different characteristics, the packetized coded audio can be reconstructed without re-encoding. For example, in the example above, successive packets contained 3, 3, and 4 blocks, in that order. It is possible to reassemble such a packet train onto another channel that is smaller but still capable of transmitting 500 packets per second. This can be achieved by parsing the packets sufficiently to identify the boundaries of the coding blocks and extension data, and then repacketizing them to contain two blocks in each new packet.

[0243] <Buffer synchronization on the decoder side> At the start of the encoded stream, the decoder finds its FIFO buffer empty. If the decoding process is started from that point and proceeds without error, the decoder will be able to remove from the FIFO buffer an amount that exactly matches the amount of enhanced data generated by the encoder. In such a situation, no synchronization is required.

[0244] However, in streaming audio formats, it is desirable to support the decoder being able to start decoding at any packet boundary in the middle of the stream, or to recover from missing packets. For this reason, part of the packet header preferably includes a field that allows the decoder to properly synchronize the amount of data in the FIFO buffer at the beginning of that packet.

[0245] FIG. 13 shows a data packet 1300 that includes a packet header 1310 , which includes a synchronization field 1311 . As in FIG. 12, the packet includes base layer data for blocks 1320 and 1321 followed by a second portion 1342 B of the enhancement data corresponding to block 1322 , followed by enhancement data 1343 .

[0246] When a packet arrives at the decoder, the contents of the decoder's FIFO buffer at that time are shown as 1301. The content begins with extension data 1340 for block 1320 contained in the received packet, followed by extension data 1341 for block 1321, and then a first portion 1342A of extension data corresponding to block 1322.

[0247] FIG. 13 shows how the total size of the extension data 1340, 1341, and 1342A is reflected in the sync field 1311. The presence of such a synchronization field allows the decoder to identify the correct extension data to use to decode the first block and subsequent blocks in a packet, provided that sufficient extension data is provided in subsequent packets once the decoding process has started (or resumed).

[0248] Even if there is not enough data provided, the decoder can synchronize the FIFO buffer to the correct size, since the size of the extension data does not depend on the value of its contents. In this way, the buffer occupancy is properly synchronized and can remain synchronized thereafter. As a result, even if the correct data is not immediately available, it will already be available by the time the decoder begins consuming data from the first receivable packet. Preferably, the decoder also knows how much initial data is missing so that it can avoid adjusting the audio using that missing data.

[0249] The sync field is preferably a simple count value representing the number of bits expected to be in the decoder's FIFO buffer. This value is a non-negative integer with a format-dependent maximum value, making it suitable for storage in a fixed-length field. Also, it is preferable not to include this synchronization field in every packet header, as this comes at a cost in data rate. By including this synchronization field more frequently, the duration of degraded playback that occurs when playback is started in the middle of a stream or after a packet is lost can be reduced. However, there is a lower limit to the duration of such quality degradation, which corresponds to the period of time that the enhanced data remains in the decoder's FIFO buffer.

[0250] <Buffer overflow> Ideally, the rate control servo mechanism will make buffer overflow a rare occurrence, but it is desirable to have some protection in place in case it does occur. Encoder-side buffer overflow occurs when a lossless encoder requires more data than the channel can provide. If a packet contains the base layer data for a block, all enhancement data associated with that block must have been sent in that packet or an earlier packet. Otherwise, the decoder buffer will underflow and lossless decoding of that block will not be possible.

[0251] If the encoder determines that there is insufficient free space in a packet to accommodate the required enhancement data, it can locally increase the data rate by increasing the packet size or by containing fewer base layer blocks (i.e., increasing the local packet density). Depending on the application requirements, local data rate increases may not be practical. In such cases, the next best solution from a sound quality perspective is to retroactively correct the pre-quantization decisions for blocks for which no part of the enhancement data was sent to the decoder in a previous packet.

[0252] However, tracing back requires computational resources that may not be immediately available. In such a case, it is necessary to accept that a buffer underrun occurs on the decoder side, making it impossible to perform lossless decoding and forcing playback using only the base layer signal for a certain period of time. This allows all extension data corresponding to blocks that cannot be fully contained (and the remaining blocks in that packet) to be discarded, thereby slightly easing buffer stress.

[0253] Preferably, the next packet uses a FIFO synchronization field so that processing of the correct extension data resumes as soon as possible. At the decoder, if there is insufficient data in the decoder FIFO to extend a block, the decoder will preferably recognize that the encoder buffer has overflowed and will cease converting audio using the extension data until synchronization is reset.

[0254] <Buffer underflow> Encoder-side buffer underflow occurs when the channel provides more capacity than the lossless encoder is using and the packet generator does not have enough data to fill the packets. This situation is more likely to occur in situations such as silent audio, where lossless encoders have lower data rates.

[0255] To eliminate the buffer underflow, the data rate must be reduced. This can be achieved by reducing the packet size, adding blocks within the packet (resulting in fewer packets than planned), or adding holes to the packet (resulting in only a portion of the data rate being used for audio).

[0256] Regarding the measure of providing a gap in a packet, it is necessary to explain how the decoder will identify the gap. In other words, it is necessary to ensure that the decoder does not mistake the missing portion for extension data and can correctly obtain the information contained therein.

[0257] FIG. 14 shows operation with a gap. FIG. 14-(a) shows a buffer underrun on the encoder side, which results in a missing portion in the middle of the packet. The left side of FIG. 14-(a) shows the related configuration shown in FIG. The lossless encoder 103 supplies the base layer data and enhancement data corresponding to each coded block to a delay line 110 and a FIFO buffer 109 in the buffer 108 . In the figure, delay line 110 is capable of storing four (k=4) base layer blocks 1420, 1421, 1422, and 1423. The corresponding extended data is stored in the FIFO, but some of it has already been sent in a previous packet, so only the tail portion 1441B of the extended data corresponding to block 1421 and extended data 1442 and 1443 corresponding to blocks 1422 and 1423 remain in the FIFO.

[0258] Here, a packet 1400 containing two blocks is required to be created. The packet contains a header 1410 and two base layer blocks 1420 and 1421 are embedded in the packet. The extended data 1441B, 1442, and 1443 are taken from the FIFO, but at this point the FIFO underruns, resulting in a missing portion 1450 in the middle of the packet. This gap can be exploited in a meaningful way to provide non-time-critical but useful data to the decoder. For example, album cover art.

[0259] FIG. 14-(a) also shows an example of what the next packet 1402 might look like. Packet 1402 includes a header 1412 , coded base layer blocks 1422 and 1423 , enhancement data 1443 and 1444 , and an additional missing portion 1452 .

[0260] FIG. 14(b) shows how data labeled as provided by packets 1400 and 1402 flows into the decoder's FIFO buffer 309. After decoding block 1423 , the read pointer of the decoder's FIFO is at location 1463 . Before block 1424 can be decoded, the read pointer must be advanced to location 1464 . So how can the decoder know that this data is missing and should be discarded from the FIFO (and interpreted as such, if possible)?

[0261] The answer is to label the data to indicate which packet it came from. The enhanced data 1444 was generated at the same time as the base layer data block 1424 . The encoder delay line has a capacity to store four base layer data blocks (k=4). Since packet 1400 begins with base layer data block 1420, this packet must have been sent before the enhancement data 1444 was encoded and placed into the encoder's FIFO buffer 109.

[0262] This observation allows the decoder to identify the dropout. If we assign index t to a base layer block, then just before decoding block t, we discard the data provided by the packet containing block tk from the decoder's FIFO. In this illustrated example, data from packet 1400 that contained block 1420 is discarded prior to decoding of block 1424 . Such data is considered to be a gap, not an extension.

[0263] To enable the detection of this missing portion, it is necessary to set the value of k, which is the size of the encoder's delay line, to the decoder. It is also necessary to attach a label to the data in the FIFO buffer to indicate which packet it was supplied by. This labeling is most easily accomplished by recording the current write pointer position after inserting each packet into the FIFO buffer. This allows the location to be identified where the read pointer should be advanced to discard the gap before later decoding the block.

[0264] <Pseudorandom offset synchronization> For lossless playback, the pseudo-random offset used in the pre-quantizer must be reproducible at the decoder. To achieve this, seed information must be transmitted within part (if not all) of the block or packet header.

[0265] Preferably, each channel is associated with a different pseudo-random sequence. The sequence is preferably long enough to prevent audible repeating patterns from occurring. A good sounding pseudorandom number generator has at least 32 bits of internal state, and sometimes more. Therefore, explicitly transmitting the internal state of the generator for the purpose of initializing it for each channel is communication expensive.

[0266] It is preferable that the number of samples is managed modulo a predetermined repeating period, and the pseudorandom number generation method is selected so that the internal state can be fast-forwarded efficiently. The decoder initializes the generator for each channel with a different standardized initial seed value for that channel, and fast-forwards the internal state based on a sample index derived from the stream. In this way, the generators for each channel are synchronized to generate pseudo-random offsets. When the number of samples reaches the repeat period, both the encoder and decoder reset the generator seeds for all channels to standardized values.

[0267] It is further preferred to maintain the block index count modulo an appropriate power of two. The sample index count is calculated by multiplying the block index count by the number of samples per block. Each packet header contains the low order bits of the block index count, and some packet headers contain the high order bits.

[0268] The advantage of this approach is that it also satisfies another desirable system characteristic. That is, if a packet transmission fails, it may not be known how many blocks were contained in the lost packet. However, when the next packet arrives, the number of missing blocks in the lost packet can be estimated to some extent by reading the lower bits of the block index in the packet header. This allows the decoder to know exactly how many samples are missing original data and need to be interpolated, and when to play back the received packets.

[0269] It should be noted that there are many known pseudo-random number generators that can be used, and the choice of which generator is beyond the scope of this specification. However, I would like to explain what I mean by the term "fast forward." For example, in a linear congruential generator, the state update equation takes the form: x n+1 = (ax n +c) modulo m therefore, x k = (a k x0+(a k -1)(a-1) -1 c) modulo m This becomes:

[0270] Using a conventional fast exponentiation algorithm, a k Since modulo m can be calculated efficiently in log2k time, if (a-1) has an inverse with respect to modulo m, and if that inverse is calculated and stored in advance, then the initial state x0 can be converted to x k can be found efficiently. This makes it possible to synchronize the pseudo-random number generator on the decoder side at any point in the stream.

[0271] When reducing the sampling rate of losslessly coded audio, as part of a pre-quantizer data rate reduction technique, both the pre-quantizer and the lossless decoder must consume a block of pseudo-random offsets, even though only a small number of pseudo-random offsets are actually required for the processing. This is to maintain synchronization between the sample index and the pseudorandom number generator seed at block boundaries.

[0272] <Entropy coding> Rice coding is a traditional method for encoding innovation data (prediction error data) in lossless codecs. However, this approach is not ideal for base layer coding. Rice coding is a type of Huffman coding optimized for the Laplace distribution, and although it does provide a reasonable fit to the innovations distribution, it is not an exact match. Furthermore, since a minimum of three bits is required per sample, operation at low bit rates is limited, and even at slightly higher bit rates, the benefits of buffering the extended data are diminished. This is because the bit rate allocated to the extension data becomes very small.

[0273] There are several other conventional entropy coding methods, but one that is particularly interesting is the Asymmetric Numeral Systems (ANS) method invented by Jarek Duda (see, for example, arXiv:0902.0271 and arXiv:1311.2540). Among them, tANS (table-driven ANS) is particularly promising, and its effectiveness can be further enhanced by adding some adaptation to the coding of base layer data.

[0274] The challenge to be addressed is that if tANS uses k-bit internal state, the probability of occurrence is 2 -KThe problem is that it is inefficient to encode less than 10 symbols. If the top bits of the base layer innovation data contain extreme outliers, it is possible to limit the range by splitting it more coarsely, but extreme values ​​will still have a low probability of occurring.

[0275] To address this issue, a preferred approach is to encode the most significant bits of the base layer innovation data in pairs. Specifically, the process is carried out in the following procedure. ·List the pairs of upper bits in descending order of occurrence probability. · Divide this list into groups. Each group, except the last, contains a power of two number of pairs. The alphabet in tANS coding is a collection of these groups, with additional bits used to specify which pairs within a group are being coded.

[0276] This method of grouping symbols and using additional bits to distinguish between group members resembles the recursive approach used in Huffman coding. In this method, two symbols with similar probabilities are grouped together into one composite symbol, and the last bit of the code is used to distinguish them.

[0277] This process can also be understood as encoding the pairs in polar coordinates. Each tANS symbol represents a group of pairs that form roughly concentric rings. Inside the ring, each pair has roughly equal probability.

[0278] Coding pairs of samples instead of individual samples has the advantage of halving the number of entropy encoding or decoding operations per block. Although tANS coding is computationally efficient, it involves parsing the bitstream into variable-length fields, which is not a cheap process in terms of computational resources.

[0279] It is possible to encode units larger than pairs, but pairs seem to be the best compromise. This is because implementations typically use lookup tables to map pairs to tANS symbols, and tables for triplets and quadruplets can become very large and unwieldy.

[0280] tANS ​​decoding decodes symbols directly from the decoding state without reading the bitstream. The bitstream is read after decoding and used to reload the decoding state before the next tANS state. This makes it easy to combine the extra bits to determine which pairs in the tANS symbols to decode, and the bits to reload the tANS state, into a single variable-length read from the bitstream.

[0281] <Servo control> As shown in FIG. 1, the rate control servo mechanism is responsible for selecting the quantization step size Δ to be used by the pre-quantizer based on information about the buffer stress and the currently supplied audio block.

[0282] Loop control is a well-studied field and does not require general explanation. However, the choice of Δ affects how the level of the pre-quantization noise varies with the audio signal, and there are also acoustic considerations that must be taken into account.

[0283] First, it is desirable that transient events in the audio do not increase the noise level that precedes the event. Second, it is preferable that the noise level be stable.

[0284] Figure 15 shows how these considerations can be combined with practical loop control considerations. It is preferable to avoid increasing Δ by analyzing the current block. This is because this would increase the noise level at the beginning of the block. On the other hand, the acoustic features that cause this block to be coded at a higher data rate than the previous block usually start somewhere in the middle of the block.

[0285] As a result, when an audio block is received (step 1500), a tentative Δ LOOP is selected (step 1501). Then, by analyzing the current audio block, we can find Δ and Δ LOOP The larger of (max(Δ,Δ LOOP )) estimates the amount of data required to encode the block (step 1502).

[0286] If this estimate is below the channel capacity (NO at step 1503), then there is no need to increase Δ. Even if the buffer is currently under enough stress to request an increase, the next block may be less stressed and the request may not occur, so the increase in Δ is postponed. Therefore, Δ has min(Δ,Δ LOOP ) is set and the process ends (step 1510).

[0287] Next, consider whether there is an imminent risk of buffer overflow. If there is little risk of buffer overflow and it is safe (NO in step 1504), information about the current block may be largely ignored. However, since we know that buffer stress worsens after encoding, even if a decrease in Δ is proposed, it can be forgone as it may be quickly reversed. Therefore, Δ has max(Δ,Δ LOOP) is set and the process ends (step 1511).

[0288] Alternatively, if there is a risk of buffer overflow (YES at step 1504), then allowing the buffer to overflow is more severe than allowing the noise to build up prior to the transient. Therefore, the objective must be abandoned for the time being and a decision must be made to stabilize the buffer stress based on all available information, including the current audio block (step 1512).

Claims

1. 1. A method for encoding input blocks of audio into data packets, each input block containing one or more channels of audio samples, the method comprising: receiving an input block of said audio; determining a quantization step size Δ for each audio channel within each block based on a rate control mechanism; determining a pseudo-random offset for each sample in the input block, the pseudo-random offset for each channel forming a pseudo-random sequence with a seed; quantizing each sample in the input block with noise shaping to obtain a pre-quantized block, wherein each sample value in the pre-quantized block is congruent modulo Δ with a corresponding pseudo-random offset; losslessly encoding the pre-quantized block based on Δ to obtain a lossless encoded data block, the dependence on Δ being such that smaller values ​​of Δ result in larger sizes of the lossless encoded data block, the lossless encoding being an injection mapping such that for any pre-quantized block, losslessly encoding another pre-quantized block that is also congruent modulo Δ with the corresponding pseudo-random offset necessarily results in a different block of lossless encoded data; a buffering step of buffering the losslessly encoded data block in a buffer; generating data packets for subsequent transmission based on the buffered data, at least some of the data packets including data representing the seed of the pseudo-random number sequence.

2. the rate control mechanism receives information about the buffer; The method of claim 1 , wherein the quantization step size Δ is determined based on the fullness of the buffer.

3. further comprising a step of separating the lossless encoded data in each lossless encoded data block into a first portion and a second portion and buffering the first and second portions separately in the buffering step; the first portion includes base layer data; the second portion includes extension data; decoding the base layer data without the enhancement data to obtain an approximation of the pre-quantized block; 3. The method of claim 1 or claim 2, wherein the data packets are generated such that each packet contains an integer number of base layer data blocks and any free space is filled with enhancement data.

4. The expanded data is stored in a first-in, first-out (FIFO) buffer; 4. The method of claim 3, wherein the data packets are generated using base layer data blocks from one end and FIFO buffered enhancement data from the other end.

5. further comprising an analyzing step of analyzing samples in the input block; The method of claim 1 , wherein the quantization step size Δ is further determined based on an analysis of the samples.

6. 6. The method of claim 5, wherein the quantization step size .DELTA. is increased if the analyzing step determines that a buffer is at risk of overflowing.

7. An encoder configured to encode input blocks of audio into data packets using a method according to any one of claims 1 to 6.

8. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 6.

9. 1. A method for decoding data packets into output blocks of audio containing one or more channels of output audio samples, comprising: receiving the data packet; extracting, for each channel and block based on the data of the data packet, information indicative of a quantization step size Δ and a seed; determining an offset for each sample in a block, the offset for each channel being a pseudo-random sequence based on the corresponding seed; decoding the data to obtain an innovation sample (prediction error) for each sample in a block based on the data; filtering the innovation sample with a quantization process to generate a filtered sample corresponding to each sample in the block, the filtered sample being based on a corresponding innovation sample and also congruent modulo Δ with a corresponding offset; generating an output block of audio based on the filtered samples.

10. The first part of each data packet is decoded without delay; 10. The method of claim 9, wherein the second portion of each data packet is buffered and delayed before decoding.

11. A decoder configured to use the method of claim 9 or claim 10 to decode data packets into output blocks of audio.

12. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 9 or claim 10.

13. A codec comprising a combination of an encoder according to claim 7 and a decoder according to claim 11.

14. 1. A method of encoding audio into data, comprising: receiving input blocks of audio, each input block containing one or more channels of audio samples quantized to the input audio precision; determining a pre-quantization precision for each channel of each block, where a block has one or more channels, and the pre-quantization precision is coarser than the input audio precision; generating a pre-quantized block, and if the pre-quantization resolution is coarser than the input audio resolution, generating the pre-quantized block by quantizing each sample in the input block to the pre-quantization resolution with noise shaping having a noise transfer function that follows a noise equal loudness curve in a corner frequency range of 1 kHz to at least 13 kHz; losslessly encoding the pre-quantized block to generate a losslessly encoded data block.

15. The method of claim 14, wherein the corner frequency is at least 15 kHz.

16. 16. The method of claim 14 or 15, wherein the gain of the noise transfer function forms a plateau in a frequency band above the corner frequency.

17. 16. The method of claim 14 or 15, wherein the gain of the noise transfer function in a frequency band above the corner frequency reaches a peak and then decays.

18. 18. The method of any one of claims 14 to 17, wherein the gain of the noise transfer function in a frequency band above the corner frequency varies depending on the input block.

19. 20. The method of claim 18, wherein the gain of the noise transfer function in a frequency band above the corner frequency follows a smoothed spectrum of the input block.

20. An encoder configured to encode audio into data using a method according to any one of claims 14 to 19.

21. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 14 to 19.

22. 1. A method for reducing audible transients occurring when noise shaping of an audio signal is stopped, the method comprising: modifying next n quantized sample values; multiplying a vector containing state variables of the noise shaper and / or a vector containing differences between one or more previous outputs and corresponding inputs of the noise shaper by a pre-computed matrix to obtain an intermediate representation containing up to n values; quantizing the n or fewer values ​​in the intermediate representation using direct or backsubstitution to generate n or fewer quantized intermediate values; multiplying the n or fewer quantized intermediate values ​​by a matrix of pre-computed integer values ​​to generate n modifications to the quantized sample values; applying the n modification values ​​to the quantized sample values.

23. 23. An apparatus configured to use the method of claim 22 to reduce audible transients occurring when noise shaping on an audio signal is stopped.

24. 23. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 22.

25. 1. A method for generating a compressed bitstream by losslessly compressing an audio signal comprising one or more channels, the method comprising, for each channel: receiving a sequence of audio samples, each audio sample having a quantized value that is an integer multiple of a corresponding step size Δ plus a corresponding pseudo-random offset; predicting the value of each audio sample by filtering previous audio sample values; generating a sequence of innovation samples by subtracting a corresponding predicted value from the value of each audio sample; generating a sequence of integer innovation samples by performing a rounding division on each innovation sample by a corresponding step size Δ; a symbol generating step of generating symbols based on the integer innovation samples; the method further comprising: generating base layer data by entropy encoding the symbols from all channels; generating the compressed bitstream based on the base layer data.

26. 26. The method of claim 25, wherein the sequence of audio samples is received as a plurality of blocks of audio samples, and wherein audio samples within a block are quantized using a different value of the step size Δ than audio samples in one or more other blocks.

27. 27. The method of claim 25 or claim 26, further comprising embedding information in the compressed bitstream specifying the corresponding step size Δ and the corresponding pseudo-random offset for the audio sample.

28. 28. A method according to any one of claims 25 to 27, wherein there is more than one channel.

29. 30. The method of claim 28, wherein audio samples of one channel are quantized with a different pseudo-random offset than audio samples of another channel.

30. 30. A method according to claim 28 or claim 29, wherein the step size Δ used in one channel is different to the step size Δ used in another channel.

31. the symbol generating step includes performing a further rounding division on each integer innovation sample; 31. The method of any one of claims 25 to 30, wherein the step of generating a compressed bitstream is also based on the remainder of the further rounded division.

32. 32. The method of claim 31, wherein the symbol generating step includes adding the remainder of the further rounded division to a subsequent integer innovation sample.

33. An encoder configured to losslessly compress an audio signal comprising one or more channels to generate a compressed bitstream using a method according to any one of claims 25 to 32.

34. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 25 to 32.

35. 1. A method for decoding a bitstream into an audio signal having one or more channels, comprising: receiving a compressed bitstream together with a specification of a step size Δ and a specification of a pseudo-random offset; entropy decoding a portion of the compressed bitstream to generate a sequence of decoded symbols for each channel; an integer innovation sample sequence generation step for generating, for each channel, a sequence of integer innovation samples based on the decoded symbols for that channel; generating, for each channel, a sequence of predicted samples; generating, for each channel, a sequence of pseudo-random offsets based on said pseudo-random offset specification; and calculating, for each channel, a sequence of audio samples, said calculating step comprising: multiplying each integer innovation sample in the sequence of integer innovation samples by a corresponding step size Δ to obtain a multiplication result; adding the corresponding predicted sample to the multiplication result to obtain an addition result; quantizing the sum to a value congruent to the corresponding pseudo-random offset modulo the corresponding step size Δ; A method, wherein each prediction sample in the sequence of prediction samples is generated by filtering a previously calculated audio sample.

36. 36. The method of claim 35, wherein one or more of the specifications are decoded from the compressed bitstream.

37. 37. A method according to claim 35 or claim 36, wherein the specification of the step size Δ allows Δ to take one or more different values.

38. 38. A method according to any one of claims 35 to 37, wherein one or more channels are designated.

39. 39. The method of claim 38, wherein the sequence of pseudo-random offsets is different for different channels.

40. 40. A method according to claim 38 or claim 39, wherein the step size Δ used in one channel is different to the step size Δ used in another channel.

41. 41. A method according to any one of claims 35 to 40, wherein the step of generating a sequence of integer innovation samples is also based on extension data decoded from another part of the bitstream.

42. 42. The method of claim 41, wherein the dependence on the extended data includes adding and subtracting values ​​to successive samples.

43. A decoder configured to decode a bitstream into an audio signal having one or more channels using a method according to any one of claims 35 to 42.

44. 43. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 35 to 42.

45. A codec comprising the combination of an encoder according to claim 33 and a decoder according to claim 43.

46. 1. A method for losslessly compressing a sequence of audio samples from an audio signal having one or more channels into data packets, comprising: dividing the sequence of audio samples into a sequence of audio blocks, each audio block comprising a plurality of audio samples; encoding each audio block into a data block and an enhancement block; generating a sequence of data packets, each data packet containing data from an integer number of data blocks and extension blocks; the data blocks contain information that allows an approximate reconstruction of the audio signal, the combination of the data blocks and the extension blocks contains information that allows a complete reconstruction of the audio signal; For all block indices t, Data block t is not included in a data packet after data block t+1, Any data contained in extension block t+1 is not contained in a data packet earlier than any data contained in extension block t; A method in which any data included in extension block t is not included in a data packet after data block t.

47. 47. The method of claim 46, wherein the integer number of data blocks included in a data packet is not constant for all data packets.

48. 48. A method according to claim 46 or claim 47, wherein the integer number of data blocks in one or more data packets is zero.

49. 49. An encoder configured to losslessly compress a sequence of audio samples from an audio signal having one or more channels into data packets using a method according to any one of claims 46 to 48.

50. 49. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 46 to 48.

51. 1. A method for decoding a sequence of data packets into audio samples in one or more channels, comprising: receiving a data packet in the sequence of data packets and parsing an integer number of data blocks and bufferable data from the data packet; placing the bufferable data into a first-in, first-out (FIFO) buffer; and decoding each data block in turn into audio samples using the enhanced data obtained from the FIFO buffer.

52. 52. The method of claim 51, wherein the integer number of data blocks parsed from a data packet is not constant for all data packets in the sequence of data packets.

53. 53. A method according to claim 51 or claim 52, wherein in one or more data packets in the sequence of data packets, the integer number of data blocks parsed from the data packet is zero.

54. 54. A decoder configured to use a method according to any one of claims 51 to 53 to decode a sequence of data packets into audio samples in one or more channels.

55. 54. A computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 51 to 53.

56. A codec comprising the combination of an encoder according to claim 49 and a decoder according to claim 54.

Images