Improvements to audio coding

Abstract

Methods and devices for improved encoding and decoding of audio signals integrating scalable codecs and buffering are described.

Description

[0001] IMPROVEMENTS TO AUDIO CODING

[0002] Field of the Invention

[0003] The present invention relates to methods and devices for improved encoding and decoding of audio signals.

[0004] Background to the Invention

[0005] Many methods of audio coding intrinsically produce variable rate encoded data on a variety of scales for a variety of reasons, from the stochastic nature of how data emerges from entropy coding, or a desire to move approximation error in time to reduce its impact or the variation with time of information content of the audio to be coded.

[0006] Many methods of transmitting audio, for example over wireless links, operate at a fixed or peak limited datarate.

[0007] The mismatch between the variable datarate from coding and a fixed datarate channel is often accommodated by a buffering model of some sort.

[0008] The present invention obtains various practical advantages through improved ways of integrating audio coding and buffering.

[0009] Summary of the Invention

[0010] According to a first aspect of the present invention, there is provided a method for encoding a sequence of input blocks of audio, each input block containing one or more channels of audio samples, the method comprising, for each input block: preconditioning the input block using one or more of: (i) gain adjustment (ii) quantisation, (iii) noise shaping, (iv) filtering, (v) bandsplitting and (vi) matrixing, furnishing a preconditioned block containing preconditioned samples; encoding the preconditioned block to furnish base data and enhancement data; placing the base data from an encoded block into an output stream; pushing the enhancement data into a first-in-first-out (FIFO) buffer, pulling a quantity of previously-pushed enhancement data from the FIFO; and, placing the pulled enhancement data into the output stream, wherein the preconditioned samples can be approximately reconstructed from the base data and more accurately reconstructed from the base data and the enhancement data together.

[0011] The invention is applicable to both lossy and lossless coding of audio signals. Audio encoders might exploit the typical non-flat spectrum of audio by filtering, lossy encoders will reduce the signal’s entropy by quantisation (which might be noise shaped to reduce audibility), gain may be applied to present the latter processes of the encoder with a consistent signal level and other processes may be performed. The invention does not relate to these initial encoding steps so we group them into an optional preconditioning step.

[0012] The invention builds on the idea of scalable coding, where encoded data divides into a base stream and extension stream so that the best experience is delivered by a decoder that has access both streams but loss of the extension stream still delivers an acceptable experience.

[0013] The enhancement data is FIFO buffered before formatting into the output stream whilst the base data experiences a constant delay. In this way, the buffering allows variable rate data from the encoding process to be matched to a constant or peak rate limited stream whilst the straightforward treatment of the base data ensures it is reliably available to decoders.

[0014] Preferably, the output stream is divided into packets, the steps of placing are configured to place base data from an integer number of preconditioned blocks contiguously into a packet and the quantity of enhancement data to be pulled is determined in dependence on the space remaining in the packet after the base data has been placed.

[0015] In this way, each packet containing base data from an integer number of blocks ensures that a decoder in possession of that packet can produce an acceptable rendition of that portion of audio regardless of what has happened to surrounding packets.

[0016] Base data will often be variable sized and filling the remainder of the packet with enhancement bits from the FIFO provides a flexible and fine-grained way to fill that unused space. Any bits remaining in the FIFO can be used to fill unused space in subsequent packets.

[0017] Preferably, the enhancement data is placed after the base data in the packet and early bytes of pulled enhancement data are placed later in the packet than subsequent bytes of pulled enhancement data.

[0018] Alternatively, the base data is placed after the enhancement data in the packet and base data furnished from early samples are placed later in the packet than base data furnished from later samples.

[0019] Writing the latter section in the packet backwards allows a decoder to parse both sections before completely parsing the earlier section without incurring the overhead of requiring the packet to contain data explicitly indicating the boundary. This is useful for low latency operation of the decoder. Preferably, some but not all packets contain a header field specifying how many total bits of enhancement data have been transmitted in earlier packets from both the first preconditioned block whose base layer data is placed in the packet and subsequent blocks

[0020] In this way the decoder can check if its FIFO buffer contains the expected amount of data and resynchronise it if required. The check only needs to be made occasionally, so there is no need to incur the data overhead of including the field in every packet.

[0021] Preferably, the step of losslessly encoding comprises: splitting the preconditioned samples into base symbols and enhancement symbols, entropy coding the base symbols to furnish the base data; and, deriving the enhancement data in dependence on the enhancement symbols.

[0022] Base symbols will usually have a non-uniform probability distribution and so coding efficiency benefits from entropy coding to produce the base data. Enhancement symbols though will often have a near uniform probability distribution and the complexity of entropy coding in producing the enhancement data will often not be warranted.

[0023] Preferably, when the base data has a low datarate, the step of deriving the enhancement data comprises entropy coding the enhancement symbols conditionally on the base symbols.

[0024] When the base data has a low datarate, the enhancement symbols often distinguish between values of the preconditioned samples that have significantly different probabilities. In this case there can be worthwhile coding efficiency gains from entropy coding the enhancement symbols. The conditional probability distribution for each enhancement symbol based on the corresponding base symbol will be very different to the aggregate distribution of enhancement symbols so the entropy coding benefits from being based on probabilities derived from the base symbols.

[0025] Preferably, the step of entropy coding the enhancement symbols is performed only on blocks where for any value of base symbol there are only two possible values of enhancement symbol.

[0026] Entropy coding comes at a computational cost, which is often lower when the symbols to be coded only take binary values. Moreover when there are more than two possible values of enhancement symbols for any base layer symbol, there is an alternate approach of conveying more information in the base layer data instead of the enhancement layer. Thus the cost benefit ratio of applying entropy coding to the enhancement data is maximised when it just codes for two possible values on each enhancement symbol.

[0027] Preferably, the step of splitting incorporates noise shaping.

[0028] In this way, the base layer symbols can represent an audibly closer approximation to the fully decoded signal compared to the straightforward approach of diverting one or more least significant bits from the preconditioned samples into the enhancement data. This usually comes at a datarate cost since noise shaping increases the entropy of the base symbols.

[0029] Preferably, the step of splitting maps pairs of successive preconditioned samples to base symbols and maps two pairs of successive preconditioned samples to each base symbol, wherein the two pairs of successive preconditioned samples mapped to a base symbol sum to the same value.

[0030] In this way, there is one base symbol per pair of preconditioned samples, thus the computational cost of entropy coding is incurred less frequently. Moreover since all pairs which map to any given base symbol have the same sum of samples, the sum of samples is invariant to loss of enhancement data. This improves base layer audio quality by noise shaping the step of splitting without increasing the entropy of the base symbols.

[0031] According to a second aspect of the present invention, there is provided an encoder adapted to encode input blocks of audio to an output stream using the method of the first aspect.

[0032] According to a third aspect of the present invention, there is provided a computer readable medium comprising instructions that, when executed by one or more processors, cause said one or more processors to perform the method of the first aspect.

[0033] According to a fourth aspect of the present invention, there is provided an encoder for an audio signal, the encoder comprising: an input configured to receive the audio signal; an output for furnishing a compressed stream of data, the compressed stream comprising base data and enhancement data, a splitter configured to separate the audio signal into base symbols and enhancement symbols, a first entropy coder configured to receive the base symbols and to furnish the base data; a second entropy coder configured to receive the enhancement symbols and to furnish the enhancement data; and a combiner configured to receive the base data and the enhancement data and to furnish the compressed stream, wherein the combiner comprises a variable delay unit whereby the enhancement data within the compressed stream has a varying time relationship to the base data.

[0034] Typically the audio signal is a PCM audio signal and, following some preprocessing, the splitter splits each preprocessed PCM word into its more significant bits to form a base symbol and its less significant bits to form an enhancement symbol.

[0035] The base data and enhancement data within the compressed stream, taken together, allow a PCM signal, either original audio or preprocessed, to be reconstituted.

[0036] The base data relating to a particular portion of the audio signal can thus be placed at a predetermined position within the compressed stream while the enhancement data may flow as required to accommodate variations in channel capacity or base data rate.

[0037] Preferably, the second entropy coder is configured to furnish the enhancement data in dependence on the base symbols.

[0038] Knowledge of a base symbol allows the second entropy coder to know more accurately a probability distribution to assume when coding a corresponding enhancement symbol and hence to code the enhancement symbol more efficiently.

[0039] In some embodiments the compressed stream comprises packets of data. Packets are required by most modem digital transmission systems.

[0040] Preferably, the combiner comprises a fixed delay unit to receive the base data and a first-in-first-out (FIFO) buffer to receive the enhancement data.

[0041] The FIFO buffer provides allows enhancement data transmission to be delayed in response to surges in data from the entropy coder. The fixed delay in the combiner allows base and enhancement data to be synchronised without delaying the base data in a decoder.

[0042] According to a fifth aspect of the present invention, there is provided a method for decoding packets of data to output blocks of audio each containing one or more channels of output audio samples, the method comprising: receiving a packet of data; retrieving a portion of base data from the packet; retrieving first enhancement data from the packet and pushing the first enhancement data into a first-in-first-out (FIFO) buffer; pulling second enhancement data from the FIFO buffer; decoding the portion of base data and the second enhancement data to furnish a block of preconditioned samples; and postconditioning the block of preconditioned samples using one or more of (i) gain adjustment, (ii) filtering, (iii) bandjoining and (iv) matrixing, furnishing the output blocks of audio.

[0043] The invention provides for the decoding of a stream of packets each of which contains one or more contiguous portions of compressed base data, each portion coarsely representing a block of audio. In addition the packet may contain enhancement data which does not necessarily relate to any of the blocks coarsely represented within the same packet but rather may allow the decoder to enhance the decoding of later packets within the stream. Accordingly, the enhancement data, if any, is not acted on immediately but is pushed into a FIFO.

[0044] To provide enhanced decoding of a block represented by a portion, the base data is decoded together with second enhancement data pulled from the FIFO. The second enhancement data will in general have been pushed to the FIFO when receiving previous packets but some or all of it may have been pushed when receiving the current packet.

[0045] Finally, some preconditioning processes used by an encoder (e.g. gain adjustment and prediction filtering) can be reversed in a postconditoning process prior to furnishing output blocks of audio.

[0046] Preferably, the step of decoding comprises: entropy decoding the base data to furnish base symbols; entropy decoding the second enhancement data in dependence on the base symbols to furnish the preconditioned samples.

[0047] In some cases, and particularly when encoding at low data rates, it is advantageous to entropy code enhancement symbols in dependence on the corresponding base symbols. In which case the decoder must first entropy decode base data to base symbols and then the enhancement data can be decoded in dependence on the base symbols. In some cases, this process will yield preconditioned samples directly.

[0048] Alternatively, the step of decoding comprises : entropy decoding the base data to furnish base symbols; entropy decoding the second enhancement data to furnish enhancement symbols in dependence on the base symbols; and combining the base symbols with the enhancement symbols to furnish the preconditioned samples. The base data may be decoded to symbols and then the enhancement data may be decoded to symbols in dependence on the base symbols, following which the two sets of symbols may be combined to furnish preconditioned samples in a process that mirrors the splitting step used by some encoders.

[0049] Preferably, base data precedes first enhancement data in the packet and first enhancement data towards the end of the packet is pushed into the FIFO before first enhancement data towards the beginning of the packet.

[0050] Alternatively, first enhancement data precedes base data in the packet and base data towards the end of the packet is entropy decoded before base data towards the beginning of the packet.

[0051] Reading the latter section of the packet backwards allows the latter section to be processed before the boundary of the two sections is deduced. This allows low latency operation of the decoder without requiring the packet overhead of a field specifying the boundary between the two sections.

[0052] Preferably, some packets contain a field specifying how full the decoder’s FIFO buffer should be, in which case the method comprises the additional steps of: reading a header field from the packet; and adjusting the amount of data in the Fl FO buffer to match the number of bits specified in the header field.

[0053] In this way the decoder can recover synchronisation of its FIFO buffer after missing a packet or starting decode mid-way through a stream of packets.

[0054] At low datarates, preferably a pair of successive preconditioned samples are furnished from each base symbol, said pair being selected in dependence on the second enhancement data from two possible pairs of preconditioned samples that map to the base symbol, wherein the two possible pairs of successive preconditioned samples each sum to the same value.

[0055] In this way, when the encoder has used the corresponding method, a pair of decoded samples can be produced from a single base symbol whilst the sample sum being invariant to the enhancement data reduces the audio impact of missing enhancement data.

[0056] According to a sixth aspect of the present invention, there is provided a decoder adapted to decode packets of data to output blocks of audio using the method of the fifth aspect.

[0057] According to a seventh aspect of the present invention, there is provided a computer readable medium comprising instructions that, when executed by one or more processors, cause said one or more processors to perform the method of the fifth aspect.

[0058] According to an eighth aspect of the present invention, there is provided a codec comprising an encoder according to the second aspect in combination with a decoder according to the sixth aspect.

[0059] According to a ninth aspect of the present invention, there is provided a decoder for a stream of packets of data representing an audio signal, the packets each containing base data plus enhancement data, the enhancement data having a varying time relationship relative to the base data, the decoder comprising: an input configured to receive the stream of packets of data; a memory coupled to the input, a first entropy decoder coupled to the input and configured to receive base data and to furnish base symbols; a second entropy decoder coupled to the memory and configured to receive enhancement data and to furnish enhancement symbols; and a joiner configured to receive the base symbols and the enhancement symbols and to furnish decoded audio samples to an output of the decoder.

[0060] The first entropy decoder decodes the base data within a current packet but the second entropy decoder decodes data supplied on demand from the memory. Thus the second entropy decoder decodes enhancement data that may have been transmitted in an earlier packet. This feature allows the second entropy decoder to temporarily demand enhancement data at a higher rate than a preceding transmission channel is able to supply.

[0061] In some embodiments, the memory is configured to store recently-received packets of data.

[0062] The memory stores packets received from the input and provides random access to recent packets so that the second decoder may select the enhancement data it needs.

[0063] In some embodiments, the memory is configured as a FIFO buffer for enhancement data.

[0064] A Fl FO buffer provides a convenient and economical way to organise the memory, the second decoder using a ‘pull’ operation to retrieve the enhancement data it needs.

[0065] Preferably, the second entropy decoder is configured to decode enhancement data in dependence on base symbols received from the first entropy decoder.

[0066] In this preferred embodiment, the decoder is adapted to decode a stream in which the enhancement data has been coded in dependence on base symbols internal to an encoder. Accordingly, the decoder must first entropy-decode the base data to furnish base symbols and then it can decode the enhancement data in dependence on the base symbols.

[0067] As will be appreciated by those skilled in the art, the present invention is capable of various implementations according to the application, as will be apparent from the following discussion.

[0068] Brief Description of the Drawings

[0069] Embodiments of the invention will now be described by way of example with reference to the accompanying figures in which:

[0070] Figure 1 shows the main features of a generic time domain codec;

[0071] Figure 2 shows the concept of a scalable codec;

[0072] Figure 3 shows how a scalable codec can combined with variable delay buffering on the enhancement data only;.

[0073] Figure 3a shows how the encoder places base layer data for a block into a packet and then fills up the remaining packet free space with enhancement data from a FIFO buffer;

[0074] Figure 3b shows how the corresponding decoder decodes base layer data directly from the packet but pushes the enhancement data into a FIFO buffer. It produces a full quality decode by pulling previously pushed enhancement data from the FIFO buffer;

[0075] Figure 4 shows how the FIFO buffer fullness in the encoder and decoder varies but always adds up to match the constant delay experienced by base layer data. Figure 4a shows an example of an emptier encoder buffer and a fuller decoder buffer, while Figure 4b shows an example of the reverse situation;

[0076] Figure 5 shows how several blocks of base layer data might be placed in a packet and the remaining space filled up with enhancement data backwards, starting from the end of the packet;

[0077] Figure 6 shows how an occasional packet can contain a header field for synchronisation describing how much data should be in the decoder FIFO immediately prior to receipt of the packet;

[0078] Figure 7 shows an illustrative probability density function for the innovation signal and how an efficient scalable codec might split this up into base layer symbols (each representing the areas between solid lines) and enhancement symbols (representing the finer divisions within the base layer symbol). Three choices of subdivisions are shown representing different amounts of base layer and enhancement entropy; Figure 8a shows a scalable encoder splitting the quantised innovation into base symbols which are entropy encoded for the base layer and enhancement symbols. A corresponding decoder entropy decodes the base symbols and recombines them with the enhancement data if available, but otherwise uses value for enhancement data corresponding to a mid-value of the range represented by the base symbol;

[0079] Figure 8b shows the encoder adding the previous enhancement symbol to the quantised innovation and the decoder subtracting it. This ensures the consequence of missing enhancement data has no DC content, improving the audible outcome;

[0080] Figure 8c adds the further feature to Figure 8b of entropy encoding the enhancement data. The probabilities of each value of enhancement are needed for the entropy coding and depend on the corresponding base layer symbol. Consequently dotted paths are shown connecting the base layer symbols to the enhancement entropy encoder and decoder;

[0081] Figure 9 shows a particularly useful case where base symbols describe exactly two (circled) possible pairs of samples; and

[0082] Figure 10 shows an example of how a codec might combine multiple aspects of the invention.

[0083] Detailed Description

[0084] Figure 1 shows an example audio codec. It is not intended to show any specific codec but a generic representation of typical features. The encoder is shown in Figure 1a and the corresponding decoder is shown in Figure 1 b.

[0085] As shown in Figure 1a, a block containing a channel of audio 100 is presented and processed sample by sample.

[0086] Each sample has subtracted from it a prediction 123 of its value. This reduces the signal power, exploiting audio’s characteristic of not having a flat spectrum. The result 120 of this subtraction is called the innovation - as it represents what was new and unpredicted about the sample.

[0087] A division, by an amount 107 chosen for the block, normalises the innovation, so that its probability density function is reasonably stable from block to block.

[0088] Innovation samples are then quantised by the quantiser 101 producing quantised innovation 121 to reduce entropy and so encoded datarate at the cost of introducing approximation error. The tradeoff between datarate and approximation error is set by a precision parameter 108 for the quantisation, defining its stepsize. Figure 1 also covers the case of a lossless codec such as FLAC (Free Lossless Audio Codec), where the precision 108 will match the level 107 and the quantiser 101 will do nothing.

[0089] The quantised innovation samples 121 are interpreted as symbols for coding with an entropy coder 102 to furnish data 122. The choice of coding table is also parameterised by the precision parameter 108. We don’t need to specify any specific method of entropy coding; arithmetic coding, range coding or ANS are all possible entropy encoding methods that could be used in an audio codec.

[0090] A buffer 103 might be used to accommodate the mismatch between the variable datarate produced by the encoder and a fixed datarate channel and the data is then formatted into a packet 104 for onward transmission.

[0091] The above prediction may be calculated by filtering 106 previous quantised innovation samples 121 (after inverting 105 the action of the quantiser and level division).

[0092] Except at very low datarates (where spatial information may be discarded), each channel of multichannel audio is often processed separately and the resultant data multiplexed into the output packet. There may be an initial matrixing stage to exploit inter-channel correlation but it rarely amounts to much more than processing the difference of two channels instead of one of the supplied channels.

[0093] In the corresponding decoder shown in Figure 1 b, the encoder actions are inverted in reverse order, except for quantisation which irreversibly discarded information.

[0094] Data 132 from a received packet 114 may interact with a buffer 113 to reverse the effect of encoder buffering 103. It is then entropy decoded 112 to produce the quantised innovation 131 which replicates 121 in the encoder. The quantised innovation 131 is inverse quantised 115 and restored to the correct level to generate an approximation 130 (accurate except for error introduced by the quantiser 101) to the innovation 120. Prediction 133 is produced by filtering 116 prior samples of 130. Adding the prediction 130 then produces the output block 110 of audio.

[0095] Figure 2 illustrates the concept of a scalable codec. A block of audio 200 is encoded to produce base data 210, describing an approximation of the audio.

[0096] The coding error can be generated by decoding 202 the base data and subtracting it from the original audio. This error is encoded (either lossily or losslessly) to produce enhancement data 211.

[0097] In Figure 2, a first decoder 204 only has access to the base data 220 (corresponding to the base data 210) and can produce decoded audio at the corresponding quality level. A second decoder 205 has access to both base data 220 and enhancement data 221 (corresponding to enhancement data 211) and can produce higher quality decoded audio.

[0098] Ideally all decoders would have access to both base and enhancement data and produce the higher quality decoding, but there are various possible reasons why access to the enhancement data might not be available.

[0099] Perhaps there is not enough bandwidth to the first decoder and the enhancement data need discarding.

[0100] Perhaps the channel is susceptible to errors or lost packets. Error correction can be expensive (in bandwidth or computation) and with scalable coding it can be focussed on the comparatively small amount of base data so that enhancement data transmission remains fragile but base data transmission is robust.

[0101] In either case, acceptable reproduction can occur from the base layer data, even when problems with the channel make best quality reproduction impractical.

[0102] Desirably, the combination of base and enhancement data would require no higher bandwidth than would a single encoding to the higher quality level. The simple scheme shown in Figure 2 of encoding the coding error from an arbitrary lossy encoder is unlikely to come close to achieving this.

[0103] Differential buffering

[0104] According to an aspect of the invention, in a scalable encoder variable delay buffering is applied to the enhancement data but not to the base layer data.

[0105] An audio codec will typically output varying rate data across various scales, whether this variance arises from entropy coders coding symbols of differing probability to differing numbers of bits, variation in the characteristics of the audio to be coded or codec preferences about how best to distribute approximation error with time.

[0106] For some transmission channels, such as storing audio in a file system, this is not a problem. But many transmission channels have a fixed or peak limited datarate. Variable delay buffering is very useful to couple the variable rate data arising from an encoder to a fixed or peak limited datarate channel, for example MP3 uses a bit reservoir.

[0107] However, buffering comes with disadvantages, especially complicating situations like dealing with missing packets or seeking within a stream.

[0108] Instead of applying buffering to the whole data output from the encoder, we propose using a scalable codec and applying variable delay buffering to the enhancement data only. In this way, the disadvantages of buffering only threaten reliable delivery of the enhancement data and a scalable codec can produce acceptable decoded audio even if the enhancement data is temporarily unavailable.

[0109] Figure 3 shows a way of implementing this differential buffering, using a first-in- first-out (FIFO) buffer.

[0110] We will use the term “push” for inserting data at the head of the FIFO buffer and “pull” for removing data from the tail of the FIFO buffer. We will also use the terms “unpush” and “unpull” for the rare operations of removing recently pushed data from the head and re-inserting recently pulled data at the tail.

[0111] Figure 3a shows an encoder, where a block of audio 300 is presented to a scalable encoder 301 which produces base data 307 and enhancement data 308 describing the block of audio. The base data for the block is treated as an indivisible unit, which is passed into a delay line 302. The enhancement data for the block is treated as a sequence of bits which is pushed into a FIFO buffer 303 in which it will be delayed by some variable amount.

[0112] To produce a packet 304 for the transmission channel, an integer number of base data blocks 305 are pulled from the delay line and placed sequentially into the packet. However much space this leaves in the packet is filled with enhancement data 306 pulled from the FIFO buffer 303.

[0113] So long as the FIFO buffer neither underflows or overflows, the available data capacity in the packets is fully utilised.

[0114] To avoid buffer under or overflow, a lossy encoder would preferably have a rate control mechanism ensuring that the encoder FIFO buffer neither overflowed nor underflowed by adjusting the degree of codec approximation error to match the datarate to the channel.

[0115] A lossless encoder does not have the luxury of adjusting codec approximation error, but it could respond to the threat of buffer overflow by switching to a lossy mode. Or perhaps there might be the possibility of responding to the threat of buffer overflow by increasing the capacity of the transmission channel. For example, if available packet sizes are coarsely quantised, then the choice of packet size can match channel capacity to the audio entropy over the medium term whilst the FIFO buffering matches encoder output to channel capacity over the short term thus ensuring that all channel capacity demanded is well utilised.

[0116] Figure 3b shows a corresponding decoder.

[0117] On receipt of a packet, the base layer data 315 it contains can be parsed by the scalable decoder 311 . The remaining space in the packet contains enhancement data 316 which is pushed into the decoder’s FIFO buffer.

[0118] For a full decode of both base and enhancement data to decoded audio 310, the decoder pulls enhancement data 318 from its FIFO buffer 313. Much of this will typically be enhancement data that was pushed when decoding previous packets but depending on the buffer fullness some of this data may be enhancement data from the current packet. The enhancement data has no visible block boundaries but is pulled bitwise from the FIFO buffer as and when required to complement the parsing and decoding of the base layer data 315. Since it has access to both base and enhancement data, the scalable decoder can perform the best quality decode.

[0119] Should a packet be lost in transmission, then missing packet concealment will probably be required to cover the timespan described by its base layer.

[0120] When the next packet arrives, its base layer data 315 can be decoded by the scalable decoder to give an acceptable decode without requiring the missing packet concealment to be extended further.

[0121] Full quality decoding will have to wait until the decoder’s FIFO buffering has been resynchronised and data from after the missed packet flows through to the FIFO’s output. But the codec’s scalability ensures adequate audio performance is maintained through the waiting period. The impact of the missing packet is minimised to concealment of the actual gap and audio degradation from full to acceptable quality for a recovery period. Since a short term drop back to acceptable audio quality is not a severe consequence, the codec bitstream format does not need to incur much overhead in order to minimise the duration of the quality drop.

[0122] Figure 4 clarifies how the variable delay in both the encoder and decoder sums to a constant delay duration, matching the constant delay applied to the base layer data.

[0123] In Figure 4a, the encoder FIFO buffer is almost empty, so there is little delay in the encoder FIFO 303 applied to the enhancement data. We label this delay f (which is not usually integer) blocks. In contrast, the decoder FIFO 313 is almost full and imposes a delay of N-f blocks, so that the total delay is N blocks matching the constant delay in the base layer path.

[0124] Suppose the encoder produces data faster than the channel capacity for a short time. Data is pushed into the encoder FIFO 303 faster than the channel can transmit it and the encoder FIFO 303 fills up to the situation illustrated in Figure 4b. At the decoder side, the decoder is consuming data faster than the transmission channel is pushing data into the decoder FIFO, and so the decoder FIFO 313 empties, but the total delay across both FIFOs remains constant.

[0125] Establishing that this constant delay matches the delay in the base layer data path requires synchronisation which will be discussed later with reference to Figure 6. Packet layout

[0126] Figure 5 shows a desirable way to organise the enhancement data in the packet.

[0127] It would be natural for data pulled from the encoder FIFO to flow into the packet so that the first bit of enhancement data immediately follows the base layer data.

[0128] But we propose that it is better for the first bit of enhancement data to be at the end of the packet, filling the packet backwards until the last bit of enhancement data is the bit immediately following the base layer data.

[0129] This is because, in order to access the first bit of enhancement data, the decoder needs to deduce where in the packet it lies. If the enhancement data is laid out in forwards order then either the packet format has to contain a header field indicating where the boundary lies (costing datarate) or the base layer data needs to be completely parsed beforehand (imposing an awkward ordering on decoder operations, especially if the packet describes multiple blocks). Whereas if the enhancement data is laid out in reverse order, the first bit of enhancement data is known to be at the end of the packet and a decoder in receipt of a packet already knows how big it is and hence where the end lies.

[0130] The essential component in the desirable layout is that both base layer data and enhancement data run from each end of the packet towards the middle. Alternatively, it is equally sensible for the enhancement data to run from the start of the packet forwards towards the middle and the base layer data to run backwards from the end of the packet towards the start.

[0131] Figure 5 shows the layout of an example packet describing 3 blocks of 2 channel audio.

[0132] After a packet header 510, three blocks 520, 521 and 522 of base layer data are arranged, each having a block header, data describing channel 0’s base layer and data describing channel 1’s base layer.

[0133] The rest of the packet is filled with enhancement data 530.

[0134] We number the corresponding enhancement data for blocks 520, 521 and 522 as 540, 541 and 542. However, enhancement data is advanced relative to the base layer data and flowed into the packet without regard for block boundaries. In this example, 540 and the first portion of enhancement data 541 have already been transmitted in earlier packets and so are not shown. Enhancement data starts with the second half 541 B of block 541 and continues with 542 and subsequent enhancement blocks 543, 544 and the first half 545A of block 545. These subsequent blocks won’t be needed for decode of this packet, but will remain in the decoder FIFO buffer for decode of subsequent packets.

[0135] Enhancement data 530 fills the remainder of the packet backwards from the end of the packet. Hence 541 B is situated later in the packet than 542 etc. This backwards arrangement is helpful because the decoder will need to read enhancement data 541 B when decoding block 521. This is likely to be before it has parsed block 522 and so determined the boundary between base layer data and enhancement data.

[0136] Decoder FIFO synchronisation

[0137] Figure 6 shows how a field in the packet header can be used to synchronise the decoder FIFO.

[0138] Such a field does not need to be contained in every packet header, because the use of scalable coding means acceptable decode can proceed when the decoder doesn’t know how full its FIFO buffer should be and so cannot identify the correct enhancement data. (Such situations can arise starting decode part way through a stream or after a missing packet).

[0139] Packet 600 contains a header 610 and two blocks 620, 621 of base layer data. The remainder of the packet is filled up with enhancement data 642B (which is the latter part of enhancement data for subsequent block 622) and 643 (corresponding to block 623).

[0140] On receipt of packet 600, the decoder FIFO contents are shown in 601 . Decode of block 620 has not yet started (but the previous block has been decoded). Consequently, the next data to be pulled out of the decoder FIFO buffer is the corresponding enhancement data 640 for block 620. The FIFO buffer must also contain enhancement data 641 for block 621 and the first portion 642A of enhancement data for block 622 up to the point of the data in packet 600.

[0141] The encoder about to transmit packet 600 knows the size of 640, 641 and 642A in bits, so it can add them up. This sum computes the total amount of enhancement data starting with that corresponding to block 620 (the first base layer block in packet 600) that has been transmitted in previous packets. The calculated total can then be included in a field 611 of the packet header 610.

[0142] This field specifies exactly how many bits should be in the decoder FIFO buffer on receipt of packet 600. If the decoder reads this field and finds it does not match the current size of its FIFO buffer then it knows that its FIFO has lost synchronisation with the encoder’s FIFO. It can adjust the size of its FIFO buffer to contain the correct amount of data by pulling any excess bits if the buffer was too full or by un-pulling recently pulled data if the buffer was too empty.

[0143] Efficient scalable codec

[0144] We now turn our attention to another aspect of the invention concerning design features of a scalable codec that does not sacrifice much efficiency compared to its non-scalable equivalent, nor add much complexity, and thus makes the above differential buffering concept attractive. In Figure 1 , our interest starts with the quantised innovation signal 121 produced by the quantiser 101 . This will define the full quality signal produced by a decoder from both base and enhancement data together.

[0145] That quantiser 101 , and everything to its left, we shall generically describe as preconditioning the signal, a process that might comprise any or all of:

[0146] • matrixing to exploit redundancy between different channels of a multichannel audio signal

[0147] • bandsplitting

[0148] • gain

[0149] • filtering (performed in Figure 1 by subtracting the prediction) to exploit the non-flat spectrum of audio

[0150] • quantisation to reduce the signal entropy (and so encoded datarate) at the cost of approximation error

[0151] • noise shaping around the quantiser to craft the approximation error for reduced audibility

[0152] (To avoid extraneous detail, not all of those processes are shown in Figure 1 which is only a representative example of an encoder).

[0153] Consequently we will sometimes use the term preconditioned signal to refer to signal 121 , reflecting that it has undergone some of these typical processing operations. We will also use the term preconditioned signal to refer to the corresponding decoder signal 131 to which will be applied some or all of the following postconditioning operations to invert corresponding encoder operations and furnish the audio output signal:

[0154] • matrixing

[0155] • bandjoining

[0156] • gain

[0157] • filtering (performed in Figure 1 by adding the prediction)

[0158] Figure 7 shows a curve representing a possible probability density function of the innovation signal 120. The vertical lines show dividing points between ranges that quantise to different values of the quantised innovation signal.

[0159] In Figure 7a, half of the vertical lines are solid. They represent the boundaries of base level symbols, each of which covers two neighbouring possibilities for the quantised innovation. For example, base layer symbol 700 covers the range of quantised innovation symbols 701 and 702, and base layer symbol 710 covers the range of the next two quantised innovation symbols.

[0160] The enhancement data then signifies which of the two possible values of quantised innovation corresponding to a base level symbol was actually the quantised innovation. It takes one of two values for each sample, perhaps 0 for the left hand value and 1 for the right hand value. So quantised innovation symbol 702 would be represented by base layer symbol 700, enhancement datum 1 .

[0161] Or the enhancement data could be the actual quantised innovation itself, since only two values of the quantised innovation have non-zero conditional probability given the base layer symbol.

[0162] Figure 7b shows base symbols representing four neighbouring quantised innovation values, so the enhancement data represents a four way decision for each sample. Now more of the information in the quantised innovation is carried in the enhancement layer and less in the base layer. The higher datarate for the enhancement layer makes the FIFO buffering more effective at accommodating datarate variability for any given value of N defining the total buffering duration. But it also makes for a noisier decode of the base layer.

[0163] Figure 7b also shows how it is not necessary for the base layer ranges to be symmetric around zero.

[0164] Figure 7c is like Figure 7a in that each base symbol represents two values of quantised innovation, but quantisation is coarser, corresponding to a lower datarate. The point to draw attention to is that for many base layer symbols, the two possible values of quantised innovation have quite different probabilities whilst they were far more similar in the higher datarate Figure 7a.

[0165] Figure 8a shows how the architecture of Figure 1 can be extended to become scalable. Preconditioning operations (not shown) from Figure 1 produce the quantised innovation 121. Instead of directly entropy coding the quantised innovation 121 , it is split 800 into more coarsely quantised base layer symbols 840 each representing several (typically 2fe, a power of two) quantised innovation values. The base layer symbols are entropy coded 820 to produce base data.

[0166] Information denoting which of the several quantised innovation symbols was actually present out of the several possibilities is then an enhancement symbol 860.

[0167] Often the enhancement symbols are reasonably equiprobable, so there is not a large loss in coding efficiency from directly representing which of the 2fepossibilities as a k bit binary word of enhancement data.

[0168] This treatment has similarities with the common lossless coding practice of using rice coding for the innovation signal, which forms a codeword for x by using a Huffman code for [2“fe%] (which combines 2fevalues into one Huffman coded symbol) followed by the verbatim k least significant bits of %. The key difference is that the current invention doesn’t combine these two elements into one composite codeword, instead the least significant bits are separated off as enhancement data. This splitting into two data streams does not increase the data compared to what such a lossless encoder would produce, so this scalablilty introduces no inefficiency.

[0169] The base data and enhancement data could be formed into packets for transmission by the method of Figure 3, but scalable codecs are independently useful with or without the advantages of differential buffering.

[0170] The encoding process is easily reversed in the decoder by entropy decoding 830 the base layer data to base symbols 850 and joining 810 them to the enhancement data 870 to produce quantised innovation 131. This joining operation might be as straightforward as a left shift of the base symbol, shifting in k bits from the enhancement data.

[0171] The decoder can then produce its decoded output from 131 by similar postconditioning stages to those shown in Figure 1 .

[0172] If the genuine enhancement data is not currently available to the decoder, then it could sensibly replace the enhancement datum with an average value 871 for the enhancement data. For example, if enhancement data takes values in {0,1, 2, 3} then it might be sensible to choose 871 as 1 .5 for the missing enhancement data before joining it to the left shifted base symbol.

[0173] Of course, there are alternative arithmetically equivalent possibilities which may be more computationally convenient. For example, when enhancement data is not available, the entropy decoder might directly emit suitable reconstruction values for the output of Figure 8a directly.

[0174] Figure 8b shows an improvement on Figure 8a. In the encoder, the Split operation 800 is enhanced by adding the previous value of the enhancement to the quantised innovation before the split. In the decoder the Join operation is enhanced by subtracting the previous value of the enhancement after the join.

[0175] This noise shapes the quantisation internal to the Split 800 producing the base layer signal 840 making it a closer approximation to the quantised innovation 121 at low frequencies. (Hence, we label the enhanced operations NSSplit 801 and NSJoin 811). The effect of the shaping is most easily seen by considering the decoder - each enhancement symbol affects two output innovation samples with opposite sign. Consequently, enhancement has no effect at DC and so the base layer symbols must exactly represent the quantised innovation samples at DC. For a codec operating over a wide bandwidth, such as 24kHz, this shaping improves the quality of a base layer only decode.

[0176] The noise shaping does however come with a disadvantage: the addition in the encoder slightly increases the entropy of the base layer symbols 840, thus increasing the datarate. Nevertheless, so long as the base layer datarate is sufficiently large, this increase in entropy is minimal and less important than the improvement in base layer only decode. To run through a worked example, suppose the base symbols group 256 values of quantised innovation together.

[0177] If we name the quantised innovation x, then we might define the splitting operation as (b, i.e. the split operation forms the base symbol by shifting x right by 8 and the enhancement by extracting the bottom 8 bits by masking. The corresponding joining operation would be x = 256b + e.

[0178] The noise shaped split is then defined in terms of the z-transform as:

[0179] And the noise shaped join as x = 256b + e — z-1e

[0180] Quantifying the inefficiency from direct coding enhancement

[0181] When the base symbols are coarsely quantised and the base data has a low datarate, we are in the situation illustrated in Figure 7c. The enhancement data are often far from equiprobable and there is inefficiency in coding them verbatim, which increases the datarate.

[0182] Figure 7c shows an N(0,1) distribution quantised with stepsize 0.75. (Innovation is not normally distributed, it has fat tails, but the assumption will suffice for calculating illustrative figures).

[0183] Quantised innovation symbols 720, 721 , 722, 723, 724 and 725 have respective probabilities 0.135, 0.256, 0.285, 0.186, 0.071 and 0.016.

[0184] For each of the 3 base layer symbols covering 720-725, the conditional probability of the left symbol being present is tabulated below along with the binary decision’s entropy:

[0185] So when symbol 724 or 725 is presented, then representing it in 1 bit instead of entropy coding costs (1 - 0.684) = 0.316 bits. Symbol 724 and 725 occur (0.071 + 0.016) = 0.0863 of the time, and so this inefficiency is 0.316 * 0.0863 = 0.027 bits per sample.

[0186] Summing across all the base symbols, the total inefficiency is 0.091 bits per sample.

[0187] This inefficiency can also be calculating comparing the entropy of the quantised innovation (2.495 bits per sample) with the entropy of the base symbols plus the cost of directly coding the enhancement (1.586 + 1 = 2.586 bits per sample). The difference is once again 0.091 bits per sample.

[0188] Tabulating how the inefficiency varies with quantisation stepsize, we see how it is insignificant at stepsizes below around 0.6 but rapidly becomes extremely significant when operating at lower datarates with coarser quantisation. The datarates given are for 2 channel 48kHz.

[0189] The above figures assume a normal distribution for the innovation which is inaccurate (it is thick tailed) but adequate to give a feel for the numbers.

[0190] Incorporating enhancement entropy coding

[0191] Figure 8c illustrates entropy coding of the enhancement 860 to remove this inefficiency. The probability distribution assumed in the enhancement coding is conditional on the corresponding base layer symbol 840, hence the dotted line connecting the base layer symbol 840 to the enhancement entropy coder 821 as context for the coding.

[0192] On the decode side, the encoder’s entropy coder is inverted by an entropy decoder 831 for the enhancement, once again with the dotted line signifying that the required probabilities are specified by the corresponding base layer symbol 850.

[0193] A practical codec may well have a large operating range, perhaps running from 200kbps (at 48kHz stereo) up to lossless encoding of 24 bit audio.

[0194] The base and enhancement layer datarates will both vary greatly over this range.

[0195] Entropy coding of the enhancement is useful at low datarates, but makes an insignificant difference at higher datarates. Also, the computational complexity of entropy coding a binary decision will, for many coding techniques, be cheaper than that of entropy coding a multiway decision.

[0196] It is sensible for enhancement entropy coding to be deployed in some operating modes but not others. It is most appropriate at low datarates where the innovation is coarsely quantised.

[0197] For example, Figure 7c presents a clear case for entropy coding the enhancement data, but perhaps Figure 7a does not, as here the base symbols are more precise and so the enhancement symbols are more equiprobable and so the benefit from enhancement entropy coding smaller. Also, perhaps Figure 7b does not present a clear case where the 4-way entropy coding is more expensive to implement. Moreover, if the enhancement coding inefficiency were considered worth reducing, it could be reduced by grouping two quantised innovation symbols per base symbol instead of four.

[0198] Two dimensional coding

[0199] So far we have considered having one base layer symbol and one enhancement symbol per sample (on each encoded audio channel).

[0200] But the approach is not limited to that, and Figure 9 illustrates a particularly useful configuration where base layer symbols describe two successive quantised innovation samples and there is one binary enhancement datum per base layer symbol.

[0201] Figure 9 shows a 2-dimensional grid. The x axis 900 is an even sample of quantised innovation (121 or 131) and the y axis 901 is the subsequent odd sample of quantised innovation. The grid points are thus the possible values for the pair of samples. The ellipses represent base layer symbols (840 or 850), and each ellipse covers two grid points. Each base layer symbol thus represents two pairs of quantised innovation samples and the corresponding binary enhancement symbol (860 or 870) resolves which of the two pairs of quantised innovation samples was present in the encoder and is to be used for full quality decode.

[0202] Dashed lines 910 and 911 are lines of constant sum for the pairs of quantised innovation samples (for line 910 the pair adds to -1 , for line 911 the pair adds to 2). The 45 degree orientation of the ellipses corresponds to both pairs of quantised innovation samples lying on the same line of constant sum. Consequently even if the decoder is unable to access enhancement, there is no effect at DC and this configuration benefits from the same noise shaping effect as was described with reference to Figure 8b. However, in this case the noise shaping has no inefficiency impact because it only acts within a base layer symbol, not between successive symbols.

[0203] There is also computational benefit from base layer symbols describing a pair of samples, since entropy encoding and decoding can be computationally expensive and the cost of each encode or decode is now spread over two samples rather than one.

[0204] Figure 9 also provides an opportunity to provide a different perspective on how the base layer and enhancement fit together.

[0205] Base layer symbol 920 corresponds to two (930 and 931) pairs of quantised innovation samples (930 is (-4,4) and 931 is (-3,3)). In the encoder a pair of quantised innovation samples with values (-4,4) might split to base symbol 920 and enhancement 0, whilst (-3,3) might map to base symbol 920 and enhancement 1.

[0206] In the decoder base symbol 920 and enhancement 0 would join to a pair of quantised innovation samples 131 with values (-4,4) whilst base symbol 920 and enhancement 1 would join to a pair of quantised innovation samples with values (-3,3). If enhancement were unavailable, then base symbol 920 would produce a pair of quantised innovation samples corresponding to the cross 932 with values (-3.5, 3.5). This could be done by the entropy decoding process directly outputting (-3.5, 3.5) or by substituting a mid-value of 0.5 for the enhancement used in a joining operation.

[0207] Combination of aspects

[0208] Figure 10 shows an example of how the various aspects of the invention can combine into a composite codec design.

[0209] In the encoder of Figure 10a, quantised innovation samples 121 are split 800 into base layer symbols 840 and enhancement symbols 860 with noise shaping (from Figure 8b).

[0210] The base layer symbols are entropy coded 820 to form base layer data. Each block of encoded base layer is treated as an indivisible unit and passed into a delay line 302.

[0211] The enhancement symbols are entropy coded 821 according to probabilities dependent on the corresponding base symbols to furnish enhancement data which is pushed bit by bit into a FIFO buffer 303.

[0212] A packet is filled with an integer number of base layer data blocks 305 taken from the delay line 302 and the remaining space is filled backwards from the end of the packet with data 306 pulled from the FIFO buffer 303.

[0213] The matched decoder is shown in Figure 10b.

[0214] Enhancement data 316 is taken backwards from the end of the packet and pushed into a FIFO buffer 313.

[0215] The boundary between the base layer blocks and enhancement data is not yet known because it is not conveyed by a header field and both have unknown sizes until the base layer blocks have been parsed. Consequently, the whole packet is pushed into the FIFO buffer with the intent of “unpushing” what is subsequently known to be base layer data 315 after it has been fully parsed.

[0216] Each base layer block is then processed in turn, reading data from the start of the packet and entropy decoding 820 the base layer symbols 850 it describes.

[0217] Each base layer symbol defines the probabilities for entropy decoding 821 enhancement symbols 870 from data pulled from the FIFO buffer 313. If valid data is not available from the FIFO (perhaps due to a prior missing packet or skipping ahead in the packet stream), then a representative 871 of an average enhancement symbol is furnished instead.

[0218] Base layer symbols 850 and enhancement symbols 870 are joined and added to the previous enhancement symbol to furnish a replica 131 of the quantised innovation symbols presented to the encoder.

[0219] Of course, Figure 10 is simply an example combination of some embodiments of the different aspects of the invention. Many other combinations are possible and sensible. The invention applies to multi-channel audio, simply by multiplexing the base and enhancement layer symbols or data together and using common FIFO buffers for the enhancement data. Since distinct channels are likely to code with different parameters it is likely more sensible for the multiplexing to switch channels after a whole block of samples rather than switch every sample.

Claims

Claims1. A method for encoding a sequence of input blocks of audio, each input block containing one or more channels of audio samples, the method comprising, for each input block: preconditioning the input block using one or more of: (i) gain adjustment (ii) quantisation, (iii) noise shaping, (iv) filtering, (v) bandsplitting and (vi) matrixing, to furnish a preconditioned block containing preconditioned samples; encoding the preconditioned block to furnish base data and enhancement data; placing the base data from an encoded block into an output stream; pushing the enhancement data into a first-in-first-out (FIFO) buffer, pulling a quantity of previously-pushed enhancement data from the FIFO buffer; and, placing the pulled enhancement data into the output stream, wherein the preconditioned samples can be approximately reconstructed from the base data and more accurately reconstructed from the base data and the enhancement data together.

2. A method according to claim 1 , wherein the output stream is divided into packets, the wherein the base data from an integer number of preconditioned blocks is placed contiguously into a packet and the quantity of enhancement data to be pulled and placed is determined in dependence on the space remaining in the packet after the base data has been placed.

3. A method according to claim 2, wherein the enhancement data is placed after the base data in the packet and wherein earlier bytes of pulled enhancement data are placed later in the packet than subsequent bytes of pulled enhancement data.

4. A method according to claim 2, wherein the base data is placed after the enhancement data in the packet and wherein base data furnished from preconditioned samples earlier in the preconditioned block is placed later in thepacket than base data furnished from preconditioned samples later in the preconditioned block.

5. A method according to any of claims 2 to 4 wherein the packet contains a header field specifying how many total bits of enhancement data have been transmitted in earlier packets from both the first preconditioned block whose base layer data is placed in the packet and subsequent blocks.

6. A method according to claim 5, wherein a second packet from the packets does not contain a header field specifying this information.

7. A method according to any preceding claim, wherein the step of encoding the preconditioned block comprises: splitting the preconditioned samples into base symbols and enhancement symbols, entropy coding the base symbols to furnish the base data; and, deriving the enhancement data in dependence on the enhancement symbols.

8. A method according to claim 7, wherein the step of deriving the enhancement data comprises entropy coding the enhancement symbols conditionally on the base symbols.

9. A method according to claim 8, wherein the step of entropy coding the enhancement symbols is performed only on blocks where for any value of base symbol there are only two possible values of enhancement symbol.

10. A method according to any of claims 7 to 9, wherein the step of splitting incorporates noise shaping.

11. A method according to any of claims 7 to 10, wherein the step of splitting maps pairs of successive preconditioned samples to base symbols and maps two pairs of successive preconditioned samples to each base symbol, wherein the twopairs of successive preconditioned samples mapped to a base symbol sum to the same value.

12. An encoder adapted to encode input blocks of audio to an output stream using the method of any of claims 1 to 11 .

13. A computer readable medium comprising instructions that, when executed by one or more processors, cause said one or more processors to perform the method of any of claims 1 to 11 .

14. A method for decoding packets of data to output blocks of audio each containing one or more channels of output audio samples, the method comprising: receiving a packet of data; retrieving a portion of base data from the packet; retrieving first enhancement data from the packet and pushing the first enhancement data into a first-in-first-out (FIFO) buffer; pulling second enhancement data from the FIFO buffer; decoding the portion of base data and the second enhancement data to furnish a block of preconditioned samples; and postconditioning the block of preconditioned samples using one or more of (i) gain adjustment, (ii) filtering, (iii) bandjoining and (iv) matrixing, to furnish the output blocks of audio.

15. A method according to claim 14, wherein the step of decoding comprises: entropy decoding the base data to furnish base symbols; and entropy decoding the second enhancement data in dependence on the base symbols to furnish the preconditioned samples.

16. A method according to claim 14 or claim 15, wherein the step of decoding comprises: entropy decoding the base data to furnish base symbols; entropy decoding the second enhancement data to furnish enhancement symbols in dependence on the base symbols; andcombining the base symbols with the enhancement symbols to furnish the preconditioned samples.

17. A method according to any of claims 14 to 16, wherein base data precedes first enhancement data in the packet of data and wherein first enhancement data towards the end of the packet is pushed into the FIFO buffer before first enhancement data towards the beginning of the packet.

18. A method according to claim 15 or claim 16, wherein first enhancement data precedes base data in the packet and wherein base data towards the end of the packet is entropy decoded before base data towards the beginning of the packet.

19. A method according to any of claims 14 to 18, wherein the method comprises the additional steps of: reading a header field from the packet; and adjusting the amount of data in the FIFO buffer to match the number of bits specified in the header field.

20. A method according to any of claims 15 to 19, wherein a pair of successive preconditioned samples are furnished from each base symbol, said pair being selected in dependence on the second enhancement data from two possible pairs of preconditioned samples that map to the base symbol, wherein the two possible pairs of successive preconditioned samples each sum to the same value.

21. A decoder adapted to decode packets of data to output blocks of audio using the method of any of claims 14 to 20.

22. A computer readable medium comprising instructions that, when executed by one or more processors, cause said one or more processors to perform the method of any of claims 14 to 20.

23. A codec comprising an encoder according to claim 12 in combination with a decoder according to claim 21 .

24. An encoder for an audio signal, the encoder comprising: an input configured to receive the audio signal; an output for furnishing a compressed stream of data, the compressed stream comprising base data and enhancement data, a splitter configured to separate the audio signal into base symbols and enhancement symbols, a first entropy coder configured to receive the base symbols and to furnish the base data; a second entropy coder configured to receive the enhancement symbols and to furnish the enhancement data; and a combiner configured to receive the base data and the enhancement data and to furnish the compressed stream, wherein the combiner comprises a variable delay unit whereby the enhancement data within the compressed stream has a varying time relationship to the base data.

25. An encoder according to claim 24, wherein the second entropy coder is configured to furnish the enhancement data in dependence on the base symbols.

26. An encoder according to claim 24 or claim 25, wherein the compressed stream comprises packets of data.

27. An encoder according to any of claims 24 to 26, wherein the combiner comprises a fixed delay unit to receive the base data and a first-in-first-out (FIFO) buffer to receive the enhancement data.

28. A decoder for a stream of packets of data representing an audio signal, the packets each containing base data plus enhancement data, the enhancement data having a varying time relationship relative to the base data, the decoder comprising: an input configured to receive the stream of packets of data;a memory coupled to the input, a first entropy decoder coupled to the input and configured to receive base data and to furnish base symbols; a second entropy decoder coupled to the memory and configured to receive enhancement data and to furnish enhancement symbols; and a joiner configured to receive the base symbols and the enhancement symbols and to furnish decoded audio samples to an output of the decoder.

29. A decoder according to claim 28, wherein the memory is configured to store recently-received packets of data.

30. A decoder according to claim 28, wherein the memory is configured as a first-in-first-out (FIFO) buffer for enhancement data. 31 . A decoder according to any of claims 28 to 30, wherein the second entropy decoder is configured to decode enhancement data in dependence on base symbols received from the first entropy decoder.

Images