Method of decoding a video signal

Abstract

A method of decoding a video signal comprising: obtaining an encoded frame from the video signal, the encoded frame comprising a base encoding and an enhancement encoding; determining whether to refresh a temporal buffer; and if it is determined that the temporal buffer is to be refreshed: reconstructing a decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer; and writing a set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame.

Description

[0001] METHOD OF DECODING A VIDEO SIGNAL

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to methods for use in video coding technology.

[0004] BACKGROUND

[0005] In an enhancement-type coding algorithm, such as MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC), one or more layers of residual data can be used to improve the performance of a base coding algorithm.

[0006] At an enhancement-type encoder, the residual data is calculated based on a comparison of a base decoded video signal and an original input video signal. In general the base decoded video signal is the result of encoding and then decoding the video signal, according to a base codec. Each image element (such as a pixel) of the original input video signal can have a value that is different from the corresponding image element of the base decoded video signal. In general, the differences can comprise a mixture of higher values (positive values of the residual data) for some image elements, lower values (negative values of the residual data) for other image elements and zero values.

[0007] The encoder produces a video signal comprising a base encoded video signal and enhancement encoded video signal. The base encoded video signal comprises the original input video signal encoded by the base codec. The enhancement encoded video signal comprises the residual data.

[0008] At an enhancement-type decoder, the residual data is used to recover the original input video signal based on the base decoded video signal. This typically involves adding the positive values of the residual data to the base decoded video signal and subtracting the negative values of the residual data from the base decoded video signal.

[0009] Encoding and decoding may be used to compress and / or secure content communicated over a network, such as in a streaming service. Alternatively, encoding and decoding may be used in other data transportation / transmission contexts such as physical media (e.g. DVDs, portable flash memory).

[0010] The encoder may for example be implemented at a content creation, content distribution or content streaming service.

[0011] The decoder may for example be implemented in consumer hardware for viewing decoded content, such as in a display device (e.g. a television), or in a separate device for receiving encoded content and supplying decoded content to the display device (e.g. a set-top box or a DVD player).

[0012] In WO 2020 / 188272, which is incorporated herein by reference, the residual data produced by the encoder and used by the decoder is reduced. This is achieved by using temporal prediction. In temporal prediction, a temporal buffer at the decoder stores residual data associated with a preceding frame of the video signal. In order to obtain a subsequent frame of the video signal, the decoder reads the residual data from the temporal buffer, and recovers the subsequent frame of the input video signal by combining subsequent frame data of the base decoded video signal with the residual data stored in the buffer and any subsequent frame data of the enhancement encoded video signal.

[0013] When temporal prediction is used, the subsequent frame data of the enhancement encoded video signal can be residual data calculated based on a comparison of an original input video signal against a sum of a base decoded video signal and the residual data stored in the temporal buffer. In other words, the subsequent frame data of the enhancement encoded video signal may comprise adjustments to the residual data stored in the temporal buffer so that the residual data becomes applicable to the subsequent frame.

[0014] This has the advantage that, when the residual data of the subsequent frame is identical or similar to the residual data of the preceding frame, a reduced amount of residual data needs to be included in the enhancement encoded video signal.

[0015] In WO 2020 / 188272, the encoder can control usage of temporal prediction at the decoder. This control may be achieved with parameters in the video signal, including a “temporal_enabled” parameter (specifying whether temporal prediction should be used when decoding a frame) and a “temporal refresh bit” parameter (specifying whether the temporal buffer should be refreshed). Refreshing the temporal buffer includes setting the residual data of the temporal buffer to zero values.

[0016] In general it is desirable both to reduce the size of the encoded video signal and reduce the processing resources required to encode and decode the video signal. In this application, it is specifically contemplated how to perform temporal prediction with reduced processing resources without increasing the size of the encoded video signal.

[0017] SUMMARY OF INVENTION

[0018] According to a first aspect, there is described a method of decoding a video signal comprising: obtaining an encoded frame from the video signal, the encoded frame comprising a base encoding and an enhancement encoding; determining whether to refresh a temporal buffer; and if it is determined that the temporal buffer is to be refreshed: reconstructing a decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer; and writing a set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame.

[0019] The base encoding and enhancement encoding of the encoded frame may be stored and / or transmitted separately. For example, in a video signal, the base encoding of the encoded frame may form part of a base encoded video signal, and the enhancement encoding of the encoded frame may form part of an enhancement encoded video signal. This allows for greater flexibility between the base encoded video signal and enhancement encoded video signal. For example, the enhancement encoded video signal may have a higher frame rate than the base encoded video signal, and the base encoding may be reused for multiple encoded frames. Similarly, the base encoded video signal may have a higher frame rate than the enhancement encoded video signal, and the enhancement encoding may be reused for multiple encoded frames. Alternatively, the base encoding and the enhancement encoding of the encoded frame may be stored and / or transmitted together. For example, the base encoding and enhancement encoding each form part of a single combined video signal.

[0020] According to the claimed features, a temporal refresh operation is merged with temporal prediction of a frame. This eliminates a previously required operation of writing zero values to the temporal buffer during the temporal refresh operation and reading those same zero values from the temporal buffer in the next temporal prediction operation. This reduces the processing resources required when decoding a video signal that supports temporal prediction.

[0021] The temporal predictions may be residuals. Specifically, temporal predictions may be residuals generated or intended for one frame, that are at least partially reused in the process of decoding a subsequent frame.

[0022] Writing the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, may specifically comprise: obtaining a set of residuals from the enhancement encoding of the encoded frame, wherein the residuals are differences between: a base decoded frame obtained by decoding the base encoding; and an input frame, wherein the base encoding is generated (i.e. was generated by an encoder) by encoding the input frame; and writing the set of residuals to the temporal buffer. The residuals may then be read from the temporal buffer when decoding a subsequent frame. In other words, the input frame is a frame that was previously encoded by an encoder to generate the encoded frame. In other words,

[0023] In some embodiments, each frame of the video signal has a frame area divided into a plurality of tiles and / or a plurality of blocks, and the set of residuals includes residual values for a subset of the plurality of tiles or a subset of the plurality of blocks, wherein writing the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, further comprises: writing a zero value to the temporal buffer for a tile of the frame area which is not in the subset of the plurality of tiles and / or for a block of the frame area which is not in the subset of the plurality of blocks. In other words, a zero value may be written to a tile or a block for which no (non-zero) residual value was included in the enhancement encoding of the encoded frame.

[0024] Herein a “block” is a coding unit for encoding and decoding. For example, the size of a block may depend upon a directional decomposition transform used in the encoding and decoding.

[0025] Herein a “tile” is a group of blocks that cover a region of a frame. Temporal prediction may be applied for a whole frame or for individual tiles of a frame. A tile size parameter may be included as overhead in the video signal. Increased tile size can reduce overhead for controlling temporal prediction on a per-tile basis, while reduced tile size can increase the flexibility of temporal prediction.

[0026] Additionally, a frame may be partitioned into a plurality of “planes”, and each plane maybe partitioned into tiles and / or blocks. Planes may for example be colour channels which combine to give a multicolour image.

[0027] In tile-divided and / or block-divided implementations, the method may comprise: constructing a complete set of new temporal predictions for all tiles and / or all blocks of the frame area, consisting of the set of residuals obtained from the enhancement encoding of the encoded frame and zero values for any tiles and / or blocks for which no residual values are included in the enhancement encoding of the encoded frame; and writing the complete set of new temporal predictions to the temporal buffer. More specifically, residual values for a complete frame area may be written to the temporal buffer in a single step.

[0028] In some embodiments, the method further comprises: if it is determined that the temporal buffer is not to be refreshed: reading the temporal buffer to obtain a set of previous temporal predictions; and reconstructing the decoded frame based on the base encoding and enhancement encoding of the encoded frame and based on the set of previous temporal predictions. The previous temporal predictions may comprise a set of residuals for an earlier frame in the video signal preceding the encoded frame. Additionally, the enhancement encoding of the encoded frame may comprise adjustments to the set of previous temporal predictions such that it is applicable to the base encoding in order to reconstruct the decoded frame.

[0029] In some embodiments, the method further comprises: receiving a refresh parameter in the video signal, wherein the determining whether to refresh the temporal buffer is based on the refresh parameter. The refresh parameter may be part of the encoded frame. Alternatively, the refresh parameter may be part of a control component of the video signal, separate from the encoded frame. For example, the refresh parameter may be included in the video signal in a similar way to the “temporal_enabled” parameter of WO 2020 / 188272.

[0030] According to a second aspect, the following disclosure provides a decoder for decoding a video signal, the decoder comprising: a memory comprising a temporal buffer configured to store a set of temporal predictions; and a processor configured to: obtain an encoded frame from the video signal, the encoded frame comprising a base encoding and an enhancement encoding; determine whether to refresh a temporal buffer; and if it is determined that the temporal buffer is to be refreshed: reconstruct a decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer; and write a set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame. In other words, the decoder is configured to perform a method according to the first aspect.

[0031] The processor may be configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by: obtaining a set of residuals from the enhancement encoding of the encoded frame, wherein the residuals are differences between: a base decoded frame obtained by decoding the base encoding; and an input frame, wherein the base encoding is generated (i.e. was generated by an encoder) by encoding the input frame; and writing the set of residuals to the temporal buffer.

[0032] In some embodiments, each frame of the video signal has a frame area divided into a plurality of tiles and / or a plurality of blocks, and the set of residuals includes residual values for a subset of the plurality of tiles or a subset of the plurality of blocks, wherein the processor is configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by further: writing a zero value to the temporal buffer for a tile of the frame area which is not in the subset of the plurality of tiles and / or a block of the frame area which is not in the subset of the plurality of tiles.

[0033] Optionally, the processor is configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by: constructing a complete set of new temporal predictions for all tiles and / or all blocks of the frame area, consisting of the set of residuals obtained from the enhancement encoding of the encoded frame and zero values for any tiles and / or blocks for which no residual values are included in the enhancement encoding of the encoded frame; and writing the complete set of new temporal predictions to the temporal buffer.

[0034] The processor may be further configured to: it is determined that the temporal buffer is not to be refreshed: read the temporal buffer to obtain a set of previous temporal predictions; and reconstruct the decoded frame based on the base encoding and enhancement encoding of the encoded frame and based on the set of previous temporal predictions. Optionally, the previous temporal predictions comprise a set of residuals for an earlier frame in the video signal preceding the encoded frame.

[0035] The processor may be configured to: receive a refresh parameter in the video signal; and determine whether to refresh the temporal buffer based on the refresh parameter. The refresh parameter may be part of the encoded frame. The refresh parameter may alternatively be part of a control component of the video signal, separate from the encoded frame.

[0036] According to a third aspect, a decoder is hard-coded to perform a method according to the first aspect, or configured to perform a method according to the first aspect using a mixture of one or more processors executing computer program instructions to perform one part of the method and application-specific hardware configured to perform another part of the method. According to a fourth aspect, there is provided a computer program comprising instructions which, when executed by one or more processors, cause the processors to perform a method according to the first aspect.

[0037] According to a fifth aspect, there is provided a computer-readable storage medium storing instructions which, when executed by one or more processors, cause the processors to perform a method according to the first aspect.

[0038] According to a sixth aspect, there is provided a chipset configured to perform a method according to the first aspect. For example, the chipset may comprise at least one ASIC adapted to perform all or part of the method according to the first aspect. The chipset may further comprise a memory storing instructions which, when executed by one or more processors, cause the processors to perform a further part of a method according to the first aspect.

[0039] According to a seventh aspect, there is provided a set-top box comprising a decoder according to the second aspect or the third aspect.

[0040] According to an eighth aspect, there is provided a method of retrofitting a decoder, chipset or set-top box to provide a decoder, chipset or set-top box configured to perform a method according to the first aspect. For example, after a customer purchases a device, the device may be retrofitted by means of the manufacturer providing an update for the device, wherein performing the update constitutes making a device having the features of one of the preceding aspects.

[0041] BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a schematic diagram illustrating a background example of an encoding process using temporal prediction;

[0043] Figure 2 is a schematic diagram illustrating a background example of a decoding process using temporal prediction;

[0044] Figures 3A and 3B are schematic diagrams each illustrating a background example of a further encoding process; Figures 4A and 4B are schematic diagrams each illustrating a background example of a further decoding process;

[0045] Fig. 5 is a flow chart schematically illustrating a background example method of refreshing a temporal buffer at a decoder;

[0046] Fig. 6 is a flow chart schematically illustrating a method of refreshing a temporal buffer at a decoder, according to the invention.

[0047] DETAILED DESCRIPTION

[0048] Described herein is a hybrid backward-compatible coding technology.

[0049] The examples described herein provide a flexible, adaptable, highly efficient and computationally inexpensive coding format which combines a different video coding format, a base codec, (e.g. AVC, HEVC, or any other present or future codec) with at least two enhancement levels of coded data.

[0050] The general structure of the encoding scheme uses a down-sampled source signal encoded with a base codec, and adds one or more levels of correction data to the decoded output of the base codec to generate a corrected picture.

[0051] In some cases, the base encoding is decodable by a hardware decoder while the enhancement encoding is suitable for software processing.

[0052] Although the decoded output of the base codec is not intended for viewing, it is a fully decoded video at a lower resolution, making the output compatible with existing decoders and, where considered suitable, also usable as a lower resolution output.

[0053] Figure 1 shows a first example encoder 100. The illustrated components may also be implemented as steps of a corresponding encoding process.

[0054] In the encoder 100, an input full resolution video 102 is processed to generate various encodings. A first encoding (base encoding 110) is produced by feeding a base encoder 106 (e.g., AVC, HEVC, or any other codec) with a down-sampled version of the input video, which is produced by down-sampling 104 the input video 102. A second encoding (level 1 encoding 116, an example of an enhancement encoding) is produced by applying an encoding operation 114 to the residuals obtained by taking the difference 112 between the reconstructed base codec video and the down-sampled version of the input video. The reconstructed base codec video is obtained by decoding the output of the base encoder 106 with a base decoder 108. A third encoding (level 2 encoding 128, another example of an enhancement encoding) is produced by processing 126 the residuals obtained by taking the difference 124 between an up-sampled version of a corrected version of the reconstructed base coded video and the input video 102. The corrected version of the reconstructed base codec video is obtained by combining 120 the reconstructed base codec video and the residuals obtained by applying a decoding operation 118 to the level 1 encoding 116.

[0055] The level 1 encoding operation 114 operates with an optional level 1 temporal buffer 130, which may be used to apply temporal processing as described further below. The level 2 encoding operation 126 also operates with an optional level 2 temporal buffer 132, which may be used to apply temporal processing as described further below. The level 1 temporal buffer 130 and the level 2 temporal buffer 132 may operate under the control of a temporal selection component 134. The temporal selection component 134 may receive one or more of the input video 102 and the output of the down-sampling 104 to select a temporal mode. This is explained in more detail in later examples.

[0056] Figure 2 shows a first example decoder 200. The illustrated components may also be implemented as steps of a corresponding decoding process. The decoder receives the three streams (a base encoding 210, a level 1 encoding 216 and a level 2 encoding 228) generated by an encoder such as the encoder 100 of Figure 1 together with headers 236 containing further decoding information. The base encoding 210 is decoded by a base decoder 208 corresponding to the base decoder used in the encoder, and its output is combined 238 with the decoded residuals obtained by decoding 240 the level 1 encoding 216. The combined video is up-sampled 242 and further combined 244 with the decoded residuals obtained by applying a decoding operation 246 to the level 2 encoding 228.

[0057] Figures 3A and 3B show different variations of a second example encoder 300, 380. The second example encoder 300, 380 may comprise an implementation of the first example encoder 100 of Figure 1. In the examples of Figures 3A and 3B, the encoding steps are expanded in more detail to provide an example of how the steps may be performed. Figure 3A illustrates a first variation with temporal prediction provided only in the second level of the enhancement process, i.e. with respect to the level 2 encoding. Figure 3B illustrates a second variation with temporal prediction performed in both levels of enhancement (i.e. levels 1 and 2).

[0058] The base encoding 310 is substantially created by a process as explained with reference to Figure 1. That is, an input video 302 is down-sampled 304 (i.e. a down-sampling operation 304 is applied to the input video 302 to generate a down- sampled input video). The down-sampled video obtained by down-sampling 304 the input video 302 is then encoded using a first base encoder 306 (i.e. an encoding operation is applied to the down-sampled input video to generate the base encoding 310 using a first or base encoder 306). Preferably the first or base encoder 306 is a codec suitable for hardware decoding. The base encoding 310 may be referred to as the base layer or base level.

[0059] As noted above, the enhancement encoded video signal may comprise two or more enhancement encodings. A first level of enhancement (described herein as “level 1”) provides for a set of correction data which can be combined with a decoded version of the base encoding to generate a corrected picture. This first enhancement encoding is illustrated in Figures 1 and 3 as the Level 1 encoding 316. The enhancement encoding may be generated by an enhancement encoder. The enhancement encoder may be different to the base encoder 306 used to generate the base encoding 310.

[0060] To generate the Level 1 encoding 316, the base encoding 310 is decoded using a base decoder 308 (i.e. a decoding operation is applied to the base encoding 310 to generate a decoded base). The difference 312 between the decoded base and the down-sampled input video obtained by down-sampling 304 the input video 302 is then created (i.e. a subtraction operation 312 is applied to the down-sampled input video and the decoded base to generate a first set of residuals). Here the term residuals is used in the same manner as that known in the art, that is, the error between a reference frame and a desired frame. Here the reference frame is the decoded base and the desired frame is the down-sampled input video. Thus the residuals used in the first enhancement level can be considered as a corrected video as they ‘correct’ the decoded base to the down-sampled input video that was used in the base encoding operation.

[0061] The difference 312 is then encoded to generate the Level 1 encoding 316 (i.e. an encoding operation is applied to the first set of residuals to generate a first enhancement encoding 316).

[0062] In the example implementation of Figures 3A and 3B, the encoding operation comprises several steps, each of which is optional and preferred and provides particular benefits.

[0063] In Figure 3, the steps include a transform step 336, a quantization step 338 and an entropy encoding step 340.

[0064] Although not shown in the Figures, in some examples, the encoding process identifies if the residuals ranking mode is selected. If residuals mode is selected the residuals ranking step may be performed (i.e. a residuals ranking operation may be performed on the first step of residuals to generate a ranked set of residuals). The ranked set of residuals may be filtered so that not all residuals are encoded into the first enhancement encoding 316 (or correction stream).

[0065] The first set of residuals, or the ranked or filtered first set of residuals are then transformed 336, quantized 338 and entropy encoded 340 to produce the Level 1 encoding 316 (i.e. a transform operation 336 is applied to the first set of residuals or the filtered first set of residuals depending on whether or not ranking mode is selected to generate a transformed set of residuals; a quantization operation 338 is applied to the transformed set of residuals to generate a set of quantized residuals; and, an entropy encoding operation 340 is applied to the quantized set of residuals to generate the first level of enhancement encoding 316). Preferably, the entropy encoding operation 340 may be a Huffman encoding operation or a run-length encoding operation or both. Optionally a control operation (not shown in the Figures) may be applied to the quantized set of residuals so as to correct for the effects of the ranking operation.

[0066] As noted above, the enhancement encoded video signal may comprise a first level of enhancement 316 and a second level of enhancement 328. The first level of enhancement 316 may be considered to be a corrected stream. The second level of enhancement 328 may be considered to be a further level of enhancement that converts the corrected stream to the original input video.

[0067] The further level of enhancement 328 is created by encoding a further set of residuals which are the difference 324 between an up-sampled version of a decoded level 1 encoding and the input video 302.

[0068] In Figure 3, the quantized (or controlled) set of residuals are inversely quantized 342 and inversely transformed 344 before a de-blocking filter (not shown in the Figures) is optionally applied to generate a decoded first set of residuals (i.e. an inverse quantization operation 342 is applied to the quantized first set of residuals to generate a de-quantized first set of residuals; an inverse transform operation 344 is applied to the de-quantized first set of residuals to generate a detransformed first set of residuals; and, a de-blocking filter operation is optionally applied to the de-transformed first set of residuals to generate a decoded first set of residuals). The de-blocking filter step is optional depending on the transform 336 applied and comprises applying a weighted mask to each block of the detransformed 344 first set of residuals.

[0069] The decoded base is combined 320 with the decoded first set of residuals (i.e. a summing operation 320 is performed on the decoded base and the decoded first set of residuals). As illustrated in Figures 3A and 3B, that combination is then up- sampled 322. A difference operation 324 is applied to generate a further set of residuals by comparison to the input video 302. The further set of residuals are then encoded as the Level 2 encoding 328. As with the Level 1 encoding 316, the encoding applied to the level 2 residuals may comprise several steps. Figure 3A illustrates the steps as temporal prediction, transform 348, quantization 350 and entropy encoding 352.

[0070] Although not shown in the Figures, in some examples, the encoding process identifies if the residuals ranking mode is selected. If residuals mode is selected the residuals ranking step may be performed (i.e. a residuals ranking operation may be performed on the further set of residuals to generate a further ranked set of residuals). The further ranked set of residuals may be filtered so that not all residuals are encoded into the Level 2 encoding 328.

[0071] The further set of residuals or the further ranked set of residuals are subsequently transformed 348 (i.e. a transform operation 348 is performed on the further ranked set of residuals to generate a further transformed set of residuals). As illustrated, the transform operation 348 may utilise a predicted coefficient or predicted average derived prior to up-sampling 322.

[0072] Figure 3A shows a variation of the second example encoder 300 where temporal prediction is performed as part of the level 2 encoding process. Temporal prediction is performed using the temporal selection component 334 and the level 2 temporal buffer 332. The temporal selection component 334 may determine a temporal processing mode as described in more detail below and control the use of the level 2 temporal buffer 332 accordingly. For example, if no temporal processing is to be performed the temporal selection component 334 may indicate that the contents of the level 2 temporal buffer 332 are to be set to 0. Figure 3B shows a variation of the second example encoder 380 where temporal prediction is performed as part of both the level 1 and the level 2 encoding process. In Figure 3B, a level 1 temporal buffer 330 is provided in addition to the level 2 temporal buffer 332. Although not shown, further variations where temporal processing is performed at level 1 but not level 2 are also possible. When temporal prediction is selected, the second example encoder 300, 380 of Figures 3Aor 3B may further modify the coefficients (i.e. the transformed residuals output by a transform component) by subtracting a corresponding set of coefficients derived from an appropriate temporal buffer. The corresponding set of coefficients may comprise a set of coefficients for a same spatial area (e.g. a same coding unit as located within a frame) that are derived from a previous frame (e.g. coefficients for the same area for a previous frame). These coefficients may be derived or otherwise obtained from a temporal buffer. Coefficients obtained from a temporal buffer may be referred to herein as temporal coefficients. The subtraction may be applied by a subtraction component such as the third subtraction components 354 and 356 (for respective levels 2 and 1 ). This temporal prediction step will be further described with respect to later examples. In summary, when temporal prediction is applied, the encoded coefficients correspond to a difference between the frame and another frame of the video signal. The other frame may be an earlier or later frame (or block in the frame). Temporal prediction may be applied selectively for groups of coding units (referred to herein as “tiles”) based on control information and the application of temporal prediction at a decoder may be applied by sending additional control information along with the encodings (e.g. within headers).

[0073] As shown in Figures 3A and 3B, when temporal prediction is active, each transformed coefficient may be: F current Ffr ffer where the temporal buffer may store data associated with a previous frame. Temporal prediction may be performed for one colour plane or for multiple colour planes. In general, the subtraction may be applied as an element wise subtraction for a “frame” of video where the elements of the frame represent transformed coefficients, where the transform is applied with respect to a particular n by n coding unit size (e.g. 2x2 or 4x4). The difference that results from the temporal prediction (e.g. the delta above) may be stored in the buffer for use for a subsequent frame. Hence, in effect, the residual that results to the temporal prediction is a coefficient residual with respect to the buffer. Although Figures 3A and 3B show temporal prediction being performed after the transform operation, it may also be performed after the quantize operation. This may avoid the need to apply the level 2 inverse quantization component 358 and / or the level 1 inverse quantize component 360. Thus, as illustrated in Figures 3A and 3B and described above, the output of the second example encoders 300, 380 after performing an encoding process is a base encoding 310 and one or more enhancement encodings which preferably comprise an level 1 encoding 316 for a first level of enhancement and an level 2 encoding 328 for a further or second level of enhancement.

[0074] Figures 4A and 4B illustrate respective variations of a second example decoder 400, 480. The variations of the second example decoder 400, 480 may be respectively implemented to correspond to the first example decoder 200 of Figure 2. As is clearly identifiable, the decoding steps and components are expanded in more detail to provide an example of how the decoding may be performed. As with Figures 3A and 3B, Figure 4A illustrates a variation where temporal prediction is used only for the second level (i.e. level 2) and Figure 4B illustrates a variation in which temporal prediction is used in both levels (i.e. levels 1 and 2). As before, further variations are envisaged (e.g. level 1 but not level 2), where the form of the configuration may be controlled using signalling information.

[0075] As shown in the example of Figure 4B, in the decoding process, the decoder 480 may parse the headers 436 (e.g. containing global configuration data, picture configuration data, and other data blocks) and configure the decoder based on those headers 436. In order to re-create the input video, the decoder 400, 480 may decode each of the base encoding 410, the first enhancement encoding 416 and the further enhancement encoding 428. The frames of the video signal may be synchronised and then combined to derive the decoded video 448.

[0076] In each decoding process, the enhancement encodings may undergo the steps of entropy decoding 450, 452, inverse quantization 454, 456 and inverse transform 458, 460 to re-create a set of residuals. The decoding processes of Figures 4A and 4B comprise retrieving an array of entropy decoded quantized coefficients representing a first level of enhancement and outputting an array of L-1 residuals. The entropy decoded quantized coefficients in this case are obtained by applying the entropy decoding 450 operation to the L-1 encoding 416. The decoding processes of Figures 4A and 4B further comprise retrieving an array of samples of output of a base decoder 408. The decoding processes of Figures 4A and 4B further comprise applying a dequantization process 454 to the array of entropy decoded quantized coefficients to derive a set of de-quantized coefficients, applying a transformation process 458 to the set of de-quantized coefficients and optionally applying a filter process (Not shown in Figures 4A and 4B) to output the array of L-1 residuals representing a first level of enhancement, which may be referred to as a preliminary set of residuals. In this case, the de-quantization process 454 is applied to entropy decoded quantized coefficients for respective blocks of a frame of the level 1 encoding 416, and the transformation process 458 (which may be referred to as an inverse transform operation) is applied to the output of the de quantization process 454 for the respective blocks of the frame. The decoding processes of Figures 4A and 4B then further comprise recreating a picture by combining 462 the array of L-1 residuals with the array of samples of output of the base decoder 408. The decoding processes of Figures 4Aand 4B comprise applying a transform process 458 from a set of predetermined transform processes according to a signalled parameter. For example, the transform process 458 may be applied on a 2x2 coding unit or a 4x4 coding unit. A coding unit may be referred to herein as a block of elements in an array, in this case the array of L-1 residuals.

[0077] The decoding processes of Figures 4A and 4B comprise retrieving an array of entropy decoded quantized coefficients representing a further level of enhancement and outputting an array of residuals. In the decoding processes shown in Figures 4A and 4B, the further level of enhancement is a second level of enhancement and the array of residuals output is an array of L-2 residuals. The method of Figures 4A and 4B further comprises retrieving the array of L-1 residuals of the first level of enhancement corresponding to the array of entropy decoded quantized coefficients representing a further level of enhancement. The method of Figures 4A and 4B further comprises applying an up-sampling process 464 to the array of residuals of the first level of enhancement. In Figures 4A and 4B, the up-sampling process 464 is applied to the combination of the array of L-1 residuals of the first level of enhancement and the corresponding array of samples of output of the base decoder 408.

[0078] In Figures 4A and 4B, the up-sampling process 464 is a modified up-sampling process, in which a modifier is added to a residual. The step of adding a modifier may be performed as part of the transform process 460. Alternatively, since the transform process 460 involves a linear transformation, the step of adding a modifier may be performed as part of the modified up-sampling process 464, as shown in Figures 4A and 4B. The step of adding a modifier therefore results in a modification of a residual. The modification may be performed based on a location of the residual in a frame. The modification may be a predetermined value.

[0079] In Figure 4A, temporal prediction is applied during the level 2 decoding. In the example of Figure 4A, the temporal prediction is controlled by a temporal prediction component 466. In this variation, control information for the temporal prediction is extracted from the level 2 encoding 428 to the temporal prediction component 466, as indicated by the arrow. In other implementations, such as those shown in Figure 4B, control information for the temporal prediction may be sent separately from the level 2 encoding 428, e.g. in the headers 436. The temporal prediction component 466 controls the use of the level 2 temporal buffer 432, e.g. it may determine a temporal mode and control temporal refresh as described with reference to later examples. The contents of the temporal buffer 432 may be updated based on data for a previous frame of residuals. When the temporal buffer 432 is applied, the contents of the buffer are added 468 to the second set of residuals. In Figure 4A, the contents of the temporal buffer 432 are added 468 to the output of a level 2 decoding component 446 (which in Figure 4A implements the entropy decoding 452, the inverse quantization 456 and the inverse transform 460). In other examples, the contents of the temporal buffer may represent any set of intermediate decoding data and as such the addition 468 may be moved appropriately to apply the contents of the temporal buffer at an appropriate stage (e.g. if the temporal buffer is applied at the dequantized coefficient stage, the addition 468 may be located before the inverse transform 460). The temporal-corrected second set of residuals are then combined 470 with the output of the up-sampling 464 to generate the decoded video 448. The decoded video 448 is at a level 2 spatial resolution, which may be higher than a level 1 spatial resolution. The second set of residuals apply a correction to the (viewable) upsampled reconstructed video, where the correction adds back in fine detail and improves the sharpness of lines and features.

[0080] The transform processes 458, 460 may be selected from a set of predetermined transform processes according to a signalled parameter. For example, the transform process 460 may be applied on a 2x2 block of elements in the array of L-2 residuals or a 4x4block of elements in the array of L-2 residuals.

[0081] Figure 4B shows a variation of the second example decoder 480. In this case, temporal prediction control data is received by a temporal prediction component 466 from headers 436. The temporal prediction component 466 controls both the level 1 and level 2 temporal prediction, but in other examples separate control components may be provided for both levels if desired. Figure 4B shows how the reconstructed second set of residuals that are added 468 to the output of the level 2 decoding component 446 may be fed back 469 to be stored in the level 2 temporal buffer 432 for a next frame (the feedback is omitted from Figure 4A for clarity). A level 1 temporal buffer 430 is also shown that operates in a similar manner to the level 2 temporal buffer 432 described above and the feedback loop for the buffer is shown in this Figure. The contents of the level 1 temporal buffer 430 are added into the level 1 residual processing pipeline via a summation 472 and the sum of the level 1 residuals is fed back 473 to the level 1 temporal buffer 430. Again, the position of this summation 472 may vary along the level 1 residual processing pipeline depending on where the temporal prediction is applied (e.g. if it is applied in transformed coefficient space, it may be located before the level 1 inverse transform component 458).

[0082] Figure 4B shows two ways in which temporal control information may be signalled to the decoder. A first way is via headers 436 as described above. A second way, which may be used as an alternative or additional signalling pathway is via data encoded within the residuals themselves. Figure 4B shows a case whereby data 474 may be encoded into an HH transformed coefficient and so may be extracted following entropy decoding 452. This data 474 may be extracted from the level 2 residual processing pipeline and passed to the temporal prediction component 466.

[0083] Each enhancement encoding or both enhancement encodings may be encapsulated into one or more enhancement bitstreams using a set of Network Abstraction Layer Units (NALUs). The NALUs are meant to encapsulate the enhancement bitstream in order to apply the enhancement to the correct base reconstructed frame. The NALU may for example contain a reference index to the NALU containing the base decoder reconstructed frame bitstream to which the enhancement has to be applied. In this way, the enhancement can be synchronised to the base stream and the frames of each bitstream combined to produce the decoded output video (i.e. the residuals of each frame of enhancement level are combined with the frame of the base decoded stream). A group of pictures may represent multiple NALUs.

[0084] Each frame may be composed of three different planes representing a different colour component, e.g. each component of a three-channel YUV video may have a different plane. Each plane may then have residual data that relates to a given level of enhancement, e.g. a Y plane may have a set of level 1 residual data and a set of level 2 residual data. In certain cases, e.g. for monochrome signals, there may only be one plane; in which case, the terms frame and plane may be used interchangeably. The level-1 residuals data and the level-2 residuals data may be partitioned as follows. Residuals data is divided into blocks whose size depends on the size of the transform used. The blocks are for example a 2x2 block of elements if a 2x2 directional decomposition transform is used or a 4x4 block of elements if a 4x4 directional decomposition transform is used. A tile is a group of blocks that cover a region of a frame (e.g. a M by N region, which may be a square region). A tile is for example a 32x32 tile of elements. As such, each frame may be divided into a plurality of tiles, and each tile of the plurality of tiles may be divided into a plurality of blocks. For colour video, each frame may be partitioned into a plurality of planes, where each plane is divided into a plurality of tiles, and each tile of the plurality of tiles is divided into a plurality of blocks.

[0085] With reference to Figures 4A and 4B, the following example relates to a temporal prediction process applied during the level 2 decoding. However, it is to be appreciated that the following temporal prediction process may additionally or alternatively be applied during the level 1 decoding.

[0086] In this example, the decoder 400, 480 is configured to receive a temporal_enabled parameter which specifies whether temporal prediction should be used when decoding a picture. The temporal_enabled parameter may be referred to herein as a first parameter with a first value that indicates temporal processing is enabled. The value of the temporal_enabled parameter may have a bit length of one bit. In this example, a value of 1 specifies that temporal prediction will be used when decoding a picture and a value of 0 specifies that temporal prediction will not be used when decoding a picture. The temporal_enabled parameter may be received once for a group of pictures associated with the encoded streams discussed above, the group of pictures being a collection of successive pictures within a coded video stream. The temporal_enabled parameter may be comprised in temporal control information signalled to the decoder, e.g. via headers 436 as described above.

[0087] Inputs to a temporal prediction process may include:

[0088] • a parameter nTbS specifying the size of the current transform block. For example, nTbS is equal to 2 when a 2x2 directional decomposition transform is to be used and nTbS is equal to 4 when a 4x4 directional decomposition transform process is to be used.

[0089] • a parameter temporal_tile_intra_signalling_enabled which specifies whether temporal prediction should be used when decoding a particular tile. In the decoding process described herein, generating the decoded video may be performed block wise. In this way, generating a block of elements in a frame of the decoded video can be performed without using another block of elements in the same frame of the decoded video that was previously generated. For this reason, the temporal prediction process can be performed in parallel for all blocks of elements in a frame as opposed to sequentially performing the temporal prediction process for each block of elements in the frame.

[0090] The above description of Figures 1 to 4B explains temporal prediction and the content of the temporal buffer(s) according to examples. The following methods explain how the temporal buffer may be “refreshed”, contrasting the approach in a background example, shown in Figure 5, against a new approach as shown in Figure 6.

[0091] Referring to Figure 5, this method comprises two independent procedures: a refresh procedure (steps S510 and S520) and a frame decoding procedure (steps S610 to S640).

[0092] At step S510, the decoder 400, 480 obtains a temporal_refresh parameter which specifies whether the temporal buffer 432 should be refreshed for the frame. If a frame comprises multiple planes, the refresh may be applied for all planes in the frame (i.e. for the frame that comprises the planes), or for specific indicated planes. The value of the temporal_refresh parameter may have a bit length of one bit. In this example, a value of 1 specifies that the temporal buffer 432 is to be refreshed for the frame and a value of 0 indicates that the temporal buffer 432 is not to be refreshed for the frame. The temporal_refresh parameter may be received once for each picture in the coded video stream. The temporal_refresh parameter may be comprised in the temporal control information signalled to the decoder, e.g. via headers 436 as described above. If the decoder comprises multiple temporal buffers 430, 432, the temporal_refresh parameter may be used to refresh all buffers or refresh one or more individual temporal buffers.

[0093] At step S520, the decoder 400, 480 performs a temporal refresh by writing zeros to the temporal buffer 432. The refresh may be applied to the whole frame area or may be applied to one or more specific blocks, tiles and planes and applied to temporal buffer(s) for one or more of the enhancement layers 430, 432, according to the temporal_refresh parameter.

[0094] At step S610, the decoder 400, 480 obtains an encoded frame from the video signal (e.g. bytestream).

[0095] At step S620, in order to decode the encoded frame, the decoder 400, 480 reads the contents of the temporal buffer(s) 430, 432. The contents of the temporal buffer are all zeros when step S620 is performed, and therefore the temporal buffer has no effect on decoding. However, step S620 occurs because the method of decoding (steps S610 to S640) is performed independently of the refresh (steps S510 to S520).

[0096] Then, at step S630, the decoder 400, 480 decodes the encoded frame according to one of the above-described methods. This method makes no assumption about the values stored in the temporal buffer(s) and therefore the steps 468, 472 of adding the contents of the temporal buffer(s) are performed regardless of the fact that those contents are currently zero values.

[0097] Finally, at step S640, the decoder 400, 480 updates the temporal buffer(s) with the latest residual values. This may be implemented as the feedback 473 and 469 towards the L-1 and L-2 temporal buffers 430, 432 in Figure 4B.

[0098] As mentioned above, the background example method of Figure 5 contains several inefficiencies as a result of performing the refresh operation independently from the next frame decode. The temporal buffer is read at step S620 despite containing zero values. Additionally, step S630 comprises performing arithmetic operations to combine the enhancement layer of the decoded frame with the temporal buffer, despite the fact that the temporal buffer currently contains zero values.

[0099] In view of these inefficiencies that the inventors have observed, a new method is proposed as illustrated in Figure 6. Referring to Figure 6, step S710 may be the same as the previously described step S510. The decoder 400, 480 obtains a temporal_refresh parameter which specifies whether the temporal buffer 432 should be refreshed for the frame. If a frame comprises multiple planes, the refresh may be applied for all planes in the frame (i.e. for the frame that comprises the planes), or for specific indicated planes. The value of the temporal_refresh parameter may have a bit length of one bit. In this example, a value of 1 specifies that the temporal buffer 432 is to be refreshed for the frame and a value of 0 indicates that the temporal buffer 432 is not to be refreshed for the frame. The temporal_refresh parameter may be received once for each picture in the coded video stream. The temporal_refresh parameter may be comprised in the temporal control information signalled to the decoder, e.g. via headers 436 as described above. If the decoder comprises multiple temporal buffers 430, 432, the temporal_refresh parameter may be used to refresh all buffers or refresh one or more individual temporal buffers.

[0100] However, after step S710, the decoder 400, 480 does not refresh the temporal buffer until it is next decoding a frame. In other words, the decoder 400, 480 skips a step S520 of writing anything to the temporal buffer, at this point. The decoder 400, 480 may optionally store, in memory, an indication that a refresh is pending. Alternatively, the decoder may have a live execution state and may add a refresh indicator to its live execution state.

[0101] At step S720, the decoder 400, 480 obtains an encoded frame from the video signal (e.g. bytestream). This is similar to step S610 of the background example, except the decoder 400, 480 recalls that a temporal refresh is pending (for example by referring to an indication stored in memory or included in its program state.

[0102] For example, the decoder 400, 480 can wait to refresh the temporal buffer by storing a “dirty” flag associated with the temporal buffer in a working memory. The “dirty” flag may have a length of one bit, and indeed may be a copy of the temporal_refresh parameter. When the decoder 400, 480 decodes the next frame at steps S710 to S740, the decoder 400, 480 detects the “dirty” flag and responds by performing a combined decode / refresh procedure as follows. At step S730, the decoder 400, 480 reconstructs the decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer that is to be refreshed. For example, referring to Figure 4B, if the temporal_refresh parameter indicated to refresh the L-1 temporal buffer 430, then the decoder 400, 480 skips the steps of reading the L-1 temporal buffer and adding 472 the contents of the level 1 temporal buffer 430 into the level 1 residual processing pipeline. Similarly, referring to Figure 4B, if the temporal_refresh parameter indicated to refresh the L-2 temporal buffer 432, then the decoder 400, 480 skips the steps of reading the L-2 temporal buffer and adding 468 the contents of the level 2 temporal buffer 432 into the level 2 residual processing pipeline.

[0103] Finally, at step S740, the decoder 400, 480 writes the latest residual values to the temporal buffer(s). This may be implemented as the feedback 473 and 469 towards the L-1 and L-2 temporal buffers 430, 432 in Figure 4B.

[0104] However, unlike the “update” of step S640, step S740 is a refresh of the temporal buffer. More specifically, the refreshed area may be the whole frame area or may be one or more specific blocks, tiles and planes and the refresh may be applied to temporal buffer(s) for one or more of the enhancement layers 430, 432, according to the temporal_refresh parameter. Therefore, in a case where the enhancement encoding of the encoded frame includes residual values for only a subset Si of the tiles, blocks or planes of the frame, the decoder 400, 480 nevertheless writes data to the whole of the refreshed area of the temporal buffer. This refresh may be performed as a single write operation for the whole of the refreshed area. This refresh may comprise writing zero values to the temporal buffer for any tiles, blocks or planes of the frame for which no residual values were explicitly included in the enhancement encoding. In other words, the refresh may comprise writing zero values to the temporal buffer for any tiles, blocks or planes which are not part of the subset Si.

[0105] In the background example of Figure 5, the separate refresh (step S520), read (step S620) and update (S640) steps each involve copying data to / from the temporal buffer. By instead performing a single write step (S740), a significant amount of memory bandwidth is freed for other decoder operations. Additionally, in the context of streaming video, it is likely that the time between steps S710 and S720 (or between steps S510 and S610) is short, and there may be no wait at all. As a result, the processing resources saved by avoiding a separate refresh procedure (steps S520 etc.), and instead performing a combined decode / refresh procedure, have an appreciable effect on the maximum frame rate of the decoder.

[0106] Step 740 may alternatively be performed before step S730, because step S730 does not need to include reading the temporal buffer.

[0107] Although one or more embodiments have been described in relation to LCEVC, aspects of the invention may also be implemented in other hierarchical coding schemes. In such embodiments, the ‘base encoder’ may correspond to an encoder configured to encode a low layer (corresponding to a low quality) of a hierarchical coding scheme. In such embodiments, the ‘enhancement encoder’ may correspond to an encoder configured to encode a high layer (corresponding to a high quality, i.e. a higher quality than the low layer) of the hierarchical coding scheme. For example, the base encoder may correspond to an encoder configured to encode a lowest layer of the hierarchical coding scheme and the enhancement encoder may correspond to a encoder configured to encoding a (e.g. first) enhancement layer of the hierarchical coding scheme. More generally, the base encoder may correspond to a encoder configured to encode a nth layer of a hierarchical coding scheme and the enhancement encoder may correspond to a encoder configured to encode a (n+1)th layer of the hierarchical coding scheme.

[0108] A base encoder may itself be a multi-layer encoder. In particular, a base encoder may comprise a base encoder and one or more enhancement encoders.

[0109] An enhancement encoder may comprise multiple encoders. In particular, an enhancement encoder may comprise multiple enhancement encoders. A base encoding may be the output of a one or more layers of encoding. For example, a base encoding may be a first coding layer combined one or more further (e.g. enhancement) coding layers.

[0110] An enhancement encoding may comprise one or more layers of enhancement. For example, a base encoding may a base layer (i.e. output by a single layer codec such as HEVC, WC, and so forth) combined with a first layer of LCEVC residuals, whilst the enhancement encoding may be a second layer of LCEVC residuals. In a further example, a base encoding may a lowest layer encoded in accordance with the SMPTE VC-6 standard combined with one or more VC-6 enhancement layers, whilst the enhancement encoding may be one or more 'higher' layer enhancement layers of the VC-6 standard.

[0111] The above embodiments are to be understood as illustrative examples. Further embodiments are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed within the scope of the accompanying claims.

Claims

CLAIMS1 . A method of decoding a video signal comprising: obtaining an encoded frame from the video signal, the encoded frame comprising a base encoding and an enhancement encoding; determining whether to refresh a temporal buffer; and if it is determined that the temporal buffer is to be refreshed: reconstructing a decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer; and writing a set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame.

2. A method according to claim 1 , wherein writing the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, comprises: obtaining a set of residuals from the enhancement encoding of the encoded frame, wherein the residuals are differences between: a base decoded frame obtained by decoding the base encoding; and an input frame, wherein the base encoding is generated by encoding the input frame; and writing the set of residuals to the temporal buffer.

3. A method according to claim 2, wherein each frame of the video signal has a frame area divided into a plurality of tiles and / or a plurality of blocks, and the set of residuals includes residual values for a subset of the plurality of tiles or a subset of the plurality of blocks, wherein writing the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, further comprises: writing a zero value to the temporal buffer for a tile of the frame area which is not in the subset of the plurality of tiles and / or for a block of the frame area which is not in the subset of the plurality of blocks.

4. A method according to claim 3, comprising:constructing a complete set of new temporal predictions for all tiles and / or all blocks of the frame area, consisting of the set of residuals obtained from the enhancement encoding of the encoded frame and zero values for any tiles and / or blocks for which no residual values are included in the enhancement encoding of the encoded frame; and writing the complete set of new temporal predictions to the temporal buffer.

5. A method according to any preceding claim, further comprising: if it is determined that the temporal buffer is not to be refreshed: reading the temporal buffer to obtain a set of previous temporal predictions; and reconstructing the decoded frame based on the base encoding and enhancement encoding of the encoded frame and based on the set of previous temporal predictions.

6. A method according to claim 5, wherein the previous temporal predictions comprise a set of residuals for an earlier frame in the video signal preceding the encoded frame.

7. A method according to any preceding claim, further comprising: receiving a refresh parameter in the video signal, wherein the determining whether to refresh the temporal buffer is based on the refresh parameter.

8. A method according to claim 7, wherein the refresh parameter is part of the encoded frame.

9. A method according to claim 7, wherein the refresh parameter is part of a control component of the video signal, separate from the encoded frame.

10. A decoder for decoding a video signal, the decoder comprising: a memory comprising a temporal buffer configured to store a set of temporal predictions; and a processor configured to:obtain an encoded frame from the video signal, the encoded frame comprising a base encoding and an enhancement encoding; determine whether to refresh a temporal buffer; and if it is determined that the temporal buffer is to be refreshed: reconstruct a decoded frame based on the base encoding and enhancement encoding of the encoded frame, without reading the temporal buffer; and write a set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame.

11. A decoder according to claim 10, wherein the processor is configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by: obtaining a set of residuals from the enhancement encoding of the encoded frame, wherein the residuals are differences between a base decoded frame obtained by decoding the base encoding and an input frame, wherein the base encoding is generated by encoding the input frame; and writing the set of residuals to the temporal buffer.

12. A decoder according to claim 11 , wherein each frame of the video signal has a frame area divided into a plurality of tiles and / or a plurality of blocks , and the set of residuals includes residual values for a subset of the plurality of tiles or a subset of the plurality of blocks, wherein the processor is configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by further: writing a zero value to the temporal buffer for a tile of the frame area which is not in the subset of the plurality of tiles and / or a block of the frame area which is not in the subset of the plurality of tiles.

13. A decoder according to claim 12, wherein the processor is configured to write the set of new temporal predictions to the temporal buffer, based on the enhancement encoding of the encoded frame, by: constructing a complete set of new temporal predictions for all tiles and / or all blocks of the frame area, consisting of the set of residuals obtained from theenhancement encoding of the encoded frame and zero values for any tiles and / or blocks for which no residual values are included in the enhancement encoding of the encoded frame; and writing the complete set of new temporal predictions to the temporal buffer.

14. A decoder according to any of claims 10 to 13, wherein the processor is further configured to: it is determined that the temporal buffer is not to be refreshed: read the temporal buffer to obtain a set of previous temporal predictions; and reconstruct the decoded frame based on the base encoding and enhancement encoding of the encoded frame and based on the set of previous temporal predictions.

15. A decoder according to claim 14, wherein the previous temporal predictions comprise a set of residuals for an earlier frame in the video signal preceding the encoded frame.

16. A decoder according to any of claims 10 to 15, wherein the processor is configured to: receive a refresh parameter in the video signal; and determine whether to refresh the temporal buffer based on the refresh parameter.

17. A decoder according to claim 16, wherein the refresh parameter is part of the encoded frame.

18. A decoder according to claim 16, wherein the refresh parameter is part of a control component of the video signal, separate from the encoded frame.

Images