Video Encoder Including Frame Buffer Assembly - Patent application

Abstract

A video encoder (100) encodes a sequence of frames (101) to obtain a sequence of encoded frames (102). A frame buffer assembly (104) provides, for a current frame to be encoded, a representation (125) resulting from lossy compression of a decoded version of a previously encoded frame. An encoding assembly (103) encodes the current frame at least in part with reference to the representation (125) of the previously encoded frame. The video encoder generates a data set constituting an encoded video comprising the sequence of encoded frames (102). The data set includes an indication (124) of the lossy compression applied by the frame buffer assembly (104) to provide a representation (125) of a previously encoded frame that, in relation to the previously encoded frame, served at least in part as a reference for encoding the current frame.

Description

[Technical field]

[0001] Aspects of the invention relate to a video encoder including a frame buffer assembly. The video encoder may be used, for example, in a camera device, to encode a sequence of captured frames. The camera device may then store the encoded sequence of frames and / or transmit the sequence to another device, constituting a video image. Other aspects of the invention relate to a method for encoding a sequence of frames, a computer program for a video encoder, a dataset including the encoded sequence of frames, and a video decoder. [Background technology]

[0002] There are many video coding techniques, including inter-mode coding. In inter-mode coding, a frame, or at least a part thereof, is coded with reference to one or more previously coded frames. Typically, a differential representation of the frame portion to be coded is generated. This differential representation corresponds to the difference between the frame portion to be coded, on the one hand, and a reference frame portion obtained from a decoded version of the previously coded frame, on the other hand. The differential representation therefore contains a residual, which is preferably as small as possible for efficient coding.

[0003] Motion prediction may be performed to find a frame portion in a decoded version of one or more previously encoded frames that results in the smallest residual when used as a reference frame portion. Motion vector data is then provided that indicates the best referenced frame portion for the frame portion to be encoded. Alternatively, motion prediction may be omitted. In that case, a predefined position within a frame may serve as a criterion for specifying the reference frame portion to be used in inter mode encoding.

[0004] Inter mode coding requires a so-called frame buffer, in which the decoded version of one or more previously coded frames is at least temporarily stored. Typically, a frame buffer is a dedicated memory circuit that provides a sufficiently large memory capacity at an acceptable cost. A frame buffer can be, for example, a Dynamic Random Access Memory (DRAM). Although the cost may be acceptable, the frame buffer can still represent a significant part of the total cost of the video coder. This is especially true for high definition video coders.

[0005] Furthermore, when coding video in inter mode, a relatively large amount of data needs to be written to and read from the frame buffer per unit of time, which requires a relatively large bandwidth, which may entail a relatively large power consumption. Again, the power consumption required to obtain a sufficiently large bandwidth may constitute a significant portion of the total power consumption of the video coder. Again, this is particularly the case for high definition video coders.

[0006] Frame buffer compression has been proposed to make frame buffers smaller, reducing costs, and reducing bandwidth requirements and power consumption. Instead of storing a decoded version of a previously encoded frame in the frame buffer, a compressed version of this decoded version is stored. In general, frame buffer compression must be mathematically lossless, or at least visually lossless. Otherwise, the current frame portion would be coded in inter mode against a degraded version of what would have been the reference frame portion if frame buffer compression had not been used. Frame buffer compression can therefore introduce visual artifacts, especially when several subsequent frames are coded at least partially in inter mode. This is due to an accumulation of errors across these frames.

[0007] Patent document 1 describes a frame buffer compression technique that is particularly suitable for applications where the encoded video data rate is relatively slow. In this frame buffer compression technique, the decoded version of an encoded frame is effectively divided into relatively large parts, which are then encoded for storage in a frame buffer. The encoding may be lossy to some extent, and a relatively high compression ratio may be obtained. In principle, such lossy encoding may affect the encoding efficiency, or the image quality, or both. However, it has been found that when the encoded video data rate is relatively slow, the loss in the encoding efficiency, or the image quality, or both, may be relatively small and may even become inconsequential. The image quality may be relatively close to that provided by a conventional video coder that does not compress the reference frame, or that compresses the reference frame slightly in a lossless or near-lossless manner. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] International Publication No. 2019 / 025640 Summary of the Invention [Problem to be solved by the invention]

[0009] What is needed is a video coding technique that allows for superior overall performance in terms of cost, power consumption, and image quality, regardless of the video data rate. [Means for solving the problem]

[0010] An aspect of the invention as defined in claim 1 provides a video encoder configured for encoding a sequence of frames to obtain an encoded sequence of frames, the video encoder comprising: a frame buffer assembly configured to provide, for a current frame to be encoded, a representation of a previously encoded frame, said representation resulting from lossy compression of a decoded version of the previously encoded frame; a coding assembly configured to code a current frame at least in part with reference to a representation of a previously coded frame, - the video encoder is configured to provide a dataset, the dataset comprising a sequence of encoded frames and including an indication of a lossy compression applied to provide a representation thereof which, relative to previously encoded frames, served at least in part as a reference for encoding a current frame.

[0011] A further aspect of the invention as defined in claim 11 is a method for encoding a sequence of frames to obtain an encoded sequence of frames, the method comprising the steps of: - providing, for a current frame to be encoded, a representation of a previously encoded frame by lossy compression of a decoded version of the previously encoded frame; - encoding a current frame at least in part with reference to a representation of a previously encoded frame; - providing a data set comprising a sequence of encoded frames and including an indication of the lossy compression applied to provide a representation thereof which, relative to previously encoded frames, served at least in part as a reference for encoding the current frame.

[0012] A yet further aspect of the invention as defined in claim 12 provides a computer program for a video encoder, the computer program comprising a set of instructions enabling the video encoder to perform the method as defined above.

[0013] A still further aspect of the invention as defined in claim 13 provides a dataset, the dataset comprising: - a sequence of encoded frames, the sequence including encoded frames that have been at least partially encoded with reference to representations of other previously encoded frames, the representations resulting from lossy compression of decoded versions of the other previously encoded frames; - an indication of the lossy compression applied to provide a representation of the encoded frame that served at least in part as a reference for encoding the encoded frame relative to other previously encoded frames.

[0014] A yet further aspect of the invention as defined in claim 14 provides a video decoder configured to decode a sequence of encoded frames to obtain a sequence of decoded frames, the video decoder comprising: a frame buffer assembly configured to provide a representation of previously decoded frames for a coded frame to be decoded; a decoding assembly configured to at least partially decode the encoded frames with reference to a representation of a previously decoded frame, the video decoder is configured to obtain from a dataset comprising a sequence of encoded frames, the sequence comprising an indication of lossy compression applied by a frame buffer assembly in the video encoder that generated the sequence of encoded frames, the indication comprising lossy compression applied by a frame buffer assembly in the video encoder that generated the sequence of encoded frames, the frame buffer assembly of the decoder is configured to use the indicators obtained from the dataset to obtain a representation that serves at least in part as a reference for decoding the current frame in order to process the previously decoded frames in a manner equivalent to the lossy compression applied by the frame buffer assembly in the video encoder; Effect of the Invention

[0015] In each of these aspects, the measure of lossy compression allows the decoder to recreate with sufficient accuracy a representation of a previously coded frame that served as a reference in the inter mode coding, even when the representation was obtained by compressing the previously coded frame to a relatively large extent. Moreover, this also significantly reduces the accumulation of errors over a series of frames that are at least partially coded in inter mode. Thus, a relatively high compression ratio can be used for the frame buffer compression without significant loss of image quality. This in turn allows a significant reduction in the memory capacity that the frame buffer must provide, thus reducing costs, significantly reducing bandwidth requirements and reducing power consumption. Moreover, the measure of lossy compression can be provided without requiring significant resources and processing that could offset the cost and power consumption reduction. Thus, the present invention allows video coding with good overall performance in terms of cost, power consumption and image quality, regardless of the video data rate.

[0016] For purposes of illustration, some embodiments of the invention will now be described in detail with reference to the accompanying drawings, in which further features will be presented and in which advantages will become apparent, some of which will be defined in the dependent claims. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic block diagram of a video encoder. [Diagram 2] FIG. 2 is a schematic block diagram of a frame buffer assembly in a video encoder. [Diagram 3] FIG. 3 is a conceptual diagram of a data set that may be provided by a video encoder. [Figure 4] FIG. 4 is a schematic block diagram of a video decoder. [Diagram 5] FIG. 5 is a schematic block diagram of a frame buffer assembly in a video decoder. [Figure 6]FIG. 6 is a schematic block diagram of an alternative video encoder. [Figure 7] FIG. 7 is a schematic block diagram of a frame buffer assembly in an alternative video encoder. [Figure 8] FIG. 8 is a schematic block diagram of an alternative video decoder. [Figure 9] FIG. 9 is a schematic block diagram of a frame buffer assembly in an alternative video decoder. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] FIG. 1 illustrates a schematic block diagram of a video encoder 100. Basically, the video encoder 100 may encode a sequence of frames 101 that may constitute a video to obtain a sequence of encoded frames 102. The sequence of encoded frames 102 may thus constitute an encoded video that includes a smaller amount of data than the video in its original form. The video encoder 100 may thus encode the video to enable efficient storage of the video, or efficient transmission of the video, or both. The video encoder 100 may be part of a device capable of capturing video, such as a smartphone, and may add value to the device due to its efficiency.

[0019] The video encoder 100 includes an encoding assembly 103 and a frame buffer assembly 104. The encoding assembly 103 includes various functional modules, such as a motion estimation and compensation module 105, an intra / inter mode adaptation module 106, a decorrelation module 107, a quantization module 108, an entropy coding module 109, a data packaging module 110, and a controller 111. The above-mentioned modules may perform encoding-related operations and may therefore be considered to form an encoding chain 105-111.

[0020] The encoding assembly 103 further comprises functional modules that perform decoding-related operations and thus may be considered as forming decoding chains 112-114, such as a dequantization module 112, an inverse transformation module 113 and a reconstruction module 114. The functional modules 105-114 of the encoding assembly 103 may for example each be implemented in the form of a dedicated circuit, a programmable circuit or a suitably programmed processor, or any combination of these.

[0021] The video encoder 100 basically works as follows. In practice, the video encoder 100 divides a frame to be encoded into blocks of pixels. The blocks of pixels constitute frame portions, which have a specific location within the frame to be encoded. The video encoder 100 may encode each frame portion separately that together form the frame to be encoded. The frame to be encoded will be referred to as the current frame in the following for convenience and clarity.

[0022] The frame portion may be coded in one of two modes: intra mode and inter mode. In inter mode, the frame portion is coded with reference to a representation 115 of a previously coded frame. The frame buffer assembly 104 provides the representation 115 of a previously coded frame. In intra mode, the frame portion is coded without reference to other frames. In this mode, the frame portion may be coded "as is" or may be coded with reference to one or more other frame portions in the current frame.

[0023] The controller 111 may determine whether the frame portion is to be coded in inter mode or not. This determination may be made based on a predefined coding scheme. Alternatively, or supplementarily, the determination may be made dynamically depending on whether the intra mode or the inter mode is expected to be more advantageous in terms of, for example, data compression ratio, or image quality, or both.

[0024] In inter mode, the motion prediction and compensation module 105 searches in the representation 115 of the previously coded frame for a portion 116 that is most similar to the frame portion to be coded. This best-matching frame portion 116 may have a position in the previously coded frame that is different from the specific position that the frame portion to be coded has in the current frame. The motion prediction and compensation module 105 provides motion vector data that identifies this position difference. Furthermore, the motion vector data may identify the previously coded frame to which the best-matching frame portion 116 belongs. This is because the first encoder may search several previously coded frames to find the best-matching frame portion 116.

[0025] The intra / inter mode application module 106 provides an intra / inter prescribed frame portion 117, which is applied to the decorrelation transformation module 107. In inter mode, the intra / inter prescribed frame portion 117 is a differential representation of a frame portion. This differential representation corresponds to the difference between the frame portion to be coded on the one hand and the best-matching frame portion 116 found by the motion prediction and compensation module 105 on the other hand. The differential representation contains a residual that represents this difference.

[0026] In intra mode, the intra / inter prescribed frame portion 117 applied by the decorrelation transform module 107 may simply correspond to the frame portion to be coded. In that case, the intra / inter prescribed frame portion 117 comprises pixels. Alternatively, the intra / inter mode application module 106 may provide a representation of the frame portion to be coded that references one or more other portions in the current frame. Such other portions that may serve as references may be neighbors of the frame portion to be coded. In this alternative, the intra / inter prescribed frame portion 117 may comprise a residual.

[0027] The decorrelation transform module 107 applies a decorrelation transform to the intra / inter prescribed frame portion 117 provided by the intra / inter mode application module 106. The decorrelation transform module 107 thus provides a transformed intra / inter prescribed frame portion 118, which is a representation of the intra / inter prescribed frame portion 117 in the transformed domain. The transformed intra / inter prescribed frame portion 118 actually contains transformed samples that are substitutes for the pixels or residuals contained in the decorrelated transformed intra / inter prescribed frame portion 117. The decorrelation transform is performed such that the transformed samples in the transformed intra / inter prescribed frame portion 118 generally have lower entropy than the pixels or residuals contained in the intra / inter prescribed frame portion 117. The decorrelation transform can be, for example, in the form of a discrete cosine transform (commonly referred to by the acronym DCT) or a wavelet transform.

[0028] The quantization module 108 quantizes the transformed samples in the transformed intra / inter prescribed frame portion 118. Thus, the quantization module 108 provides a quantized version 119 of the transformed intra / inter prescribed frame portion 118. By quantizing the transformed samples, the quantized version 119 may contain a smaller amount of data than the transformed intra / inter prescribed frame portion 118 provided and quantized by the decorrelation transform module 107. Thus, the quantization provides data compression. In principle, this data compression is lossy, i.e., the quantization involves a loss of information.

[0029] The quantization module 108 may quantize the transform samples with a resolution that depends on how much the transform samples affect image quality. If the transform samples affect image quality to a relatively large extent, the resolution may be relatively high. Conversely, if the transform samples affect image quality to a relatively small extent, the resolution may be relatively low. The transform samples have a predetermined location in the transform intra / inter defined frame portion 118. The location may indicate how much the transform samples affect image quality.

[0030] The entropy coding module 109 applies entropy coding to the quantized version 119 of the transformed intra / inter prescribed frame portion 118. The entropy coding thus provides a quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118. Due to the entropy coding, this quantized and coded version 120 may contain a smaller amount of data than the quantized version 119 of the transformed intra / inter prescribed frame portion 118. The entropy coding thus provides further data compression. This further data compression may be lossless, and entropy coding, as opposed to quantization, does not necessarily involve a loss of information. The entropy coding may be based on the techniques described, for example, in US Pat. No. 9,332,258.

[0031] The controller 111 may control the data compression provided by the quantization in the quantization module 108 and the entropy coding in the entropy coding module 109. This control may be based on the data contained in the intra / inter prescribed frame portion 117 on the one hand and on the target amount of data to be included in the quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118 on the other hand. Based on these inputs, the controller 111 may set the resolution of the quantization performed by the quantization module 108. The controller 111 may further select an entropy coding scheme that provides a high level of data compression.

[0032] The data packaging module 110 adds the quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118 to a data set that will constitute the coded video. The data packaging module 110 may further add an indication of whether intra or inter mode has been applied. If the frame portion is coded in inter mode, the data packaging module 110 will further add motion vector data provided for this frame portion by the motion estimation and compensation module 105. The data packaging module 110 will further add information about how the quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118 was generated. This information may include, for example, information about the resolution of the applied quantization.

[0033] Thus, the data packaging module 110 constructs the encoded frame portions and embeds them in the data set together with information identifying the encoded frame portions. Once the video encoder 100 has encoded all frame portions in the current frame, the data set will contain an encoded version of the current frame, which will be referred to as the encoded current frame. The data packaging module 110 may output the data set in the form of a data stream having a particular structure. Alternatively, the data packaging module 110 may output the data set in the form of a data file, for example for storage in a device.

[0034] As mentioned above, the video encoder 100 shown in Fig. 1 comprises the above mentioned decoding chains 112-114, which include a dequantization module 112, an inverse transform module 113 and a reconstruction module 114. The decoding chains 112-114 make it possible to obtain a decoded version of the current frame encoded in the video encoder 100. The decoding chains 112-114 basically operate as follows.

[0035] The dequantization module 112 applies inverse quantization to the quantized version 119 of the transformed intra / inter prescribed frame portion 118. Inverse quantization is an operation that reverses the quantization in the quantization module 108 to roughly regenerate the aforementioned transformed samples. The dequantization module 112 thus provides a dequantized version 121 of the transformed intra / inter prescribed frame portion 118. This dequantized version 121 can be considered as a coarse representation of the transformed intra / inter prescribed frame portion 118 provided by the transform coding modules in the coding chains 105-111.

[0036] The inverse transform module 113 applies an inverse decorrelation transform to the dequantized version 121 of the transformed intra / inter prescribed frame portion 118. The inverse transform module 113 thus provides a decoded version 122 of the intra / inter prescribed frame portion 117. The decoded version 122 corresponds to a certain extent to the intra / inter prescribed frame portion 117 provided by the intra / inter mode application module 106 in the coding chain 105-111. If the quantization resolution is relatively high, the decoded version 122 will correspond to a relatively high degree to the above-mentioned intra / inter prescribed frame portion 117. In the extreme case where no quantization is applied, the decoded version 122 will correspond exactly to the intra / inter prescribed frame portion 117. Conversely, if the quantization resolution is relatively low, the decoded version 122 will correspond to the intra / inter prescribed frame portion 117 to a relatively low degree.

[0037] The reconstruction module 114 provides a decoded version 123 of the frame portion coded in the coding chains 105-111 as described above. If the frame portion is coded in inter mode, the reconstruction module 114 adds a decoded version 122 of the intra / inter definition frame portion 117 to the best-matched frame portion 116 which serves as a reference. As described above, the motion estimation and compensation module 105 provides this best-matched frame portion 116. The above-mentioned addition generates a decoded version 123 of the frame portion, which for convenience will be referred to hereinafter as the decoded frame portion 123.

[0038] If the frame portion was coded in intra mode, the reconstruction module 114 may simply take the decoded version 122 of the intra / inter prescribed frame portion 117 provided by the inverse transform module 113 as the decoded frame portion 123. As mentioned above, the intra mode may allow the representation of the frame portion with reference to one or more other portions in the current frame. The reconstruction module 114 may then use these references to provide the decoded frame portion 123 based on the decoded version of the intra / inter prescribed frame.

[0039] The frame buffer assembly 104 receives the decoded frame portion 123 provided by the reconstruction module 114 at the end of the decoding chain 112-114. Thus, when the current frame is coded in its entirety, the frame buffer assembly 104 receives the respective decoded frame portions, all of which may be obtained as described above. The frame buffer assembly 104 thus receives a decoded version of the current frame, which for convenience will be called the decoded current frame. The frame buffer assembly 104 may then provide a representation of the decoded current frame when a subsequent frame is coded at least partially in inter mode. This representation may then serve as a reference for coding one or more frame portions of this subsequent frame in inter mode.

[0040] Up to this point, operations have been described that may correspond to some extent to those performed by a video encoder conforming to one of the video encoding standards, e.g., MPEG-2, MPEG-4, or HEVC. What distinguishes the video encoder 100 shown in FIG. 1 from the video encoders described above are the additional operations performed by the frame buffer assembly 104 and the data packaging module 110. Generally speaking, these operations include:

[0041] The frame buffer assembly 104 applies lossy compression to the decoded current frame, which provides a compressed version of the decoded current frame. The frame buffer assembly 104 at least temporarily stores the compressed version. The compressed version of the decoded current frame may then be later decompressed to provide a decompressed version of the decoded current frame. The decompressed version then constitutes a representation of the current frame that may be used as a reference to encode one or more frame portions in a subsequent frame in inter mode.

[0042] Additionally, frame buffer assembly 104 provides an indication 124 of the applied lossy compression. Indicator 124 may indicate the degree to which the decoded current frame is compressed for temporary storage within frame buffer assembly 104. Data packaging module 110 may include indicator 124 in a data set that includes the encoded current frame and other encoded frames that form the encoded video.

[0043] FIG. 2 illustrates a frame buffer assembly 104 in the video encoder 100. FIG. 2 provides a schematic block diagram of the frame buffer assembly 104. The frame buffer assembly 104 includes a data memory 201, a controller 202, and various functional modules. These functional modules include a decorrelation module 203, a delay buffer 204, a quantization module 205, and an entropy coding module 206. The above-mentioned functional modules may be considered as constituting a buffer compression chain 203-206. The frame buffer assembly 104 further includes functional modules such as an entropy decoding module 207, a dequantization module 208, and an inverse transform module 209, which may be considered as constituting a buffer decompression chain 207-209. The functional modules of the frame buffer assembly 104 may be implemented, for example, in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processor, or any combination thereof, respectively.

[0044] For the sake of distinction, the controller 202 of the frame buffer assembly 104 will be referred to below as the buffer controller 202. For the same reason, the decorrelation module 203, the quantization module 205, the entropy coding module 206, the entropy decoding module 207, the dequantization module 208, and the inverse transform module 209 will be referred to below as the buffer decorrelation module 203, the buffer quantization module 205, the buffer entropy coding module 206, the buffer entropy decoding module 207, the buffer dequantization module 208, and the buffer inverse transform module 209, respectively. Therefore, for the sake of distinction, the adjective "buffer" is added.

[0045] The functional modules of the frame buffer assembly 104 may be part of an integrated circuit that may further include the encoding assembly 103 of the video encoder 100 shown in FIG. 1. However, the data memory 201 of the frame buffer assembly 104 may also be in the form of a separate circuit, for example a dynamic random access memory circuit, commonly referred to by the acronym DRAM. Such an embodiment may also include a data write / read channel, through which data may be written to the data memory 201 and data may be read from the data memory 201. The data write / read channel may include a bus, having a bandwidth that sets a limit on the amount of data per unit time that can be written to and read from the data memory 201. That is, the bandwidth defines the maximum data rate of writing and reading.

[0046] The frame buffer assembly 104 basically operates as follows: It is assumed that the frame buffer assembly 104 receives decoded frame portions 123 provided by the reconstruction module 114 as described above. The current frame, and other decoded frame portions that may originate from other frames, may be processed in a manner similar to that described below.

[0047] The buffer decorrelation transform module 203 applies a decorrelation transform to the decoded frame portion 123. The buffer decorrelation transform module 203 thus provides the transformed decoded frame portion 210. The decorrelation transform applied by the buffer decorrelation transform module 203 may be similar to or different from the decorrelation transform applied by the decorrelation transform module 107 in the encoding assembly 103 of the video encoder 100 shown in FIG. 1. In either case, the transformed decoded frame portion 210 actually contains transformed samples that are substitutes for the pixels contained in the decorrelated decoded frame portion 123. Again, the latter decorrelation transform is performed such that the transformed samples in the transformed decoded frame portion 210 generally have lower entropy than the pixels in the decoded frame portion 123 provided by the reconstruction module 114.

[0048] The delay buffer 204 may temporarily store the transform decoded frame portions 210 provided by the buffer decorrelation transform module 203. This temporary storage may aid in achieving good compression with minimal loss of image quality, as described below with respect to the buffer controller 202.

[0049] The buffer quantization module 205 quantizes the transform samples in the transform decoded frame portion 210. Thus, the buffer quantization module 205 provides a quantized compressed version 211 of the transform decoded frame portion 210. That is, by quantizing the transform samples, the buffer quantization module 205 provides data compression as described above with respect to the quantization module 108 in the encoding assembly 103. The data is compressed to a degree that depends on the resolution of the quantization performed. The lower the resolution, the greater the degree to which the data is compressed, but the greater the loss of information.

[0050] The video encoder 100 is configured such that lossy compression of the decoded frame portion 123 need not significantly degrade image quality, even to a relatively large extent, as will be explained further below. In any case, like the quantization module 108 in the encoding assembly 103, the buffer quantization module 205 may quantize the transform samples to a resolution that depends on how much the transform samples affect image quality.

[0051] The buffer entropy coding module 206 applies entropy coding to the quantized, compressed version 211 of the transform decoded frame portion 210 provided by the buffer quantization module 205. Thus, the buffer entropy coding module 206 provides a quantized and encoded compressed version 212 of the transform decoded frame portion 210. That is, the entropy coding provides data compression as described above with respect to the entropy coding module 109 in the encoding assembly 103. The quantized and encoded compressed version 212 of the transform decoded frame portion 210 will be referred to hereinafter as the compressed decoded frame portion 212 for convenience.

[0052] The compressed decoded frame portions 212 are sent to the data memory 201 and stored there. In the process of encoding the current frame, the buffer compression chains 203-206 provide the respective compressed decoded frame portions in succession. These successive compressed decoded frame portions 212 sent to the data memory 201 form a memory input data stream. This memory input data stream should preferably not exceed the maximum data rate of the data write / read channel, which is defined by the bandwidth as mentioned above.

[0053] The buffer controller 202 may control the quantization resolution that the buffer quantization module 205 applies in the buffer compression chains 203-206. This control may be based on a target amount of data contained in the transform decoded frame portion 210 on the one hand and in the compressed decoded frame portion 212 on the other hand. This target amount of data may be related to the maximum data rate of the data write / read channel. For example, the target amount of data may correspond, on average, to this maximum data rate divided by the frequency at which the decoded frame portions are provided. Also, the target amount of data may be somewhat below the aforementioned percentage in order to have a certain margin. By controlling the quantization resolution in this way, the buffer controller 202 may ensure that the memory input data stream does not exceed, or only occasionally exceeds, the maximum data rate of the data write / read channel.

[0054] The delay buffer 204 allows the buffer controller 202 to control the quantization resolution by considering multiple transform-decoded frame portions, which may be adjacent in location. Some of these portions may be relatively uniform, while others may be more complex. By considering these different portions together, the quantization resolution can be controlled such that the quantization error is more evenly distributed across these portions, compared to a control that considers only one transform-decoded frame portion, or only a small number of transform-decoded frame portions. A more even distribution of the quantization error results in less loss of image quality for a given level of compression.

[0055] The buffer controller 202 may provide an indicator 124 of the lossy compression applied to the decoded frame portion 123 as described above. As described above, the indicator 124 may indicate the degree to which the decoded frame portion 123 has been compressed. In particular, the indicator 124 may indicate, directly or indirectly, the level of quantization applied by the buffer quantization module 205.

[0056] For example, the indicator 124 may relate to a target obtained by lossy compression. This compression target may correspond, for example, to the above-mentioned target data volume, the maximum data rate to be considered, or the bandwidth of the data write / read channel. In that case, the indicator 124 may be considered as an indirect indicator 124 of the applied lossy compression. The indicated compression target allows other compression chains to reproduce the lossy compression applied by the buffer compression chains 203-206 in the frame buffer assembly 104 shown in FIG. 2. Such other compression chains may be similar to the buffer compression chains 203-206. If the indicator 124 of lossy compression is of a general characteristic, such as, for example, the maximum data rate to be considered, or the bandwidth of the data write / read channel, an element other than the buffer controller 202 may provide this indicator.

[0057] Additionally, the lossy compression indicator 124 provided by the buffer controller 202 may be a direct characteristic by specifying a quantization resolution. Thus, the indicator may include, for example, one or more quantization steps applied by the buffer quantization module 205 in quantizing the transform decoded frame portion 210. Different quantization steps define different raw resolutions applied to different groups of samples in the transform decoded frame portion 210.

[0058] The lossy compression indicator 124 may be provided separately for each decoded frame portion that is compressed and written to the data memory 201. That is, as described above, when the decoded frame portion 123 is compressed and written to the data memory 201, a subsequent decoded frame portion is compressed and written to the data memory 201. This subsequent decoded frame portion is then decorrelated and transformed to become another transform decoded frame portion. This other transform decoded frame portion contains a larger or smaller amount of data. This may result in that its quantization resolution may be set to a higher or lower level, respectively. Thus, the compression of the subsequent decoded frame portion may be less lossy than the compression of the decoded frame portion 123 described above, or may be even more lossy.

[0059] Once the current frame is encoded, the buffer compression chains 203-206 compress the respective decoded frame portions that constitute the decoded current frame. Thus, the buffer compression chains 203-206 generate respective compressed decoded frame portions that are written to the data memory 201. Each compressed decoded frame portion constitutes a compressed version of the decoded current frame. This compressed version will be referred to in the following as the compressed decoded current frame. Thus, the compressed decoded current frame results from a lossy compression in the buffer compression chains 203-206 described above.

[0060] The buffer decompression chains 207-209 in the frame buffer assembly 104 shown in Figure 2 may decompress the current compressively decoded frame in the following manner: The buffer entropy decoding module 207 receives the compressively decoded frame portion 212 previously written to the data memory 201 and now read from the data memory 201. The buffer entropy decoding module 207 applies entropy decoding to the read compressively decoded frame portion 212. Thus, the buffer entropy decoding module 207 may reconstruct a quantized compressed version 211 of the transform decoded frame portion 210 previously provided by the buffer quantization module 205 in the buffer compression chains 203-206.

[0061] The buffer dequantization module 208 applies inverse quantization to the quantized compressed version 211 of the transform decoded frame portion 210. Inverse quantization is an operation that reverses the quantization in the buffer quantization module 205 to roughly regenerate the transform samples of the transform decoded frame portion 210. The buffer dequantization module 208 thus provides a decompressed version 213 of the transform decoded frame portion 210 provided by the buffer decorrelation transform module 203 thus far in the buffer compression chain 203-206. This decompressed version 213 may be considered a coarse representation of the transform decoded frame portion 210.

[0062] The buffer inverse transform module 209 applies a decorrelation inverse transform to a decompressed version 213 of the transformed decoded frame portion 210. The buffer inverse transform module 209 thus provides a decompressed version 214 of the decoded frame portion 123 previously received by the frame buffer assembly 104 shown in FIG. 2, compressed in the buffer compression chains 203-206, and stored in the data memory 201. Due to the lossy compression therein, it follows that the decompressed version 214 of the decoded frame portion 123 does not perfectly correspond to the decoded frame portion 123 transmitted to the frame buffer assembly 104. The greater the extent to which the decoded frame portion 123 is compressed, generally the more lossy the compression will be, and as a result the greater the extent to which the decompressed version 214 differs from the decoded frame portion 123.

[0063] Thus, the frame buffer assembly 104 may provide a decompressed version of each of the decoded frame portions that together constitute a decompressed version of the decoded current frame, or at least a portion thereof, a representation thereof that may serve as a reference for encoding a subsequent frame, at least in part, in inter mode.

[0064] Lossy compression may result in the representation of the decoded current frame constituting the reference not completely corresponding to the decoded current frame, which may degrade the image quality of the decoded version of the encoded video produced by the video encoder 100, especially since errors accumulate during decoding when the decoded version of the frame is used as the reference. However, the video decoder may be able to regenerate the representation constituting the reference in the video encoder 100 due to the indication 124 of the applied lossy compression, thereby preventing the accumulation of errors that may significantly affect the image quality.

[0065] 1, the data packaging module 110 may include an indication 124 of the lossy compression applied by the frame buffer assembly 104 in a data set that further includes the encoded video. The data set may have a predefined structure for including the indication. The structure may provide specific locations where the indication may reside so that a video decoder can easily retrieve the indication.

[0066] Fig. 3 shows a data set 300 that may be provided by the video encoder 100. Fig. 3 provides a schematic structural view of the data set 300. In this example, the data set 300 may be in the form of a data stream. The data set 300 comprises a sequence of encoded frames 102, which is represented diagrammatically in the top part of Fig. 3. Fig. 3 shows several encoded frames 301-305 in the sequence, one of which, 303, is represented and described in more detail. For convenience, this encoded frame 303 will be referred to as the described encoded frame 303 in the following. The other encoded frames in the encoded sequence of frames 102 may have a structure similar to or even identical to the described encoded frame 303.

[0067] It is assumed that the mentioned coded frame 303 results from coding a current frame as described above with reference to figures 1 and 2. It is further assumed that this coding has been performed at least partially in inter mode. The current frame has therefore been coded with reference to one or more other previously coded frames. Any of the other coded frames located to the left of the mentioned coded frame 303 may constitute coded versions of a frame that served as a reference for the coding of the current frame. This applies to the coded frames 301, 302 depicted in figure 3. Any of the other coded frames located to the right of the mentioned coded frame 303 may be coded versions of a frame that was at least partially coded with reference to the current frame. This applies to the coded frames 304, 305 depicted in figure 3.

[0068] The described coded frame 303 includes several data fields for several types of data 306-312, which are depicted in the top-middle part of FIG. 3. These include a picture header 306, a number of coded frame parts 307-311, and a picture end indicator 312. Each coded frame part may have been generated by the video encoder 100 in a manner as described above. In principle, one or more coded frame parts may have been coded in inter mode and one or more other coded frame parts may have been coded in intra mode. It is also possible that all coded frame parts contained in the described coded frame 303 are coded in inter mode or that all coded frame parts are coded in intra mode. A first coded frame part 307 of the described coded frame 303 is depicted and described in more detail. The other coded frame parts 308-311 may have a structure similar or even identical to the first coded frame part 307.

[0069] The first encoded frame portion 307 includes several data fields for several types of data 313-316, which are depicted in the lower-middle portion of FIG. 3. These include a header 313, a mode indicator 314, reference data 315, and encoded picture data 316. The header 313 may record the location of the first encoded frame portion 307 in the described encoded frame 303. The header 313 may include further information, which is described below. The mode indicator 314 indicates whether the first encoded frame portion 307 was generated by encoding in an inter mode or an intra mode.

[0070] If the encoding was in inter mode, the reference data 315 may include the motion vector data mentioned above, which indicates one or more portions in other frames that served as references. If the encoding was in intra mode, the reference data 315 may indicate portions of the current frame itself that were used for this intra encoding. Alternatively, the reference data 315 may be omitted if no reference is made to other portions of the current frame itself. The coded picture data 316 includes a quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118 provided by the coding chains 105-111 in the video encoder 100 as described above with reference to FIG. 1.

[0071] The header 313 of the first encoded frame portion 307 is shown and described in further detail. Other encoded frame portions may include headers having a structure similar or even identical to the header 313 of the first encoded frame portion 307.

[0072] The header 313 includes several data fields for several types of data 317-321, which are depicted in the bottom portion of FIG. 3. These include a marker 317, a flag field 318, an encoding quantization indication 319, a frame buffer compression indication 320, and packet size and control data 321. The marker 317 may record the header 313 in the first encoded frame portion 307. The flag field 318 may include one or more flags that may be used to decode the first encoded frame portion 307. The encoding quantization information 319 may identify one or more quantization steps used in the quantization to obtain the first encoded frame portion 307.

[0073] A particular aspect of the header is the frame buffer compression indication 320 contained therein. The frame buffer compression indication 320 includes an indication 124 of the lossy compression that the frame buffer assembly 104 applied to store a decoded version of the first encoded frame portion 307 in its data memory 201. The frame buffer compression indication 320 thus enables a video decoder to also apply this lossy compression, or at least apply an approximation of it, or recreate the effect of the lossy compression.

[0074] In a variation of the data set 300 shown in Fig. 3, the picture header 306 may include a frame buffer compression indication that at least partially provides an indication 124 of the lossy compression applied by the frame buffer assembly 104. For example, a fixed level of compression or a predefined compression scheme may be applied by the frame buffer assembly 104 to the described encoded frame 303. The frame buffer compression indication in the picture header may then indicate a fixed level of compression or a predefined compression scheme. The frame buffer compression indication 320 in the header 313 of the first encoded frame portion 307 may then be omitted, which applies equally to each header of each other encoded frame portion belonging to the described encoded frame 303. Alternatively, the frame buffer compression indication in the encoded frame portion header may indicate an adjustment with respect to the fixed level of compression or the predefined compression scheme.

[0075] FIG. 4 shows a schematic diagram of a video decoder 400 corresponding to the video encoder 100 described above with reference to FIG. 1 and FIG. 2. FIG. 4 provides a schematic block diagram of the video decoder 400. The video decoder 400 includes a decoding assembly 401 and a frame buffer assembly 402. For distinction, the frame buffer assembly 402 will be referred to as the decoder frame buffer assembly 402 in the following. The decoding assembly 401 includes various functional modules, such as a data depackaging module 403, a motion compensation module 404, an entropy decoding module 405, a dequantization module 406, an inverse transform module 407, and a reconstruction module 408. For distinction, the dequantization module 406, the inverse transform module 407, and the reconstruction module 408 will be referred to as the decoder dequantization module 406, the decoder inverse transform module 407, and the decoder reconstruction module 408, respectively in the following. That is, the adjective "decoder" has been added to distinguish the above-mentioned modules from the corresponding modules in the video encoder 100 shown in FIG.

[0076] The video decoder 400 basically works as follows: It is assumed to receive a sequence of encoded frames 102 contained in a data set 300 as shown in Fig. 3, generated by the video encoder 100 as shown in Fig. 1. In response, the video decoder 400 provides a sequence of decoded frames 409, which comprises each decoded frame, which is a decoded version of an encoded frame in the encoded sequence of frames 102.

[0077] One or more decoded frames are at least temporarily stored in the decoder frame buffer assembly 402. The decoder frame buffer assembly 402 may then provide a representation 410 of the previously decoded frames, which may serve as a reference for decoding a currently encoded frame that is at least partially encoded in inter mode.

[0078] A decoding assembly 401 in the video decoder 400 may decode the first encoded frame portion 307 in a manner that will now be described. The decoding assembly 401 may decode other frame portions in a similar manner.

[0079] The data depackaging module 403 obtains various types of data included in the first coded frame portion 307, such as coded picture data 316, reference data 315, and a frame buffer compression indication 320 included in the header 313 as shown in FIG. 3. The reference data 315 may indicate whether an intra mode or an inter mode has been applied. The reference data 315 includes motion vector data if the first coded frame portion 307 results from coding in an inter mode. The coded picture data 316 includes a quantized and coded version 120 of the transformed intra / inter defined frame portion 118 provided by the coding chains 105-111 in the video encoder 100 as described above with reference to FIG. 1. The frame buffer compression indication 320 includes an indication 124 of the lossy compression applied to the first coded frame portion 307 by the frame buffer assembly 104 in the video encoder 100.

[0080] The entropy decoding module 405 applies entropy decoding to the quantized and coded version 120 of the transformed intra / inter prescribed frame portion 118. The entropy decoding is the inverse transform of the entropy coding applied by the entropy coding module 109 in the video encoder 100. Thus, the entropy decoding module 405 regenerates the quantized version 119 of the transformed intra / inter prescribed frame portion 118 received by the entropy coding module 109 in the video encoder 100. In the video encoder 100, the decoding chains 112-114 also receive the quantized version 119 of the transformed intra / inter prescribed frame portion 118. The decoder dequantization module 406 and the decoder inverse transform module 407 perform operations corresponding to those performed by the dequantization module 112 and the inverse transform module 113 in the decoding chains 112-114 of the video encoder 100, respectively. Thus, the decoder inverse transform module 407 provides a decoded version 122 of the intra / inter prescribed frame portion 117 .

[0081] If the first coded frame portion 307 results from coding in inter mode, the motion compensation module 404 receives motion vector data included in the reference data 315. The motion vector data indicates a particular portion of the previously decoded frame that served as a reference for coding in inter mode. The motion compensation module 404 obtains this particular portion from a representation of the previously decoded frame 410 provided by the decoder frame buffer assembly 402. The decoder reconstruction module 408 adds the decoded version 122 of the intra / inter prescribed frame portion 117 to the particular portion of the representation of the previously decoded frame that the motion compensation module 404 obtained from the decoder frame buffer assembly 402. This addition generates a decoded frame portion 411.

[0082] Ideally, the representation 410 of the previously decoded frame provided by the decoder frame buffer assembly 402 corresponds closely to the representation 115 provided by the frame buffer assembly 104 in the video encoder 100. That is, ideally, the video decoder 400 uses references that correspond closely to the references used in the video encoder 100. In that case, the decoded frame portion 411 provided by the decoder reconstruction module 408 corresponds closely to the decoded frame portion 123 provided by the reconstruction module 114 in the video encoder 100 for that frame portion.

[0083] It is now assumed that the reference used by the video decoder 400 for the corresponding frame portion is different from the reference used by the video encoder 100. In that case, the decoded frame portion 411 provided by the decoder reconstruction module 408 is different from the decoded frame portion 123 provided by the reconstruction module 114 in the video encoder 100 for the corresponding frame portion. The latter decoded frame portion 123 may even have served as a reference for encoding a subsequent frame portion belonging to a subsequent frame in inter mode. This generally leads to a larger difference between the reference used by the video decoder 400 and the reference used by the video encoder 100 for this subsequent frame portion belonging to a subsequent frame. Thus, the difference between the reference used in the video encoder 100 and the reference used in the video decoder 400 may increase over successive frames. This leads to an accumulation of errors that degrades the image quality.

[0084] The frame buffer compression representation 320 enables the decoder frame buffer assembly 402 to provide a reference that closely matches the reference provided by the frame buffer assembly 104 in the video encoder 100. The representation may further enable the decoder frame buffer assembly 402 to closely reproduce the reference provided by the frame buffer assembly 104 in the video encoder 100. Thus, the frame buffer compression representation 320 may reduce and further prevent the accumulation of errors that may occur due to data compression in the frame buffer assembly 104 of the video encoder 100. The decoder frame buffer assembly 402 may apply the same data compression to at least temporarily store the decoded frames. Alternatively, the decoder frame buffer assembly 402 may store the decoded frames uncompressed, but process the decoded frames in a manner that mimics the data compression applied in the frame buffer assembly 104 of the video encoder 100.

[0085] Figure 5 illustrates a schematic block diagram of an embodiment of the decoder frame buffer assembly 402. Figure 5 provides a schematic block diagram of this embodiment, which for convenience will be referred to hereinafter as the decoder frame buffer assembly 402. The decoder frame buffer assembly 402 includes a data memory 501 and a controller 502, which for distinction will be referred to hereinafter as the decoder buffer data memory 501 and the decoder buffer controller 502, respectively.

[0086] Like the frame buffer assembly 104 in the video encoder 100, the decoder frame buffer assembly 402 includes various functional modules that may be considered as constituting a buffer compression chain 503-505. These functional modules include a decorrelation module 503, a quantization module 504, and an entropy coding module 505, which will be referred to hereinafter as the decoder buffer decorrelation module 503, the decoder buffer quantization module 504, and the decoder buffer entropy coding module 505, respectively, for the same reason. For the same reason, the buffer compression chains 503-505 in the decoder frame buffer assembly 402 will be referred to hereinafter as the decoder buffer compression chains 503-505.

[0087] The decoder frame buffer assembly 402 further includes functional modules, such as an entropy decoding module 506, a dequantization module 507, and an inverse transform module 508, which may be considered as constituting a buffer decompression chain 506-508, which, for the sake of distinction, will be referred to hereinafter as a decoder buffer entropy decoding 506, a decoder buffer dequantization module 507, and a decoder buffer inverse transform module 508, respectively. For the same reason, the buffer decompression chains 506-508 in the decoder frame buffer assembly 402 will be referred to hereinafter as decoder buffer decompression chains 506-508. The functional modules in the decoder buffer compression chains 503-505 and the decoder buffer decompression chains 506-508 may, for example, each be implemented in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processor, or any combination thereof.

[0088] The decoder frame buffer assembly 402 basically operates as follows: It is assumed that it receives a decoded frame portion 411 provided by a decoder reconstruction module 408 as described above with reference to Figure 4. Other decoded frame portions resulting from decoding other encoded frame portions may be processed in a similar manner as described below.

[0089] The decoder buffer controller 502 receives the frame buffer compression indication 320 included in the first encoded frame portion 307 as shown in FIG. 3. The decoder buffer controller 502 controls the decoder buffer quantization module 504 to apply quantization similar to that in the buffer quantization module 205 of the frame buffer assembly 104 shown in FIG. 2. If the frame buffer compression indication 320 identifies one or more quantization steps applied, the decoder buffer quantization module 504 may also apply these quantization steps directly. As mentioned above, the frame buffer compression indication 320 may indicate a target resulting from lossy compression in the frame buffer assembly 104 of the video encoder 100. The decoder buffer controller 502 then uses this target to control the quantization in the decoder buffer quantization module 504 to reproduce or mimic the quantization in the buffer quantization module 205 of the frame buffer assembly 104 of the video encoder 100.

[0090] The decoder buffer decorrelation transform module 503 and the decoder buffer entropy coding module 505 may perform operations similar to those performed by the buffer decorrelation transform module 203 and the buffer entropy coding module 206, respectively, in the frame buffer assembly 104 of the video encoder 100. Thus, the decoder buffer decorrelation transform module 503 may apply the decorrelation transform applied by the buffer decorrelation transform module 203. The decoder buffer decorrelation transform module 503 applies this decorrelation transform to the decoded frame portion 411 provided by the decoder reconstruction module 408 to obtain a transformed decoded frame portion 509. The transformed decoded frame portion 509 is then quantized in the decoder buffer quantization module 504 as described above.

[0091] The decoder buffer entropy coding module 505 applies entropy coding to a quantized, compressed version 510 of the transform decoded frame portion 509 provided by the decoder buffer quantization module 504. This entropy coding may be similar to that applied by the buffer entropy coding module 206 in the frame buffer assembly 104 shown in FIG. 2, or a different entropy coding may be applied. Thus, the decoder buffer entropy coding module 505 provides a quantized and encoded compressed version 511 of the transform decoded frame portion 509 provided by the decoder buffer decorrelation module 503.

[0092] The quantized and encoded compressed version 511 of the transform decoded frame portion 509 provided by the decoder buffer compression chains 503-505 corresponds closely, or even more closely, to the compressed decoded frame portion 212 provided by the buffer compression chains 203-206 in the frame buffer assembly 104 of the video encoder 100. This is due to the frame buffer compressed representation 320 allowing the quantization in the decoder buffer compression chains 503-505 to correspond closely, or even more closely, to the quantization applied by the buffer compression chains 203-206 in the frame buffer assembly 104 of the video encoder 100. Thus, the quantized and encoded compressed version 511 of the transform decoded frame portion 509 provided by the decoder buffer compression chains 503-505 will be referred to as the similar compressed decoded frame portion 511 in the following. The similar compressed decoded frame portion 511 is transmitted to the decoder buffer data memory 501 and stored therein.

[0093] The decoder buffer decompression chains 506-508 in the decoder frame buffer assembly 402 shown in Figure 5 may perform operations similar to those performed by the buffer decompression chains 207-209 in the frame buffer assembly 104 shown in Figure 2. Thus, the decoder frame buffer assembly 402 may provide a decompressed version 512 of the decoded frame portion 411 provided by the decoder reconstruction module 408 shown in Figure 4 and compressed in the decoder buffer compression chains 503-505 and stored in the decoder buffer data memory 501.

[0094] The decompressed version 512 of the decoded frame portion 411 corresponds closely, or even more closely, to the decompressed version 214 of the decoded frame portion 123 provided by the frame buffer assembly 104 shown in Figure 2. Again, this is due to the frame buffer compressed representation 320 enabling quantization and dequantization in the decoder frame buffer assembly 402 to correspond closely, or even more closely, to the quantization and dequantization in the frame buffer assembly 104 of the video encoder 100.

[0095] Thus, the decoder frame buffer assembly 402 may provide a decompressed version of each of the previously decoded frame portions that together constitute a decompressed version of the previously decoded frame, which constitutes a representation 410 of the previously decoded frame that may serve at least in part as a reference for decoding a currently encoded frame that was at least in part encoded in inter mode, which may correspond closely, or even more closely, to a reference used in the video encoder 100 to generate the currently encoded frame.

[0096] The above-described embodiment is an example of a frame buffer compressed representation 320 that allows the references in the video decoder 400 to correspond to those used in the video encoder 100. This allows the references to be compressed to a relatively large extent while still maintaining satisfactory image quality, at least in the video encoder 100. This results in a reduction in the amount of data that needs to be temporarily stored, and in the bandwidth required to write and read the data for temporary storage, which in turn results in reduced cost and power consumption.

[0097] As mentioned above, the frame buffer compression indication 320 may be of a direct nature, specifying the quantization resolution applied by the frame buffer assembly 104 in the video encoder 100. Thus, the frame buffer compression indication 320 may include, for example, one or more quantization steps applied by the buffer quantization module 205 in quantizing the corresponding transform decoded frame portion. The advantage of such a direct specification of the quantization resolution in the frame buffer compression is that the decoder frame buffer assembly 402 does not need to derive the quantization resolution from an indirect indication of compression, such as, for example, the compression target as mentioned above. Thus, the direct specification of the quantization resolution allows the decoder frame buffer assembly 402 to be implemented relatively simply, and therefore the video decoder 400 to be implemented cost-effectively. This also applies to the different embodiments from those described above with reference to Figures 1, 2, 4, and 5.

[0098] FIG. 6 illustrates an alternative video encoder 600. FIG. 6 provides a schematic block diagram of the alternative video encoder 600. Like the video encoder 100 illustrated in FIG. 1, the alternative video encoder 600 may encode a sequence of frames 601 that may constitute a video to obtain a sequence of encoded frames 602. The sequence of encoded frames 602 may thus constitute an encoded video that includes a smaller amount of data than the original form of the video. The alternative video encoder 600 may therefore also be able to efficiently store the video by encoding it, or to efficiently transmit the video, or both. The alternative video encoder 600 may be part of a device capable of capturing video, such as a smartphone, and may add value to the device due to its efficiency.

[0099] The alternative video encoder 600 includes an alternative encoding assembly 603 and an alternative frame buffer assembly 604. The alternative encoding assembly 603 includes various functional modules, such as a linear transform module 605, an alternative intra / inter mode adaptation module 606, a quantization module 607, an entropy coding module 608, a data packaging module 609, and a controller 610. The above-mentioned modules may perform encoding-related operations and may therefore be considered to form an encoding chain 605-610. The alternative encoding assembly 603 further includes functional modules, such as a dequantization module 611 and an alternative reconstruction module 612, which perform decoding-related operations and may therefore be considered to form a decoding chain 611-612. The functional modules of the alternative encoding assembly 603 may be implemented, for example, in the form of dedicated circuits, programmable circuits, or suitably programmed processors, or any combination thereof, respectively.

[0100] The alternative video coder 600 basically operates as follows. The alternative coding assembly 603 may operate in a similar manner to that described in WO 2020 / 249790. In summary, the linear transformation module 605 applies a linear transformation to the current frame to be coded. The linear transformation thus provides the current frame 613 in the transform domain. The linear transformation may be, for example, a decorrelation transformation such as the discrete cosine transform (commonly referred to by the acronym DCT) or a wavelet transform. The linear transformation may also be a combination of such a transformation with a color transformation, for example transforming a frame composed of color pixels such as red, green and blue into a frame composed of a luminance component and two chrominance components. The color transformation is generally performed first.

[0101] In practice, the alternative encoding assembly 603 may divide the current frame in the transform domain 613 into blocks of transform samples, which constitute portions of the current frame in the transform domain 613, which have specific positions within the current frame in the transform domain 613. For convenience, such portions of the current frame in the transform domain 613, as well as such portions of other frames in the transform domain, will be referred to as transform domain frame portions in the following. The alternative encoding assembly 603 may individually encode each transform domain frame portion that together constitutes the current frame in the transform domain 613 in the following manner.

[0102] The transform domain frame portion may be coded in an intra mode or an inter mode, which are essentially similar to the intra mode and inter mode, respectively, described above with respect to the video encoder 100 shown in FIG. 1. In the inter mode, the transform domain frame portion is coded with reference to a representation 614 of a previously coded frame in the transform domain. The alternative frame buffer assembly 604 provides a representation 614 of a previously coded frame 613 in the transform domain. In the intra mode, the transform domain frame portion is coded without reference to other frames. In this mode, the transform domain frame portion may be coded "as is".

[0103] The controller 610 may decide whether the transform domain frame portion is coded in intra mode or in inter mode. The above-mentioned WO 2020 / 249790 describes techniques for making this decision. Essentially, the decision is based on two criteria. The first criterion is whether intra mode or inter mode is expected to be more advantageous, for example in terms of data compression rate, or image quality, or both. The second criterion is applied when inter mode is expected to be more advantageous. The second criterion concerns the respective corresponding transform domain frame portion belonging to the immediately preceding frame. The adjective "corresponding" denotes a transform domain frame portion having a position in the preceding frame in the transform domain that corresponds to the position of the relevant transform domain frame portion in the current frame 613 in the transform domain.

[0104] The second criterion is that all of the corresponding transform domain frame parts belonging to the immediately preceding frame are coded in inter mode and the number of them reaches a threshold. If the second criterion is not met, the decision is based on the first criterion described above. However, if the second criterion is met, the transform domain parts are coded in intra mode regardless of the first criterion. That is, if the second criterion is met even when the inter mode is expected to be more favorable, the intra mode is forced to be applied.

[0105] In effect, the second criterion constitutes a refresh mechanism, which ensures a reasonable delay between when the decoding of the encoded video starts and when it starts to generate properly decoded video. Such a refresh mechanism may also be applied to video encoders other than the alternative video encoder 600 shown in Figure 6. For example, the controller 610 of the video encoder 100 shown in Figure 1 may apply a refresh mechanism as described above.

[0106] An alternative intra / inter mode adaptation module 606 provides an intra / inter prescriptive transform domain frame portion 615, which is applied to the quantization module 607. In intra mode, the intra / inter prescriptive transform domain frame portion 615 may simply correspond to a transform domain frame portion that forms part of the current frame 613 in the transform domain. In that case, the intra / inter prescriptive transform domain frame portion 615 includes transform samples that are also present in the current frame 613 in the transform domain.

[0107] In inter mode, the intra / inter specified transform domain frame portion 615 is a differential representation of the transform domain frame portion. The differential representation corresponds to the difference between, on the one hand, the transform domain frame portion belonging to the current frame and, on the other hand, the corresponding portion in the representation 614 of the previously coded frame in the transform domain provided by the alternative frame buffer assembly 604. Here again, the adjective "corresponding" indicates a positional correspondence. The differential representation of the transform domain frame portion includes a residual representing said difference in the transform domain.

[0108] The quantization module 607 and the entropy coding module 608 together may be considered to form a data compression module that compresses data included in the intra / inter prescriptive transform domain frame portion 615. The quantization module 607 and the entropy coding module 608 may operate in a manner similar to that described above with respect to the quantization module 108 and the entropy coding module 109, respectively, in the video encoder 100 shown in FIG. 1. The controller 610 may control the quantization module 607 and the entropy coding module 608 in a manner similar to that described above with respect to the controller 111 in the video encoder 100 shown in FIG. 1. Thus, the entropy coding module 608 provides a quantized and coded version 616 of the intra / inter prescriptive transform domain frame portion 615, which may include an amount of data that may be close to a target amount of data.

[0109] The data packaging module 609 may operate in a manner similar to that described above with respect to the data packaging module 110 in the video encoder 100 shown in FIG. 1. Thus, the data packaging module 609 may provide a coding transform domain frame portion including a quantized and coded version 616 of the intra / inter prescribed transform domain frame portion 615 obtained as described above. The coding transform domain frame portion may further include a mode indication indicating whether the corresponding transform domain frame portion was coded in intra mode or in inter mode. The coding transform domain frame portion may still further include information about how the intra / inter prescribed transform domain frame portion 615 was generated. This information may include, for example, information about the resolution of the applied quantization.

[0110] As mentioned above, the alternative video coder 600 shown in Fig. 6 comprises the above mentioned decoding chains 611-612, which include a dequantization module 611 and an alternative reconstruction module 612. The decoding chains 611-612 are capable of obtaining a decoded version of a current frame 613 in the transform domain, which was coded in the alternative video coder 600. The decoding chains 611-612 basically operate as follows.

[0111] The dequantization module 611 applies inverse quantization to a quantized version 617 of the intra / inter prescribed transform domain frame portion 615 provided by a quantization module 607 in the coding chain 605-610. Inverse quantization is an operation that reverses the quantization in the quantization module 607. Thus, the dequantization module 611 provides a dequantized version 618 of the intra / inter prescribed transform domain frame portion 615. This dequantized version 618 may be considered as a coarse representation of the intra / inter prescribed transform domain frame portion 615 provided by an alternative intra / inter mode application module 606 in the coding chain 605-610.

[0112] The alternative reconstruction module 612 provides a decoded version 619 of the transform domain frame portion belonging to the current frame, coded in the coding chains 605-610 as described above. If an inter mode is applied to the transform domain frame portion, the alternative reconstruction module 612 adds a dequantized version 618 of the intra / inter prescribed transform domain frame portion 615 to the corresponding portion of the representation 614 of the previously coded frame in the transform domain that served as a reference. This addition generates a decoded version 619 of the transform domain frame portion, which for convenience will be referred to as the decoded transform domain frame portion 619 in the following. If an intra mode is applied to the transform domain frame portion, the alternative reconstruction module 612 may simply take the dequantized version 618 of the intra / inter prescribed transform domain frame portion 615 as the decoded transform domain frame portion 619.

[0113] The alternative frame buffer assembly 604 receives the decoded transform domain frame portion 619 provided by the alternative reconstruction module 612 at the end of the decoding chain 611-612. Thus, when the current frame is coded in its entirety, the alternative frame buffer assembly 604 receives the respective decoded transform domain frame portions, all of which may be obtained as described above. Thus, the alternative frame buffer assembly 604 receives a decoded version of the current frame 613 in the transform domain, which for convenience will be called the decoded current frame in the transform domain. The alternative frame buffer assembly 604 can then provide a representation of the decoded current frame in the transform domain when a subsequent frame is coded at least partially in inter mode. This representation can then serve as a reference for coding one or more transform domain frame portions belonging to this subsequent frame in inter mode.

[0114] Like the frame buffer assembly 104 in the video encoder 100 shown in FIG. 1, the alternative frame buffer assembly 604 applies a lossy compression. In the alternative video encoder 600, a lossy compression is applied to the decoded current frame in the transform domain, which provides a compressed version of the decoded current frame in the transform domain. The alternative frame buffer assembly 604 at least temporarily stores this compressed version. The compressed version of the decoded current frame in the transform domain may then be later decompressed to provide a decompressed version of the decoded current frame in the transform domain. This decompressed version then constitutes a representation of the current frame in the transform domain 613 that may be used as a reference for encoding one or more transform domain frame portions belonging to a subsequent frame in inter mode.

[0115] 1, the alternative frame buffer assembly 604 provides an indication 620 of the lossy compression that has been applied. The data packaging module 609 may include this indication 620 in a data set that includes the current encoded frame and other encoded frames that form the encoded video.

[0116] FIG. 7 illustrates an alternative frame buffer assembly 604 in an alternative video encoder 600. FIG. 7 provides a schematic block diagram of the alternative frame buffer assembly 604 in the alternative video encoder 600. The alternative frame buffer assembly 604 includes a data memory 701, a controller 702, and various functional modules 703-707. These functional modules include a delay buffer 703, a quantization module 704, and an entropy coding module 705. The above-mentioned functional modules may be considered to constitute a buffer compression chain 703-705. This buffer compression chain 703-705 is simpler than that in the frame buffer assembly 104 shown in FIG. 2 in the sense that the buffer compression chain 703-705 in the alternative frame buffer assembly 604 does not need to include a decorrelation transformation module.

[0117] The alternative frame buffer assembly 604 further includes functional modules, such as an entropy decoding module 706 and a dequantization module 707, which may be considered as constituting a buffer decompression chain 706-707. The buffer decompression chain 706-707 is simpler than that in the frame buffer assembly 104 shown in FIG. 2 in the sense that the buffer decompression chain 706-707 in the alternative frame buffer assembly 604 does not need to include an inverse transform module. This is because the alternative video encoder 600 applies an inter mode in the transform domain. Thus, the alternative frame buffer assembly 604 is simpler but still obtains a buffer compression efficiency similar to that of the frame buffer assembly 104 in the video encoder 100. The functional modules of the alternative frame buffer assembly 604 may be implemented, for example, in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processor, or any combination thereof, respectively.

[0118] For the sake of distinction, the controller 702 in the alternative frame buffer assembly 604 will hereinafter be referred to as the buffer controller 702. For the same reason, the quantization module 704, the entropy encoding module 705, the entropy decoding module 706, and the dequantization module 707 will hereinafter be referred to as the buffer quantization module 704, the buffer entropy encoding module 705, and the buffer dequantization module 707, respectively. Thus, for the sake of distinction, the adjective "buffer" has been added.

[0119] As with the frame buffer assembly shown in FIG. 2, the functional modules of the alternative frame buffer assembly 604 may be part of an integrated circuit that may further include the alternative encoding assembly 603 of the alternative video encoder 600 shown in FIG. 6. However, the data memory 701 of the alternative frame buffer assembly 604 may be in the form of a separate circuit, such as, for example, a dynamic random access memory circuit, commonly referred to by the acronym DRAM. Such an embodiment may also include a data write / read channel, through which data may be written to the data memory 701 and data may be read from the data memory 701. The data write / read channel may include a bus, having a bandwidth that sets a limit on the amount of data per unit time that may be written to and read from the data memory 701. That is, the bandwidth defines the maximum data rate of writing and reading.

[0120] The alternative frame buffer assembly 604 basically operates as follows: It is assumed that it receives decoded transform domain frame portions 619 provided by the alternative reconstruction module 612 as described above. Other decoded frame portions in the transform domain, which may belong to the current frame and other frames, may be processed in a similar manner as described below.

[0121] The buffer compression chains 703-705 in the alternative frame buffer assembly 604 may operate in a manner similar to that described above with respect to the delay buffer 204, the buffer quantization module 205, and the buffer entropy coding module 206 in the compression chains 203-206 of the frame buffer assembly 104 shown in FIG. 2. Thus, the buffer compression chains 703-705 provide a compressed decoded transform domain frame portion 708. The buffer controller 702 may control the buffer quantization module 704 in a manner similar to that described above with respect to the buffer controller 202 in the frame buffer assembly 104 shown in FIG. 2. Thus, the buffer controller 702 may provide to the decoded transform domain frame portion 619 an indication 620 of the lossy compression applied to obtain its compressed version.

[0122] Once the current frame is encoded, the buffer compression chains 703-705 compress the respective decoded frame portions in the transform domain which constitute the decoded current frame in the transform domain. Thus, the buffer compression chains 703-705 generate respective compressed decoded frame portions in the transform domain which are written to the data memory 701. Each compressed decoded frame portion constitutes a compressed version of the decoded current frame in the transform domain. This compressed version will be referred to in the following as the compressed decoded current frame in the transform domain. Thus, the compressed decoded current frame in the transform domain results from a lossy compression in the buffer compression chains 703-705 described above.

[0123] The buffer decompression chains 706-707 in the alternative frame buffer assembly 604 may decompress the compressed decoded current frame in the transform domain to provide a representation of the current frame in the transform domain that may be used as a reference to encode one or more transform domain frame portions belonging to a subsequent frame in inter mode. The buffer decompression chains 706-707 may operate in a manner similar to that described above with respect to the buffer entropy decoding module 207 and the buffer dequantization module 208 in the frame buffer assembly 104 shown in FIG. 2. Thus, the buffer decompression chains 706-707 may provide the decompressed decoded transform domain frame portion 709 based on the compressed decoded transform domain frame portion 708 stored in the data memory 701.

[0124] As shown in Figure 6, the data packaging module 609 may include an indication 620 of the lossy compression applied by the alternative frame buffer assembly 604 in a data set that further includes the encoded video. The data set may have a predefined structure for including the indication 620, which for convenience will be referred to hereinafter as the frame buffer compressed indication 620. The structure of the data set may be similar to that of the data set 300 shown in Figure 3. The structure provides one or more specific locations where the frame buffer compressed indication 620 may reside, such that the indication can be easily obtained by a video decoder.

[0125] Fig. 8 shows an alternative video decoder 800 corresponding to the alternative video encoder 600 shown in Fig. 6. Fig. 8 provides a schematic block diagram of the alternative video decoder 800. The alternative video decoder 800 includes an alternative decoding assembly 801 and an alternative frame buffer assembly 802. For distinction, the alternative frame buffer assembly 802 will be referred to as the alternative decoder frame buffer assembly 802 in the following. The alternative decoding assembly 801 includes various functional modules, such as a data depackaging module 803, an entropy decoding module 804, a dequantization module 805, an alternative reconstruction module 806, and an inverse transform module 807. For the sake of distinction, the entropy decoding module 804, the dequantization module 805 and the alternative reconstruction module 806 will be referred to below as the decoder entropy decoding module 804, the decoder dequantization module 805 and the alternative decoder reconstruction module 806, respectively, i.e. the adjective "decoder" has been added to distinguish the above mentioned modules from the corresponding modules in the alternative video coder 600 shown in FIG.

[0126] The alternative video decoder 800 basically operates as follows: It is assumed that the alternative video decoder 800 receives a sequence of encoded frames 602 generated by the alternative video encoder 600 of Fig. 6. In response, the alternative video decoder 800 provides a sequence of decoded frames 808, which comprises respective decoded frames, which are decoded versions of the encoded frames generated by the alternative video encoder 600 of Fig. 6. In providing the decoded frames, the alternative video decoder 800 also internally generates decoded frames in the transform domain.

[0127] The alternative decoder frame buffer assembly 802 at least temporarily stores one or more decoded frames in the transform domain, and may then provide a representation 809 of a previously decoded frame in the transform domain, which may serve as a reference for decoding a current encoded frame that is at least partially encoded in inter mode.

[0128] The alternative video decoder 800 may decode an encoded frame by successively decoding each encoded transform domain frame portion that constitutes the encoded frame, each encoded transform domain frame portion resulting from encoding a respective transform domain frame portion in the alternative video encoder 600 as described above with reference to Figures 6 and 7.

[0129] The alternative decoding assembly 801 may decode the encoded transform domain frame portion in the following manner: The data depackaging module 803 obtains various types of data from the encoded transform domain frame portion. Thus, the data depackaging module 803 provides a quantized and encoded version 810 of the intra / inter rule transform domain frame included in the encoded transform domain frame portion. The data depackaging module 803 further provides a mode indication 811 and a frame buffer compression indication 812. The frame buffer compression indication 812 may correspond to the frame buffer compression indication 620 described above with respect to the alternative video encoder 600 shown in FIG. 6, and more particularly with respect to the alternative frame buffer assembly 604 shown in FIG. 7.

[0130] The decoder entropy decoding module 804 applies entropy decoding to the quantized and coded version 810 of the intra / inter prescribed transform domain frame. The entropy decoding is the inverse of the entropy coding applied by the entropy coding module 608 in the alternative video encoder 600. Thus, the decoder entropy decoding module 804 regenerates a quantized version 813 of the intra / inter prescribed transform domain frame. This quantized version 813 may correspond to the quantized version 617 of the intra / inter prescribed transform domain frame portion 615 in the alternative video encoder 600 shown in FIG. 6. The decoder dequantization module 805 performs operations corresponding to those performed by the dequantization module 611 in the decoding chain 611-612 of the alternative video encoder 600. Thus, the decoder dequantization module 805 provides a dequantized version 814 of the intra / inter prescribed transform domain frame portion.

[0131] The alternative decoder reconstruction module 806 provides a decoded transform domain frame portion 815 based on a dequantized version of the intra / inter prescribed transform domain frame portion 814. If an intra mode is applied to generate the coded transform domain frame portion, the alternative decoder reconstruction module 806 may simply take the dequantized version of the intra / inter prescribed transform domain frame portion 814 as the decoded transform domain frame portion 815. If an inter mode is applied to the transform domain frame portion, the alternative decoder reconstruction module 806 adds the dequantized version of the intra / inter prescribed transform domain frame portion 814 to the corresponding portion in the representation 809 of the previously decoded frame in the transform domain that served as a reference. This addition then generates the decoded transform domain frame portion 815.

[0132] Thus, by decoding the respective transform domain frame portions of the coded frame as described above, the alternative video decoder 800 generates a decoded frame in the transform domain. An inverse transform module 807 can then apply an inverse transform to the decoded frame in the transform domain to obtain a fully decoded frame forming part of the sequence of decoded frames 808. The inverse transform corresponds to the inverse of the linear transform performed by the linear transform module 605 in the alternative video encoder 600 described above with reference to Fig. 6. And by decoding the respective transform domain frame portions of the coded frame, respective decoded transform domain frame portions are obtained. These latter portions together constitute a decoded frame in the transform domain.

[0133] The alternative decoder frame buffer assembly 802 may provide a representation of the decoded frames in the transform domain that may then serve as a reference. Ideally, this representation corresponds to the representation provided by the alternative frame buffer assembly 604 in the alternative video encoder 600 shown in FIG. 6. A satisfactory and even perfect correspondence may be obtained between the aforementioned representations that serve as references. This is due to the indication 620 of the lossy compression applied by the alternative frame buffer assembly 604 and included in one or more frame buffer compression representations of the encoded sequence of frames 602.

[0134] Figure 9 illustrates an embodiment of an alternative decoder frame buffer assembly 802 in an alternative video decoder 800. Figure 9 provides a schematic block diagram of this embodiment, which for convenience will be referred to hereinafter as the alternative decoder frame buffer assembly 802. The alternative decoder frame buffer assembly 802 includes a data memory 901 and a controller 902, which for distinguishing purposes will be referred to hereinafter as the decoder buffer data memory 901 and the decoder buffer controller 902, respectively.

[0135] Like the alternative frame buffer assembly 604 in the alternative video encoder 600, the alternative decoder frame buffer assembly 802 includes various functional modules that may be considered to constitute a buffer compression chain 903-904. These functional modules include a quantization module 903 and an entropy coding module 904, which, for the sake of distinction, will be referred to hereinafter as the decoder buffer quantization module 903 and the decoder buffer entropy coding module 904, respectively. For the same reasons, the buffer compression chains 903-904 in the alternative decoder frame buffer assembly 802 will be referred to hereinafter as the decoder buffer compression chains 903-904.

[0136] The alternative decoder frame buffer assembly 802 further includes functional modules that may be considered as constituting a buffer decompression chain 905-906, such as a decoder entropy decoding module 905 and a dequantization module 906, which, for the sake of distinction, will be referred to hereinafter as a decoder buffer entropy encoding module 905 and a decoder buffer dequantization module 905, respectively. For the same reason, the buffer decompression chain 905-906 in the alternative decoder frame buffer assembly 802 will be referred to hereinafter as a decoder buffer decompression chain 905-906. The functional modules in the decoder buffer compression chain 903-904 and the decoder buffer decompression chain 905-906 may, for example, each be implemented in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processor, or any combination of these.

[0137] The alternative decoder frame buffer assembly 802 basically operates as follows: It is assumed that it receives decoded transform domain frame portions 815 provided by the alternative decoder reconstruction module 806 as described above with reference to Figure 8. Other decoded transform domain frame portions resulting from decoding other encoded transform domain frame portions may be processed in a similar manner as described below.

[0138] The decoder buffer controller 902 receives the frame buffer compressed representation 812 contained in the transcoded domain frame portion as described above. The decoder buffer controller 902 controls the decoder buffer quantization module 903 to apply quantization similar to that in the buffer quantization module 704 of the alternative frame buffer assembly 604 shown in Figure 7. This control may be similar to that described above with respect to the buffer controller 702 in the alternative frame buffer assembly 604.

[0139] The decoder buffer entropy coding module 904 applies entropy coding to the quantized, compressed version 907 of the decoded transform domain frame portion 815 provided by the decoder buffer quantization module 903. This entropy coding may be similar to that applied by the buffer entropy coding module 705 in the alternative frame buffer assembly 604 shown in FIG. 5, or a different entropy coding may be applied. Thus, the decoder buffer entropy coding module 904 provides a quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 received by the alternative decoder frame buffer assembly 802.

[0140] The quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 provided by the decoder buffer compression chains 903-904 corresponds closely, or even more closely, to the compressed decoded transform domain frame portion 708 provided by the buffer compression chains 703-705 in the alternative frame buffer assembly 604 of the alternative video encoder 600. This is due to the frame buffer compressed representation 812 allowing the quantization in the decoder buffer compression chains 903-904 to correspond closely, or even more closely, to the quantization applied in the buffer compression chains 703-705 in the alternative frame buffer assembly 604 of the alternative video encoder 600. Thus, the quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 provided by the decoder buffer compression chains 903-904 will hereinafter be referred to as the similar compressed decoded transform domain frame portion 908. The similarly compressed decoded transform domain frame portion 908 is sent to the decoder buffer data memory 901 and stored therein.

[0141] 7. The decoder buffer decompression chains 905-906 in the alternative decoder frame buffer assembly 802 shown in FIG. 9 may perform operations similar to those performed by the buffer decompression chains 706-707 in the alternative frame buffer assembly 604 shown in FIG. 7. The alternative decoder frame buffer assembly 802 may thus provide similar decompression decoded transform domain frame portions 909 based on similar compression decoded transform domain frame portions 908 stored in the data memory 901. The similar decompression decoded transform domain frame portions 909 correspond closely, or even more closely, to the decompression decoded transform domain frame portions 709 provided by the alternative frame buffer assembly 604 shown in FIG. 7. Again, this is because of the frame buffer compressed representation 812 that enables quantization and dequantization in the alternative decoder frame buffer assembly 802 to correspond closely, or even more closely, to the quantization and dequantization in the alternative frame buffer assembly 604 of the alternative video encoder 600.

[0142] Thus, the alternative decoder frame buffer assembly 802 may provide respective similar decompressed decoded transform domain frame portions that together constitute a decompressed version of the previously decoded current frame in the transform domain, which constitutes a representation 809 of the previously decoded frame in the transform domain that may at least partially serve as a reference for decoding the current encoded frame that was at least partially encoded in inter mode, which may correspond closely or even more closely to the reference used in the alternative video encoder 600 to generate the current encoded frame.

[0143] The above described embodiment is another example of a frame buffer compressed representation 812 that allows the references in the alternative video decoder 800 to correspond to those used in the alternative video encoder 600. This allows the references to be compressed to a relatively large extent while still maintaining satisfactory image quality, at least in the alternative video encoder 600. This results in a reduction in the amount of data that needs to be temporarily stored, and in the bandwidth required to write and read the data to the temporary storage, which in turn results in reduced cost and power consumption.

[0144] remarks The embodiments described above with reference to the drawings are given by way of example only, the invention can be embodied in many different ways, and to illustrate this, some alternatives are simply shown.

[0145] There are many different ways to implement lossy compression in the frame buffer assembly of the video encoder of the present invention. In the above-mentioned embodiment, the lossy compression is complemented with entropy encoding and decoding. In other embodiments, the entropy encoding and decoding may be omitted. In the above-mentioned embodiment, the encoding chain in the frame buffer assembly includes a delay buffer. In other embodiments, the encoding chain may not include a delay buffer. There are also various ways to implement lossy compression. For example, lossy compression can be obtained by removing one or more least significant bits. This removal, or truncation, may be included in the GCLI encoding, for example. Lossy compression may also be applied to blocks of pixels or transform samples that do not necessarily correspond in size to the blocks of pixels or transform samples used for video encoding and decoding.

[0146] There are many different ways to provide an indication of the lossy compression applied to the frame buffer assembly. In a very basic embodiment, the indication may be in the form of a flag indicating, for example, whether lossy compression has been applied or not. The lossy compression may be predefined, and the flag may enable the decoder to apply this predefined lossy compression or its equivalent. In other embodiments, where the quantization involves truncation of binary sample values, the indication of the lossy compression may include respective truncation levels for respective portions in the decoded version of the previously encoded frame. In yet other embodiments, the indication may refer to information of the quantization applied to encode the frame or frame portion. That is, the lossy compression in the frame buffer assembly may be associated with said quantization, for example according to a predefined scheme or rule. In still other embodiments, the indication may be in the form of a profile that specifies, for example, the maximum memory bandwidth used for the frame buffer compression, or one or more memory bandwidths for the frame buffer compression. Such a profile may be predefined, allowing for a relatively simple indication.

[0147] There are many different ways to implement a decoder corresponding to the video encoder of the present invention. Such a decoder does not necessarily store the decoded frames serving as references in compressed form. The decoded frames may be stored "as is" in uncompressed form. When using the decoded frames as references, it is sufficient to apply to the decoded frames a process that mimics the lossy compression applied to the decoded frames in the video encoder. For example, the video decoder may be implemented by a software program executed in a computer having a relatively large memory capacity and a relatively large memory bandwidth that allows uncompressed storage of the decoded frames. In such an implementation, the software program may also provide the above-mentioned process based on an indication of the lossy compression applied in the video encoder, thus ensuring that the references used in the video encoder and the video decoder are the same or at least sufficiently similar.

[0148] There are many different ways to code a frame or frame portion in inter mode. For example, several previously coded frames may serve as references. In one embodiment, different frame portions of the same frame may be coded with reference to frame portions of different frames in a series of previously coded frames, each of which may serve as a reference. In another embodiment, an interpolation between two or more previously coded frames may serve as a reference.

[0149] In general, there are many different ways to implement the video encoding and decoding of the present invention. Any of the modules in the presented embodiments may be implemented by electrical circuitry, which may be dedicated or programmable, or by a suitably programmed processor, or by a combination thereof. A computer program may define one or more of the operations described with respect to the presented embodiments. In this regard, each of the schematic block diagrams of Figures 1, 2, 4-9 may also be considered to represent, at least in part, a flow chart diagram of such a computer program, and to represent a method that a processor may perform when executing the computer program. For example, the buffer control module 202 in the block diagram of Figure 2 may be considered to represent a buffer control step. Similarly, other modules may be considered to represent method steps.

[0150] The above shows that the embodiments described with reference to the drawings are illustrative of the present invention, not limiting it. The present invention can be implemented in many alternative ways, which are within the scope of the appended claims. All modifications that come within the spirit and scope of the claims are embraced within their scope. Any reference signs in the claims should not be construed as limiting the scope of the claims. The verb "comprise" in the claims does not exclude the presence of elements or steps other than those recited in the claims. The same applies to similar verbs such as "include" and "comprise". The recitation of a singular element in a claim relating to a product does not exclude that the product may include a plurality of such elements. Similarly, the recitation of a singular step in a claim relating to a method does not exclude that the method may include a plurality of such steps. The mere fact that each dependent claim defines each additional feature does not exclude combinations of additional features other than those reflected in the claims.

Claims

1. A video encoder (100, 600) is configured to encode a series of frames (101, 601) to obtain an encoded series of frames (102, 602), - A frame buffer assembly (104, 604) is configured to provide a representation (115, 614) of a previously encoded frame to the current frame being encoded by applying lossy compression involved in quantization to a decoded version of a previously encoded frame to generate a compressed version of the decoded version of the previously encoded frame, at least temporarily storing the compressed version of the decoded version of the previously encoded frame, and then decompressing the compressed version of the decoded version of the previously encoded frame. - Includes an encoding assembly (103, 603) configured to encode the current frame at least partially by reference to the representation of the previously encoded frame, - The video encoder is configured to generate a dataset (300) comprising an encoded video that can be decoded by a video decoder, the dataset comprising a set of encoded frames, the dataset having a structure that provides a specific location (320) including the lossy compression index (124, 620) applied by the frame buffer assembly to provide the representation of the previously encoded frames that have served at least partially as a reference for encoding the current frame, video encoder (100, 600).

2. The video encoder according to claim 1, wherein the lossy compression is adaptable to compress a portion of the decoded version of the previously encoded frame to a greater extent than other portions of the decoded version of the previously encoded frame.

3. The video encoder according to either claim 1 or 2, wherein the index (124, 620) of the lossy compression includes an index of the target data amount obtained by the lossy compression of the decoded version of the previously encoded frame.

4. The motion image encoder according to claim 3, wherein the amount of data of the target relates to the fixed-size portion in the representation of the previously encoded frame.

5. The video encoder according to either claim 1 or 2, wherein the index (124, 620) of the lossy compression includes at least one parameter that defines the quantization of samples in the decoded version of the previously encoded frame.

6. The motion image encoder according to claim 5, wherein the index (124, 620) of the lossy compression includes the quantization step size.

7. The video encoder according to claim 6, wherein the index (124, 620) of the lossy compression includes the respective quantization step size for each portion of the decoded version of the previously encoded frame.

8. The video encoder according to claim 5, wherein the quantization of the sample includes truncation of the binary value, and the index (124, 620) of the lossy compression includes a truncation level indicating the number of least significant bits truncated in the truncation.

9. The coded assembly (602) is, - A linear transformation module (605) configured to apply a linear transformation to the current frame to provide the current frame in the transformation domain, - An intermode application module (606) configured to provide a difference representation of the portion of the current frame in the conversion domain, wherein the difference representation corresponds, on the one hand, to the difference between the portion of the current frame in the conversion domain and on the other hand to the difference between the portion of the representation of the previously encoded frame in the conversion domain and - Includes a data compression module (607) configured to apply data compression to the difference representation of the portion of the current frame in the transformation domain, - The video encoder (600) according to claim 1, wherein the frame buffer assembly (604) is configured to apply the lossy compression to the decoded version of the previously encoded frame in the conversion domain.

10. The video encoder according to claim 9, wherein the video encoder is configured to encode the other portion of the current frame in the transformation domain by directly applying the data compression to the other portion of the current frame in the transformation domain.

11. A method for encoding a series of frames (101, 601) to obtain an encoded series of frames (102, 602), - The steps of: applying lossy compression involved in quantization to the decoded version of the previously encoded frame to generate a compressed version of the decoded version of the previously encoded frame; at least temporarily storing the compressed version of the decoded version of the previously encoded frame; and then decompressing the compressed version of the decoded version of the previously encoded frame to provide a representation (115, 614) of the previously encoded frame to the current frame to be encoded; - A step of encoding the current frame at least partially with reference to the representation of the previously encoded frames, A method comprising the steps of: generating a dataset (300) comprising an encoded video that can be decoded by a video decoder, wherein the dataset comprises a series of encoded frames, and the dataset has a structure that provides a specific location (320) including the lossy compression index (124, 620) applied to provide the representation of the previously encoded frames that have served at least partially as a reference for encoding the current frame.

12. A computer program for a video encoder, comprising a set of instructions that enables the video encoder to perform the method described in claim 11.

13. A dataset (300) comprising encoded video images that can be decoded by a video image decoder, - A set of encoded frames (102) including encoded frames (304, 305) which are at least partially encoded with reference to a representation of another previously encoded frame (303), wherein the representation includes a set of encoded frames (102) provided by a frame buffer assembly (104) by applying lossy compression involving quantization to a decoded version of the other previously encoded frame to generate a compressed version of the decoded version of the other previously encoded frame, at least temporarily storing the compressed version of the decoded version of the other previously encoded frame, and then decompressing the compressed version of the decoded version of the other previously encoded frame, - The dataset (300) has a structure that provides a specific location (320) including the lossy compression index (124, 620) applied by the frame buffer assembly to provide the representation of the other previously encoded frames that served at least partially as a reference for encoding the encoded frame.

14. A video decoder (400, 800) is configured to decode a series of encoded frames to obtain a series of decoded frames, - A frame buffer assembly (402, 802) configured to provide a representation of previously decoded frames to the encoded frame to be decoded, - A decoding assembly (401, 801) configured to decode the encoded frame at least partially with reference to the representation of the previously decoded frame, - The video decoder is configured to obtain the series of encoded frames from a structured dataset (300) that constitutes an encoded video, and to obtain the irreversible compression index (124, 620) applied by the frame buffer assembly (104, 604) in the video encoder (100, 600) that generated the series of encoded frames from a specific position (320) in the structured dataset, wherein the frame buffer assembly in the video encoder applies the irreversible compression involved in quantization to the decoded version of the previously encoded frames to generate a compressed version of the decoded version of the previously encoded frames, stores the compressed version of the decoded version of the previously encoded frames at least temporarily, and then decompresses the compressed version of the decoded version of the previously encoded frames to provide a representation (115, 614) of the previously encoded frames to the current frame to be encoded. - The frame buffer assembly of the decoder is configured to process the previously decoded frames in a manner equivalent to the lossy compression applied by the frame buffer assembly in the video encoder, using the index obtained from the dataset to obtain a representation that at least partially functions as a reference for decoding the current frame. (400, 800)

15. The decoder according to claim 14, wherein the decoder is configured to set at least one parameter of the processing applied to the frames that have been decoded so far, depending on the index (124, 620) of the lossy compression applied by the frame buffer assembly in the video encoder.

Images