Video encoder including frame buffer component

Abstract

A video encoder (100) encodes a sequence of frames (101) in order to obtain an encoded sequence of frames (102) in such a manner. A frame buffer component (104) provides a representation (125) of a previously encoded frame for a current frame to be encoded. The representation (125) is resulting from lossy compression of a decoded version of a previously encoded frame by a frame buffer component (104). The encoding component (103) encodes the current frame at least in part with reference to a representation (125) of a previously encoded frame. A video encoder generates a data set constituting an encoded video, the data set comprising a sequence of encoded frames (102). The data set includes an indication (124) associated with a previously encoded frame regarding a lossy compression applied by the frame buffer component (104) to provide a representation (125) of the previously encoded frame, the representation (125) being at least partially used as a reference for encoding the current frame.

Description

Technical Field

[0001] One aspect of the invention relates to a video encoder including a frame buffer component. The video encoder can be used, for example, in a camera device for encoding a captured frame sequence. The camera device can then store the encoded frame sequence, or can send the sequence constituting a video to another device, or the camera device can both store the encoded frame sequence and send the sequence constituting a video to another device. Other aspects of the invention relate to a method for encoding a frame sequence, a computer program for a video encoder, a data set including an encoded frame sequence, and a video decoder. Background Art

[0002] There are many video coding techniques including inter-mode coding. In inter-mode coding, a frame or at least a portion of the frame is encoded with reference to one or more previously encoded frames. Typically, a differential representation of the portion of the frame to be encoded is generated. The differential representation corresponds to the difference between the portion of the frame to be encoded on the one hand and a portion of a reference frame taken from a decoded version of a previously encoded frame on the other hand. The differential representation thus includes a residual, which is as small as possible for efficient coding.

[0003] Motion estimation may be performed to find a frame portion in a decoded version of one or more previously encoded frames which provides the smallest residual when used as a reference frame portion. Motion vector data is then provided which indicates the frame portion which has been found as the best reference for the frame portion to be encoded. Alternatively, motion estimation may be omitted. In that case, a given position within the frame may serve as a criterion for designating a reference frame portion for inter-mode encoding.

[0004] Inter-mode coding requires a so-called frame buffer in which decoded versions of one or more previously encoded frames are at least temporarily stored. Typically, the frame buffer is a dedicated storage circuit that provides a sufficiently large storage capacity at an acceptable cost. For example, the frame buffer can be a dynamic random access memory (DRAM). Although the cost is acceptable, the frame buffer may still account for a significant portion of the total cost of the video encoder. This is particularly the case for high-definition video encoders.

[0005] In addition, when encoding video in inter-frame mode, a relatively large amount of data needs to be written to the frame buffer per unit time, and a relatively large amount of data needs to be read from the frame buffer per unit time. This requires a relatively large bandwidth, which may result in relatively large power consumption. Similarly, the power consumption required to achieve a sufficiently large bandwidth may account for a large portion of the total power consumption of the video encoder. Again, this is especially true for high-definition video encoders.

[0006] Frame buffer compression has been proposed to allow smaller frame buffers to reduce costs and to allow bandwidth requirements to be relaxed to reduce power consumption. Instead of storing a decoded version of a previously encoded frame in a frame buffer, a compressed version of the decoded version is stored. Typically, the frame buffer compression should be mathematically lossless or at least visually lossless. Otherwise, if frame buffer compression is not used, the current frame portion will be encoded in inter-frame mode relative to a degraded version of the reference frame portion itself. Frame buffer compression may therefore introduce visual artifacts, especially when several consecutive frames are at least partially encoded in inter-frame mode. This is due to the accumulation of errors throughout these frames.

[0007] Patent publication WO2019025640A1 describes a frame buffer compression technology that is particularly suitable for applications where the encoded video data rate is relatively low. In this frame buffer compression technology, the decoded version of the encoded frame is effectively divided into relatively large parts, which are then encoded for storage in the frame buffer. The encoding may be lossy to a certain extent, which allows a relatively high compression ratio to be achieved. In principle, this lossy encoding may affect the coding efficiency or image quality, or both the coding efficiency and the image quality. However, it has been found that when the encoded video data rate is relatively low, the loss in coding efficiency or image quality or the loss in both coding efficiency and image quality may be relatively small and even negligible. The image quality may be relatively close to that of a conventional video encoder that does not compress reference frames or slightly compresses reference frames in a lossless or quasi-lossless manner. Summary of the invention

[0008] There is a need for a video encoding technology that can achieve better overall performance in terms of cost, power consumption, and image quality regardless of the video data rate.

[0009] According to an aspect of the present invention as defined in claim 1, there is provided a video encoder adapted to encode a sequence of frames so as to obtain a sequence of encoded frames, the video encoder comprising:

[0010] - a frame buffer component adapted to provide a representation of a previously encoded frame for a current frame to be encoded, the representation resulting from lossy compression of a decoded version of the previously encoded frame; and

[0011] an encoding component adapted to encode a current frame with reference at least in part to a representation of a previously encoded frame;

[0012] -The video encoder is arranged to provide a data set, the data set comprising a sequence of coded frames and comprising an indication of the lossy compression associated with a previously coded frame, the lossy compression being applied to provide a representation of the previously coded frame, the representation of the previously coded frame being used at least in part as a reference for encoding the current frame.

[0013] According to another aspect of the present invention as defined in claim 11, there is provided a method for encoding a sequence of frames so as to obtain a sequence of encoded frames, the method comprising:

[0014] - providing a representation of a previously coded frame for a current frame to be coded by lossy compression of a decoded version of the previously coded frame; and

[0015] encoding a current frame with reference at least in part to a representation of the previously encoded frame;

[0016] - providing a data set comprising a sequence of coded frames and comprising an indication of the lossy compression associated with the previously coded frame, the lossy compression being applied to provide a representation of the previously coded frame, the representation of the previously coded frame being used at least in part as a reference for encoding the current frame.

[0017] According to a further aspect of the invention as defined in claim 12, there is provided a computer program for a video encoder, the computer program comprising a set of instructions enabling the video encoder to perform the method defined above.

[0018] According to a further aspect of the present invention as defined in claim 13, there is provided an instruction set comprising:

[0019] - a sequence of coded frames, comprising a coded frame which is coded at least in part with reference to a representation of another previously coded frame, the representation resulting from lossy compression of a decoded version of the other previously coded frame; and

[0020] - an indication associated with the further previously coded frame of the lossy compression applied to provide a representation of the further previously coded frame, the representation of the further previously coded frame being used at least partly as a reference for encoding the coded frame.

[0021] According to a further aspect of the present invention as defined in claim 14, there is provided a video decoder adapted to decode a sequence of coded frames so as to obtain a sequence of decoded frames, the video decoder comprising:

[0022] - a frame buffer component adapted to provide a representation of previously decoded frames for a coded frame to be decoded; and

[0023] - a decoding component adapted to decode a coded frame at least partially with reference to a representation of a previously decoded frame;

[0024] - the video decoder is arranged to retrieve, from a data set comprising a sequence of coded frames, an indication of a lossy compression contained in said data set, the lossy compression being applied by a frame buffer component in a video encoder that produced the sequence of coded frames; and

[0025] - The frame buffer component of the decoder is adapted to process the previously decoded frame in a manner equivalent to the lossy compression applied by the frame buffer component in the video encoder, using the indication retrieved from the data set, so as to obtain a representation at least partially used as a reference for decoding the current frame.

[0026] In each of these aspects, the indication of lossy compression allows the decoder to accurately reproduce the representation of the previously encoded frame as a reference in the inter-frame mode encoding. This is even applicable when the representation is obtained by compressing the previously encoded frame to a relatively large extent. In addition, this also significantly reduces the error accumulation of continuous frames that are encoded at least partially in the inter-frame mode. Therefore, a relatively high compression rate can be used for frame buffer compression without significantly compromising the image quality. Subsequently, this allows a significant reduction in the storage capacity that the frame buffer should provide, thereby reducing costs, and significantly relaxes bandwidth requirements, thereby reducing power consumption. In addition, it is possible to provide an indication of lossy compression without requiring a large amount of resources and processing, which will highlight the reduction in cost and power consumption. Therefore, the present invention allows video encoding with better overall performance in terms of cost, power consumption and image quality regardless of the video data rate.

[0027] For illustrative purposes, some embodiments of the present invention are described in detail with reference to the accompanying drawings. In this description, additional features will appear, some of which are defined in the dependent claims, and advantages will be apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic block diagram of a video encoder.

[0029] Figure 2 is a schematic block diagram of a frame buffer component in a video encoder.

[0030] Figure 3 is a conceptual diagram of the datasets that a video encoder can provide.

[0031] Figure 4 is a schematic block diagram of a video decoder.

[0032] Figure 5 is a schematic block diagram of a frame buffer component in a video decoder.

[0033] Figure 6 is a schematic block diagram of an alternative video encoder.

[0034] Figure 7 is a schematic block diagram of a frame buffer component in an alternative video encoder.

[0035] Figure 8 is a schematic block diagram of an alternative video decoder.

[0036] Fig. 9 is a schematic block diagram of a frame buffer component in an alternative video decoder. DETAILED DESCRIPTION

[0037] Figure 1 A video encoder 100 is schematically shown. Figure 1 A schematic block diagram of the video encoder 100 is provided. Basically, the video encoder 100 can encode a frame sequence 101 that can constitute a video so as to obtain an encoded frame sequence 102. Therefore, the encoded frame sequence 102 can constitute an encoded video, which includes a smaller amount of data than the video in its original form. Therefore, the video encoder 100 can achieve efficient storage of video, or efficient transmission of video, or efficient storage of video and efficient transmission of video by encoding the video. The video encoder 100 can be part of a device (e.g., a smartphone) capable of capturing video, and the video encoder 100 adds value to the device due to its efficacy.

[0038] The video encoder 100 includes an encoding component 103 and a frame buffer component 104. The encoding component 103 includes various functional modules: a motion estimation and compensation module 105, an intra / inter mode application module 106, a decorrelation transformation module 107, a quantization module 108, an entropy coding module 109, a data packaging module 110 and a controller 111. The aforementioned modules can perform operations related to encoding, and thus can be regarded as forming an encoding chain 105-111.

[0039] The encoding component 103 also comprises functional modules that perform operations related to decoding and can therefore be considered to form a decoding chain 112-114: a dequantization module 112, an inverse transform module 113 and a reconstruction module 114. The functional modules 105-114 of the encoding component 103 can each be implemented in the form of a dedicated circuit, a programmable circuit or a suitably programmed processor, or any combination thereof.

[0040] The video encoder 100 operates substantially as follows. In practice, the video encoder 100 divides the frame to be encoded into pixel blocks. The pixel blocks constitute frame portions, which have specific positions in the frame to be encoded. The video encoder 100 can encode the individual frame portions that together form the frame to be encoded separately. For convenience and clarity, the frame to be encoded is referred to as the current frame hereinafter.

[0041] The frame portion may be encoded in one of two modes: intra mode and inter mode. In inter mode, the frame portion is encoded with reference to a representation 115 of a previously encoded frame. The frame buffer component 104 provides a representation 115 of a previously encoded frame. In intra mode, the frame portion is encoded without reference to another frame. In this mode, the frame portion may be encoded "as is" or may be encoded with reference to one or more other frame portions of the current frame.

[0042] The controller 111 can decide whether the frame portion is encoded in the inter-frame mode or in the inter-frame mode. This decision can be made based on a predefined coding scheme. Alternatively, or in addition, the decision can be made dynamically based on whether the intra-frame mode or the inter-frame mode is expected to be more advantageous in terms of, for example, data compression rate or image quality or both.

[0043] In inter-frame mode, the motion estimation and compensation module 105 searches for a portion 116 that is most similar to the portion of the frame to be encoded in the representation 115 of the previously encoded frame. The position of this best matching frame portion 116 in the previously encoded frame may be different from the specific position that the portion of the frame to be encoded has in the current frame. The motion estimation and compensation module 105 provides motion vector data that indicates this position difference. In addition, the motion vector data may indicate the previously encoded frame to which the best matching frame portion 116 belongs. This is because the first encoder may search in several previously encoded frames to find the best matching frame portion 116.

[0044] The intra / inter mode application module 106 provides an intra / inter defined frame portion 117 which is applied to the decorrelating transform module 107. In inter mode, the intra / inter defined frame portion 117 is a differential representation of a frame portion. The differential representation corresponds to the difference between the frame portion to be encoded on the one hand and the best matching frame portion 116 found by the motion estimation and compensation module 105 on the other hand. The differential representation comprises a residual representing this difference.

[0045] In intra mode, the intra / inter defined frame portion 117 applied to the decorrelating transform module 107 may simply correspond to the frame portion to be encoded. In that case, the intra / inter defined frame portion 117 comprises pixels. Alternatively, the intra / inter mode application module 106 may provide a representation of the frame portion to be encoded, which refers to one or more other portions of the current frame. These other portions that may be used as references may be adjacent to the frame portion to be encoded. In this alternative implementation, the intra / inter defined frame portion 117 may comprise a residual.

[0046] The decorrelating transform module 107 applies a decorrelating transform to the intra / inter-defined frame portion 117 provided by the intra / inter mode application module 106. As a result, the decorrelating transform module 107 provides a transformed intra / inter-defined frame portion 118, which is a representation of the intra / inter-defined frame portion 117 in the transform domain. The transformed intra / inter-defined frame portion 118 comprises transformed samples, which are actually substitutes for the pixels or residuals included in the intra / inter-defined frame portion 117 that has undergone the decorrelating transform. The decorrelating transform causes the transformed samples in the transformed intra / inter-defined frame portion 118 to have an entropy that is generally lower than that of the pixels or residuals included in the intra / inter-defined frame portion 117. The decorrelating transform may be performed in the form of, for example, a discrete cosine transform (usually denoted by the abbreviation DCT) or a wavelet transform.

[0047] The quantization module 108 quantizes the transformed samples in the transformed intra / inter-frame delimited frame portion 118. Thus, the quantization module 108 provides a quantized version 119 of the transformed intra / inter-frame delimited frame portion 118. Due to the quantization of the transformed samples, the quantized version 119 may contain a smaller amount of data than the transformed intra / inter-frame delimited frame portion 118 provided by the decorrelating transform module 107 and subjected to quantization. Thus, quantization provides data compression. In principle, such data compression is lossy: quantization results in a loss of information.

[0048] The quantization module 108 may quantize the transformed samples at a resolution that depends on the degree of effect of the transformed samples on the image quality. If the transformed samples have a relatively large effect on the image quality, the resolution may be relatively high. Conversely, if the transformed samples have a relatively small effect on the image quality, the resolution may be relatively low. The transformed samples have a specific position in the transformed intra / inter defining frame portion 118. This position may indicate the degree of effect of the transformed samples on the image quality.

[0049] The entropy coding module 109 applies entropy coding to the quantized version 119 of the transformed intra / inter-frame defined frame portion 118. Thus, entropy coding provides a quantized and encoded version 120 of the transformed intra / inter-frame defined frame portion 118. Due to the entropy coding, the quantized and encoded version 120 may contain less data than the quantized version 119 of the transformed intra / inter-frame defined frame portion 118. Thus, entropy coding provides further data compression. This further data compression may be lossless: in contrast to quantization, entropy coding need not result in information loss. For example, entropy coding may be based on the techniques described in patent publication US9332258.

[0050] The controller 111 may control the data compression provided by the quantization in the quantization module 108, and the data compression provided by the entropy coding in the entropy coding module 109. Such control may be based on the data included in the intra / inter-frame defining frame portion 117 on the one hand, and on the target amount of data included in the quantized and encoded version 120 of the transformed intra / inter-frame defining frame portion 118 on the other hand. Based on these inputs, the controller 111 may set the resolution of the quantization performed by the quantization module 108. The controller 111 may also select an entropy coding scheme that provides a high level of data compression.

[0051] The data packing module 110 adds the quantized and encoded version 120 of the transformed intra / inter-frame defined frame portion 118 to the data set that will constitute the encoded video. The data packing module 110 may further add an indication of whether intra-frame mode or inter-frame mode is applied. If the relevant frame portion is encoded in inter-frame mode, the data packing module 110 will further add the motion vector data provided by the motion estimation and compensation module 105 for the frame portion. The data packing module 110 will further add information about how the quantized and encoded version 120 of the transformed intra / inter-frame defined frame portion 118 was generated. For example, this information may include information about the resolution of the quantization applied.

[0052] Therefore, the data packaging module 110 synthesizes (composes) the coded frame portion and embeds the coded frame portion into the data set together with the information identifying the coded frame portion. Once the video encoder 100 has encoded all the frame portions of the current frame, the data set will include an encoded version of the current frame, which will be referred to as the encoded current frame. The data packaging module 110 can output the data set in the form of a data stream with a specific structure. Alternatively, the data packaging module 110 can output the data set, for example, in the form of a data file to be stored on a device.

[0053] As mentioned above, Figure 1 The video encoder 100 shown comprises the aforementioned decoding chain 112-114, comprising a dequantization module 112, an inverse transformation module 113 and a reconstruction module 114. The decoding chain 112-114 allows obtaining a decoded version of a current frame that has been encoded within the video encoder 100. The decoding chain 112-114 operates substantially as follows.

[0054] The dequantization module 112 applies inverse quantization to the transformed quantized version 119 of the intra / inter-delimited frame portion 118. Inverse quantization is an operation that reverses the quantization in the quantization module 108, which roughly reproduces the previously transformed samples. Therefore, the dequantization module 112 provides a dequantized version 121 of the transformed intra / inter-delimited frame portion 118. This dequantized version 121 can be regarded as a rough representation of the transformed intra / inter-delimited frame portion 118 provided by the transform coding modules in the coding chain 105-111.

[0055] The inverse transform module 113 applies the inverse operation of the decorrelation transform to the dequantized version 121 of the transformed intra / inter defined frame portion 118. As a result, the inverse transform module 113 provides a decoded version 122 of the intra / inter defined frame portion 117. The decoded version 122 corresponds to a certain extent to the intra / inter defined frame portion 117 provided by the intra / inter mode application module 106 in the encoding chain 105-111. In the case where the resolution of quantization is relatively high, the degree of correspondence between the decoded version 122 and the aforementioned intra / inter defined frame portion 117 is relatively high. In the extreme case where no quantization is applied, the decoded version 122 will correspond exactly to the intra / inter defined frame portion 117. Conversely, if the resolution of quantization is relatively low, the degree of correspondence between the decoded version 122 and the intra / inter defined frame portion 117 will be relatively low.

[0056] As described above, the reconstruction module 114 provides a decoded version 123 of the frame portion that has been encoded in the encoding chain 105-111. If the frame portion is encoded in inter mode, the reconstruction module 114 adds a decoded version 122 of the intra / inter defined frame portion 117 to the best matching frame portion 116 used as reference.

[0057] If the frame portion is encoded in intra mode, the reconstruction module 114 may simply use the decoded version 122 of the intra / inter-defined frame portion 117 provided by the inverse transform module 113 as the decoded frame portion 123. As described above, intra mode may allow a representation of a frame portion that references one or more other portions of the current frame. In that case, the reconstruction module 114 may provide the decoded frame portion 123 based on the decoded version of the intra / inter-defined frame using these references.

[0058] The frame buffer component 104 receives the decoded frame portion 123 provided by the reconstruction module 114, which is the last in the decoding chain 112-114. Therefore, once the current frame is fully encoded, the frame buffer component 104 will receive various decoded frame portions, which can all be obtained in the manner described above. Therefore, the frame buffer component 104 receives a decoded version of the current frame, which for convenience will be referred to as the decoded current frame. Then, when a subsequent frame is to be encoded at least partially in inter-frame mode, the frame buffer component 104 can provide a representation of the decoded current frame. The representation can then be used as a reference for encoding one or more frame portions of the subsequent frame in inter-frame mode.

[0059] Up to this point, the operations described may correspond to a certain extent to operations performed by a video encoder conforming to, for example, one of the following video coding standards: MPEG-2, MPEG-4 or HEVC. Figure 1 The illustrated video encoder 100 differs particularly from the previously described video encoders in the further operations performed by the frame buffer component 104 and the data packing module 110. In general, these operations include the following.

[0060] The frame buffer component 104 applies lossy compression to the decoded current frame. This lossy compression provides a compressed version of the decoded current frame. The frame buffer component 104 stores the compressed version at least temporarily. Subsequently, the compressed version of the decoded current frame can be decompressed to provide a decompressed version of the decoded current frame. The decompressed version then constitutes a representation of the current frame, which can be used as a reference for encoding one or more frame portions of subsequent frames in an inter-frame mode.

[0061] Additionally, the frame buffer component 104 provides an indication 124 of the applied lossy compression. The indication 124 may indicate the extent to which the decoded current frame was compressed for temporary storage within the frame buffer component 104. The data packaging module 110 may include the indication 124 in a data set that includes the encoded current frame and other encoded frames that form the encoded video.

[0062] Figure 2 A frame buffer component 104 in the video encoder 100 is shown. Figure 2A schematic block diagram of the frame buffer component 104 is provided. The frame buffer component 104 includes a data memory 201, a controller 202 and various functional modules. These functional modules include a decorrelation transform module 203, a delay buffer 204, a quantization module 205 and an entropy coding module 206. The aforementioned functional modules can be regarded as constituting a cache compression chain 203-206. The frame buffer component 104 also includes functional modules that can be regarded as constituting a cache decompression chain 207-209: an entropy decoding module 207, a dequantization module 208 and an inverse transform module 209. The functional modules of the frame buffer component 104 can each be implemented in the form of, for example, a dedicated circuit, a programmable circuit or a suitably programmed processor or any combination thereof.

[0063] For the sake of distinction, the controller 202 in the frame buffer component 104 will be referred to as the buffer controller 202 hereinafter. For the same reason, the decorrelating transform module 203, the quantization module 205, the entropy coding module 206, the entropy decoding module 207, the dequantization module 208 and the inverse transform module 209 will be referred to as the cached decorrelating transform module 203, the cached quantization module 205, the cached entropy coding module 206, the cached entropy decoding module 207, the cached dequantization module 208 and the cached inverse transform module 209 hereinafter, respectively. Therefore, the adjective "cached" is added for the sake of distinction.

[0064] The functional module of the frame buffer component 104 may be part of an integrated circuit, which may also include Figure 1 1 and 10. The encoding component 103 of the video encoder 100 is shown. However, the data memory 201 of the frame buffer component 104 can be in the form of a separate circuit, for example, a dynamic random access memory circuit, usually referred to by the abbreviation DRAM. Such an embodiment can then include a data write / read channel through which data can be written to the data memory 201 and through which data can be read from the data memory 201. The data write / read channel, which can include a bus, has a bandwidth that limits the amount of data that can be written to the data memory 201 and can be read from the data memory 201 per unit time. In other words, the bandwidth defines the maximum data rate for reading and writing.

[0065] The frame buffer component 104 operates substantially as follows. Assume that the frame buffer component 104 receives a decoded frame portion 123 provided by the reconstruction module 114 as described above. Other decoded frame portions, which may be from the current frame as well as from other frames, may be processed in a manner similar to that described below.

[0066] The cached decorrelating transform module 203 applies the decorrelating transform to the decoded frame portion 123. Thus, the cached decorrelating transform module 203 provides a transformed decoded frame portion 210. The decorrelating transform applied by the cached decorrelating transform module 203 may be the same as the transform applied by Figure 1 The decorrelating transform applied by the decorrelating transform module 107 in the encoding component 103 of the illustrated video encoder 100 may be similar or different. In either case, the transformed decoded frame portion 210 includes transformed samples that are actually substitutes for the pixels included in the decoded frame portion 123 that was subjected to the decorrelating transform. Here, the latter decorrelating transform causes the transformed samples in the transformed decoded frame portion 210 to have an entropy that is generally lower than the pixels in the decoded frame portion 123 provided by the reconstruction module 114.

[0067] The delay buffer 204 can temporarily store the transformed decoded frame portion 210 provided by the cache decorrelation transform module 203. This temporary storage can help achieve sufficient compression with minimal loss of image quality. This will be explained in the following section related to the cache controller 202.

[0068] The cached quantization module 205 quantizes the transformed samples in the transformed decoded frame portion 210. Therefore, the cached quantization module 205 provides a quantized compressed version 211 of the transformed decoded frame portion 210. That is, due to the quantization of the transformed samples, the cached quantization module 205 provides data compression, as explained above in connection with the quantization module 108 in the encoding component 103. The degree of data compression depends on the resolution of the quantization performed. The lower the resolution, the greater the degree of data compression, but the more information is lost.

[0069] The video encoder 100 is configured so that even if the decoded frame portion 123 is lossily compressed to a relatively large extent, the image quality will not be significantly reduced. This will be further described below. Nevertheless, like the quantization module 108 in the encoding component 103, the buffered quantization module 205 can quantize the transformed samples at a resolution that depends on the degree to which the transformed samples affect the image quality.

[0070] The cached entropy coding module 206 applies entropy coding to the quantized compressed version 211 of the transformed decoded frame portion 210 provided by the cached quantization module 205. Thus, the cached entropy coding module 206 provides a quantized and encoded compressed version 212 of the transformed decoded frame portion 210. That is, entropy coding provides data compression, as explained above in connection with the entropy coding module 109 in the encoding component 103. For convenience, the quantized and encoded compressed version 212 of the transformed decoded frame portion 210 will be referred to as the compressed decoded frame portion 212 hereinafter.

[0071] The compressed decoded frame portion 212 is sent to the data memory 201 for storage in the data memory 201. In the process of encoding the current frame, the buffer compression chain 203-206 continuously provides each compressed decoded frame portion. These consecutive compressed decoded frame portions sent to the data memory 201 form a memory input data stream. The memory input data stream should preferably not exceed the maximum data rate of the data write / read channel, which is limited by the bandwidth of the channel as described above.

[0072] The cache controller 202 can control the resolution of the quantization applied by the cache quantization module 205 in the cache compression chain 203-206. This control can be based on the data included in the transformed decoded frame part 210 on the one hand, and on the target amount of data included in the compressed decoded frame part 212 on the other hand. The target amount of data can be related to the maximum data rate of the data write / read channel. For example, the average of the target amount of data can correspond to the maximum data rate divided by the frequency of providing the decoded frame part. The target amount of data can also be slightly lower than the aforementioned ratio to have a certain margin. By controlling the resolution of the quantization in this way, the cache controller 202 can achieve that the memory input data stream does not exceed the maximum data rate of the data write / read channel, or only occasionally exceeds the maximum data rate.

[0073] The delay buffer 204 allows the buffer controller 202 to control the resolution of the quantization by considering multiple transformed decoded frame portions, which may be adjacent in position. Some of these portions may be relatively uniform, while others may be more complex. Considering these different portions allows the resolution of the quantization to be controlled so that the distribution of the quantization error across these portions is more uniform, compared to control that considers only one transformed decoded frame portion or a few transformed decoded frame portions. The more uniform the distribution of the quantization error, the less loss of image quality for a given level of compression.

[0074] As described above, the cache controller 202 may provide an indication 124 of the lossy compression applied to the decoded frame portion 123. As described above, the indication 124 may indicate the degree to which the decoded frame portion 123 is compressed. Specifically, the indication 124 may directly or indirectly indicate the level of quantization applied by the cache quantization module 205.

[0075] For example, the indication 124 may be related to a target to be achieved for the lossy compression. For example, the compression target may correspond to the aforementioned target data volume, the maximum data rate that must be observed, or the bandwidth of the data writing / reading channel. In that case, the indication 124 may be considered as an indirect indication about the lossy compression to be applied. The indicated compression target allows another compression chain to reproduce the data generated by Figure 21. The other compression chain may be similar to the cache compression chains 203-206 in the frame buffer component 104 shown in FIG. 1. The other compression chain may be similar to the cache compression chains 203-206. If the indication 124 regarding the lossy compression is of a general nature (e.g., a maximum data rate that must be observed or a bandwidth of a data write / read channel), an entity other than the cache controller 202 may provide the indication.

[0076] The indication 124 of lossy compression provided by the buffer controller 202 may also be of a direct nature by indicating the resolution of quantization. Thus, the indication may include, for example, one or more quantization step sizes to be applied by the buffer quantization module 205 when quantizing the transformed decoded frame portion 210. Different quantization step sizes may define different resolutions applicable to different groups of samples in the transformed decoded frame portion 210.

[0077] The indication 124 about lossy compression can be provided separately for each decoded frame portion, each decoded frame portion is compressed and subsequently written to the data memory 201. That is, once the decoded frame portion 123 is compressed and written to the data memory 201 as described above, the subsequent decoded frame portion will be compressed and written to the data memory 201. Afterwards, the subsequent decoded frame portion undergoes a decorrelation transform to produce another transformed decoded frame portion. The decoded frame portion after the other transform can include more or less data. This can make the quantized resolution of the decoded frame portion after the other transform can be set to a higher or lower level respectively. Therefore, the compression of the subsequent decoded frame portion can be less or more lossy than the compression of the decoded frame portion 123 discussed above.

[0078] Once the current frame is encoded, the cache compression chain 203-206 compresses the individual decoded frame portions that constitute the decoded current frame. Thus, the cache compression chain 203-206 produces individual compressed decoded frame portions that are written to the data memory 201. The individual compressed decoded frame portions constitute a compressed version of the decoded current frame. The compressed version will be referred to below as the compressed decoded current frame. Thus, the compressed decoded current frame is produced as a result of the lossy compression in the cache compression chain 203-206 described above.

[0079] Figure 2The illustrated cache decompression chain 207-209 in the frame cache component 104 can decompress the compressed decoded current frame in the following manner. The cache entropy decoding module 207 receives the compressed decoded frame portion 212 previously written to the data memory 201 and now read out from the data memory 201. The cache entropy decoding module 207 performs entropy decoding on the read compressed decoded frame portion 212. Thus, the cache entropy decoding module 207 can reconstruct the quantized compressed version 211 of the transformed decoded frame portion 210 previously provided by the cache quantization module 205 in the cache compression chain 203-206.

[0080] The cached dequantization module 208 applies inverse quantization to the quantized compressed version 211 of the transformed decoded frame portion 210. Inverse quantization is an operation that reverses the quantization in the cached quantization module 205, which roughly reproduces the transformed samples of the transformed decoded frame portion 210. Therefore, the cached dequantization module 208 provides a decompressed version 213 of the transformed decoded frame portion 210 previously provided by the cached decorrelating transform module 203 in the cached compression chain 203-206. This decompressed version 213 can be regarded as a coarse representation of the transformed decoded frame portion 210.

[0081] The cached inverse transform module 209 applies the inverse operation of the decorrelation transform to the decompressed version 213 of the transformed decoded frame portion 210. Thus, the cached inverse transform module 209 provides a decompressed version 214 of the decoded frame portion 123 that was previously decoded by Figure 2 The frame buffer component 104 is shown receiving and compressing in the cache compression chain 203-206 for storage in the data memory 201. The lossy compression in the cache compression chain 203-206 causes the decompressed version 214 of the decoded frame portion 123 to not exactly correspond to the decoded frame portion 123 sent to the frame buffer component 104. The more the decoded frame portion 123 is compressed, the greater the loss in compression is generally, and therefore, the greater the degree to which the decompressed version 214 differs from the decoded frame portion 123.

[0082] Therefore, the frame buffer component 104 can provide respective decompressed versions of the decoded frame parts, which together constitute the decompressed version of the decoded current frame. The decompressed version of the decoded current frame or at least a part of the decompressed version is a representation of the decoded current frame, so the representation can be used as a reference for encoding subsequent frames at least partially in inter-frame mode.

[0083] Lossy compression may cause the representation of the decoded current frame constituting the reference to not completely correspond to the decoded current frame. This may therefore reduce the image quality of the decoded version of the encoded video produced by the video encoder 100. In the case where the decoded version of the frame is used as a reference, this is particularly due to the accumulation of errors when decoding. However, due to the indication 124 of the applied lossy compression, the video decoder can reproduce the representation constituting the reference in the video encoder 100. This prevents the accumulation of errors that could seriously affect the image quality.

[0084] like Figure 1 As shown, the data packaging module 110 can include an indication 124 of the lossy compression applied by the frame buffer component 104 in a data set that also includes the encoded video. The data set can have a given structure that provides for the inclusion of the indication. The structure can provide a specific location where the indication may exist so that the video decoder can easily retrieve the indication.

[0085] Figure 3 A data set 300 is shown that may be provided by the video encoder 100 . Figure 3 A schematic diagram of the structure of the data set 300 is provided. In this example, the data set 300 may be in the form of a data stream. The data set 300 includes an encoded frame sequence 102, which is schematically represented in Figure 3 The top part. Figure 3 1-305 in the sequence, one coded frame 303 of the coded frames 301-305 will be shown and described in more detail. For convenience, the coded frame 303 will be referred to as the described coded frame 303 below. Other coded frames in the coded frame sequence 102 may have a structure similar to or even identical to the structure of the described coded frame 303.

[0086] Assume that the coded frame 303 described above is Figure 1 and Figure 2 The encoding of the current frame described is generated. It is further assumed that the encoding is performed at least partially in inter-frame mode. Therefore, the current frame has been encoded with reference to one or more other previously encoded frames. Any other encoded frames located to the left of the described encoded frame 303 may constitute an encoded version of the frame used as a reference for encoding the current frame. The foregoing applies to Figure 3 Any other coded frame located to the right of the described coded frame 303 may be a coded version of a frame that is coded at least partially with reference to the current frame. Figure 3 Encoded frames 304, 305 are shown.

[0087] The depicted coded frame 303 includes several data fields for several types of data 306-312, which are Figure 3 . The data include a picture header 306, a plurality of coded frame parts 307-311 and a picture end indicator 312. Each coded frame part can be generated by the video encoder 100 in the manner described above. In principle, one or more coded frame parts can be encoded in inter-frame mode, and one or more other coded frame parts can be encoded in intra-frame mode. It is also possible that all coded frame parts included in the described coded frame 303 are encoded in inter-frame mode, or that all coded frame parts are encoded in intra-frame mode. The first coded frame part 307 of the described coded frame 303 will be presented and described in more detail. The other coded frame parts 308-311 can have a structure similar to or even identical to the structure of the first coded frame part 307.

[0088] The first coded frame portion 307 includes a number of data fields for a number of types of data 313-316 that are Figure 3 The data includes a header 313, a mode indicator 314, reference data 315, and coded picture data 316. The header 313 can mark the position of the first coded frame portion 307 in the described coded frame 303. The header 313 can include more information, which will be described below. The mode indicator 314 indicates whether the first coded frame portion 307 is generated by encoding in an inter-frame mode or encoding in an intra-frame mode.

[0089] If encoding is performed in inter-frame mode, then the reference data 315 may include the aforementioned motion vector data indicating one or more portions of other frames used as reference. If encoding is performed in intra-frame mode, then the reference data 315 may indicate portions of the current frame itself that are used for intra-frame encoding. Alternatively, if no other portions of the current frame itself are referenced, the reference data 315 may be omitted. The encoded picture data 316 includes the aforementioned motion vector data indicating one or more portions of other frames used as reference. Figure 1 The encoding chain 105 - 111 in the video encoder 100 provides a quantized and encoded version 120 of the transformed intra / inter defined frame portion 118 .

[0090] The header 313 of the first coded frame part 307 will be presented and described in more detail. The other coded frame parts may include a header having a structure similar to or even identical to that of the header 313 of the first coded frame part 307.

[0091] Header 313 includes several data fields for several types of data 317-321, which are Figure 3The data includes a tag 317, a flag field 318, a coding quantization indication 319, a frame buffer compression indication 320, and a packet size and control data 321. The tag 317 can mark the header 313 and the first coded frame portion 307. The flag field 318 may include one or more flags that can be used to decode the first coded frame portion 307. The coding quantization information 319 can indicate one or more quantization steps used in quantization to obtain the first coded frame portion 307.

[0092] A special aspect of the header is the frame buffer compression indication 320 included in the header. The frame buffer compression indication 320 includes an indication 124 of the lossy compression that the frame buffer component 104 applied to store the decoded version of the first encoded frame portion 307 in its data memory 201. Thus, the frame buffer compression indication 320 enables a video decoder to also apply the lossy compression or at least an approximation thereof, or to reproduce the effect of the lossy compression.

[0093] exist Figure 3 In the variation of the data set 300 shown, the picture header 306 may include a frame buffer compression indication, which at least partially provides an indication 124 about the lossy compression applied by the frame buffer component 104. For example, a fixed compression level or a predefined compression scheme may be applied by the frame buffer component 104 to the described coded frame 303. In that case, the frame buffer compression indication in the picture header may indicate the fixed compression level or the predefined compression scheme. Then, the frame buffer compression indication 320 in the frame header 313 of the first coded frame part 307 may be omitted, and the same applies to the respective frame headers of each other coded frame part belonging to the described coded frame 303. Alternatively, the frame buffer compression indication in the header of the coded frame part may indicate an adjustment relative to the fixed compression level or the predefined compression scheme.

[0094] Figure 4 Schematically shows the same Figure 1 and Figure 2 The video encoder 100 is compatible with the video decoder 400 . Figure 4A schematic block diagram of the video decoder 400 is provided. The video decoder 400 includes a decoding component 401 and a frame buffer component 402. For the sake of distinction, the frame buffer component 402 will be referred to as the decoder frame buffer component 402 hereinafter. The decoding component 401 includes various functional modules: a data unpacking module 403, a motion compensation module 404, an entropy decoding module 405, a dequantization module 406, an inverse transform module 407, and a reconstruction module 408. For the sake of distinction, the dequantization module 406, the inverse transform module 407, and the reconstruction module 408 will be referred to as the decoder dequantization module 406, the decoder inverse transform module 407, and the decoder reconstruction module 408, respectively, hereinafter. That is, the adjective "decoder" is added to distinguish the aforementioned modules from the decoder. Figure 1 The corresponding modules in the video encoder 100 are shown separated.

[0095] The video decoder 400 basically operates as follows. Assume that the video decoder 400 receives a video signal including Figure 3 The coded frame sequence 102 in the data set 300 is shown. Figure 1 The video encoder 100 shown generates. In response, the video decoder 400 provides a decoded frame sequence 409. The decoded frame sequence 409 includes individual decoded frames, which are decoded versions of the encoded frames in the encoded frame sequence 102.

[0096] One or more decoded frames are at least temporarily stored in a decoder frame buffer component 402. The decoder frame buffer component 402 may then provide a representation of a previously decoded frame 410. The representation 410 may be used as a reference for decoding a current encoded frame that is at least partially encoded in an inter-mode.

[0097] The decoding component 401 in the video decoder 400 may decode the first encoded frame portion 307 in a manner that will now be described. The decoding component 401 may decode other frame portions in a similar manner.

[0098] The data unpacking module 403 retrieves the various types of data included in the first coded frame portion 307: the coded picture data 316, the reference data 315 and the data included in the first coded frame portion 307. Figure 3 The reference data 315 may indicate whether the intra mode or the inter mode is applied. If the first coded frame portion 307 is the result of coding in the inter mode, the reference data 315 includes motion vector data. The coded picture data 316 includes a quantized and coded version 120 of the transformed intra / inter-delimited frame portion 118, which is obtained by referring to the above reference. Figure 1The encoding chain 105-111 in the video encoder 100 is provided. The frame buffer compression indication 320 includes an indication 124 of the lossy compression applied by the frame buffer component 104 in the video encoder 100 with respect to the first encoded frame portion 307.

[0099] The entropy decoding module 405 applies entropy decoding to the quantized and encoded version 120 of the transformed intra / inter-defined frame portion 118. Entropy decoding is the inverse operation of the entropy encoding applied by the entropy encoding module 109 in the video encoder 100. Thus, the entropy decoding module 405 reproduces the quantized version 119 of the transformed intra / inter-defined frame portion 118 received by the entropy encoding module 109 in the video encoder 100. In the video encoder 100, the decoding chains 112-114 also receive the quantized version 119 of the transformed intra / inter-defined frame portion 118. The decoder dequantization module 406 and the decoder inverse transform module 407 perform operations corresponding to the operations performed by the dequantization module 112 and the inverse transform module 113 in the decoding chains 112-114 in the video encoder 100, respectively. Thus, the decoder inverse transform module 407 provides a decoded version 122 of the intra / inter-defined frame portion 117.

[0100] If the first coded frame portion 307 is the result of encoding in inter-frame mode, the motion compensation module 404 receives motion vector data included in the reference data 315. The motion vector data indicates a specific portion of a previously decoded frame that is used as a reference for encoding in inter-frame mode. The motion compensation module 404 retrieves the specific portion from a representation 410 of a previously decoded frame provided by the decoder frame buffer component 402. The decoder reconstruction module 408 adds the decoded version 122 of the intra / inter-defined frame portion 117 to the specific portion of the representation of the previously decoded frame retrieved by the motion compensation module 404 from the decoder frame buffer component 402. The addition operation produces a decoded frame portion 411.

[0101] Ideally, the representation 410 of the previously decoded frame provided by the decoder frame buffer component 402 corresponds exactly to the representation 115 provided by the frame buffer component 104 in the video encoder 100. That is, ideally, the video decoder 400 uses a reference that corresponds exactly to the reference used in the video encoder 100. In that case, the decoded frame portion 411 provided by the decoder reconstruction module 408 corresponds exactly to the decoded frame portion 123 provided by the reconstruction module 114 in the video encoder 100 for the associated frame portion.

[0102] Now assume that the reference used by the video decoder 400 for the relevant frame portion is different from the reference used by the video encoder 100. In that case, the decoded frame portion 411 provided by the decoder reconstruction module 408 is different from the decoded frame portion 123 provided by the reconstruction module 114 in the video encoder 100 for the relevant frame portion. The latter decoded frame portion 123 can further be used as a reference for encoding a subsequent frame portion belonging to a subsequent frame in inter-frame mode. This generally results in a greater difference between the reference used by the video decoder 400 and the reference used by the video encoder 100 for the subsequent frame portion belonging to the subsequent frame. Therefore, throughout the consecutive frames, the difference between the references used by the video encoder 100 and the video decoder 400 will increase. This is an accumulation of errors, which reduces the image quality.

[0103] The frame buffer compression indication 320 enables the decoder frame buffer component 402 to provide a reference that closely matches the reference provided by the frame buffer component 104 in the video encoder 100. The indication may even enable the decoder frame buffer component 402 to accurately reproduce the reference provided by the frame buffer component 104 in the video encoder 100. Thus, the frame buffer compression indication 320 allows the accumulation of errors to be reduced, and may even prevent the accumulation of errors, which would otherwise occur due to data compression in the frame buffer component 104 in the video encoder 100. The decoder frame buffer component 402 may apply the same data compression for at least temporary storage of decoded frames. Alternatively, the decoder frame buffer component 402 may store the decoded frames without compressing them, but process the decoded frames in a manner that mimics the data compression applied in the frame buffer component 104 of the video encoder 100.

[0104] Figure 5 An embodiment of a decoder frame buffer component 402 is schematically shown. Figure 5 A schematic block diagram of the present embodiment is provided. For convenience, the present embodiment will be referred to as a decoder frame buffer component 402 hereinafter. The decoder frame buffer component 402 includes a data memory 501 and a controller 502. For ease of distinction, the data memory 501 and the controller 502 will be referred to as a decoder buffer data memory 501 and a decoder buffer controller 502 hereinafter, respectively.

[0105] Like the frame buffer component 104 in the video encoder 100, the decoder frame buffer component 402 includes various functional modules that can be considered to constitute a buffer compression chain 503-505. These functional modules include a decorrelation transform module 503, a quantization module 504, and an entropy coding module 505. For the sake of distinction, these modules will be referred to as a decoder buffer decorrelation transform module 503, a decoder buffer quantization module 504, and a decoder buffer entropy coding module 505, respectively, hereinafter. For the same reason, the buffer compression chain 503-505 in the decoder frame buffer component 402 will be referred to as a decoder buffer compression chain 503-505 hereinafter.

[0106] The decoder frame buffer component 402 also includes functional modules that can be considered to constitute a cache decompression chain 506-508: an entropy decoding module 506, a dequantization module 507, and an inverse transform module 508. For ease of distinction, these modules will be referred to as decoder cache entropy decoding module 506, decoder cache dequantization module 507, and decoder cache inverse transform module 508, respectively, hereinafter. For the same reason, the cache decompression chain 506-508 in the decoder frame buffer component 402 will be referred to as decoder cache decompression chain 506-508 hereinafter. The functional modules in the decoder cache compression chain 503-505 and the decoder cache decompression chain 506-508 can each be implemented in the form of, for example, dedicated circuits, programmable circuits, or appropriately programmed processors, or any combination thereof.

[0107] The decoder frame buffer component 402 operates substantially as follows. Assume that the decoder frame buffer component 402 receives a decoded frame portion 411, which is a decoded frame portion as described above with reference to Figure 4 The decoder reconstruction module 408 provides. Other decoded frame portions resulting from decoding other encoded frame portions may be processed in a manner similar to that described below.

[0108] like Figure 3 As shown, the decoder buffer controller 502 receives the frame buffer compression indication 320 included in the first coded frame portion 307. The decoder buffer controller 502 controls the decoder buffer quantization module 504 to apply the same Figure 2104. The decoder buffer controller 502 may also directly apply the quantization step size or steps if the frame buffer compression indication 320 indicates one or more quantization step sizes to be applied. As described above, the frame buffer compression indication 320 may indicate a target to be achieved for lossy compression in the frame buffer component 104 in the video encoder 100. In that case, the decoder buffer controller 502 uses the target to control the quantization in the decoder buffer quantization module 504 so as to reproduce or mimic the quantization in the buffer quantization module 205 of the frame buffer component 104 in the video encoder 100.

[0109] The decoder buffer decorrelating transform module 503 and the decoder buffer entropy coding module 505 may perform operations similar to those performed by the cache decorrelating transform module 203 and the cache entropy coding module 206, respectively, in the frame buffer component 104 in the video encoder 100. Thus, the decoder buffer decorrelating transform module 503 may apply the decorrelating transform applied by the cache decorrelating transform module 203. The decoder buffer decorrelating transform module 503 applies the decorrelating transform to the decoded frame portion 411 provided by the decoder reconstruction module 408 so as to obtain a transformed decoded frame portion 509. The transformed decoded frame portion 509 is then subjected to quantization in the decoder buffer quantization module 504 as described above.

[0110] The decoder buffer entropy coding module 505 applies entropy coding to the quantized compressed version 510 of the transformed decoded frame portion 509 provided by the decoder buffer quantization module 504. This entropy coding may be similar to that provided by Figure 2 The entropy coding applied by the cache entropy coding module 206 in the frame buffer component 104 is shown. Different entropy coding can also be applied. Therefore, the decoder cache entropy coding module 505 provides a quantized and encoded compressed version 511 of the transformed decoded frame part 509 provided by the decoder cache decorrelating transformation module 503.

[0111] The quantized and coded compressed version 511 of the transformed decoded frame portion 509 provided by the decoder buffer compression chain 503-505 corresponds approximately or even exactly to the compressed decoded frame portion 212 provided by the buffer compression chain 203-206 in the frame buffer component 104 in the video encoder 100. This is due to the frame buffer compression indication 320, which makes the quantization in the decoder buffer compression chain 503-505 correspond approximately or even exactly to the quantization applied in the buffer compression chain 203-206 in the frame buffer component 104 in the video encoder 100. Therefore, the quantized and coded compressed version 511 of the transformed decoded frame portion 509 provided by the decoder buffer compression chain 503-505 will be referred to as the approximately compressed decoded frame portion 511 in the following. The approximately compressed decoded frame portion 511 is sent to the decoder buffer data memory 501 for storage in the decoder buffer data memory 501.

[0112] Figure 5 The decoder buffer decompression chain 506-508 in the decoder frame buffer component 402 shown may perform the same operations as those performed by Figure 2 The operations performed by the cache decompression chain 207-209 in the frame buffer component 104 shown in FIG. 1 are similar. Thus, the decoder frame buffer component 402 can provide a decompressed version 512 of the decoded frame portion 411, which is composed of Figure 4 The decoder reconstruction module 408 is shown to provide and compress in the decoder buffer compression chain 503-505 for storage in the decoder buffer data memory 501.

[0113] The decompressed version 512 of the decoded frame portion 411 is the same as that of Figure 2 The decompressed version 214 of the decoded frame portion 123 provided by the frame buffer component 104 shown corresponds approximately or even exactly. This is also due to the frame buffer compression indication 320, which enables the quantization and inverse quantization in the decoder frame buffer component 402 to correspond approximately or even exactly to the quantization and inverse quantization in the frame buffer component 104 in the video encoder 100.

[0114] Thus, the decoder frame buffer component 402 can provide respective decompressed versions of the previously decoded frame portions, which together constitute a decompressed version of the previously decoded frame. The decompressed versions constitute a representation 410 of the previously decoded frame, which can be used at least in part as a reference for decoding a currently encoded frame that is at least partially encoded in an inter-frame mode. The reference can correspond approximately or even exactly to the reference used in the video encoder 100 to generate the currently encoded frame.

[0115] The embodiments described above are examples of frame buffer compression indications 320 that enable references in the video decoder 400 to correspond to references used in the video encoder 100. As such, at least in the video encoder 100, the references can be compressed to a relatively large extent while ensuring satisfactory image quality. This, in turn, allows the amount of data that needs to be temporarily stored to be reduced, and the bandwidth required to write and read the temporarily stored data to be reduced. This, in turn, reduces costs and power consumption.

[0116] As indicated above, the frame buffer compression indication 320 may be of a direct nature, indicating the resolution of quantization applied by the frame buffer component 104 in the video encoder 100. Thus, the frame buffer compression indication 320 may include, for example, one or more quantization step sizes to be applied by the buffer quantization module 205 when quantizing the associated transformed decoded frame portion. An advantage of such direct indication of the quantization resolution in the frame buffer compression is that the decoder frame buffer component 402 does not need to derive the quantization resolution from an indirect indication of compression (e.g., a compression target as mentioned above). Thus, directly indicating the quantization resolution allows the implementation of the decoder frame buffer component 402 to be relatively simple, and therefore allows the implementation of the video decoder 400 to be cost-effective. This also applies to the above referenced embodiments of the present invention. Figure 1 , Figure 2 , Figure 4 and Figure 5 The embodiments described are different embodiments.

[0117] Figure 6 An alternative video encoder 600 is shown. Figure 6 A schematic block diagram of the alternative video encoder 600 is provided. Figure 1 Similar to the video encoder 100 shown, the alternative video encoder 600 can encode a frame sequence 601 that can constitute a video to obtain an encoded frame sequence 602. Therefore, the encoded frame sequence 602 can constitute an encoded video, which includes a smaller amount of data than the video in its original form. Therefore, the alternative video encoder 600 can also achieve efficient storage of video or efficient transmission of video, or both, by encoding the video. The alternative video encoder 600 can be part of a device that can capture video (e.g., a smartphone), and the alternative video encoder 600 adds value to the device due to its efficiency.

[0118] The alternative video encoder 600 includes an alternative encoding component 603 and an alternative frame buffer component 604. The alternative encoding component 603 includes various functional modules: a linear transformation module 605, an alternative intra / inter mode application module 606, a quantization module 607, an entropy coding module 608, a data packaging module 609 and a controller 610. The aforementioned modules can perform operations related to encoding and can therefore be regarded as forming an encoding chain 605-610. The alternative encoding component 603 also includes functional modules that perform operations related to decoding and can therefore be regarded as forming a decoding chain 611-612: a dequantization module 611 and an alternative reconstruction module 612. The functional modules of the alternative encoding component 603 can each be implemented in the form of, for example, a dedicated circuit, a programmable circuit or a suitably programmed processor or any combination thereof.

[0119] The alternative video encoder 600 operates substantially as follows. The alternative encoding component 603 can operate in a manner similar to that described in patent disclosure WO2020 / 249790. In general, the linear transform module 605 applies a linear transform to the current frame to be encoded. Therefore, the linear transform provides a current frame 613 in the transform domain. The linear transform can be a decorrelation transform, such as a discrete cosine transform (commonly represented by the abbreviation DCT) or a wavelet transform. The linear transform can also be a combination of such a transform and a color transform (for example, transforming a frame consisting of color pixels such as red, green, and blue into a frame consisting of a luminance component and two types of chrominance components). The color transform is usually performed first.

[0120] In practice, the alternative encoding component 603 may divide the current frame 613 in the transform domain into transform sample blocks. The transform sample block constitutes a portion of the current frame 613 in the transform domain, and the transform sample block has a specific position within the current frame 613 in the transform domain. For convenience, this portion of the current frame 613 in the transform domain and this portion of other frames in the transform domain will be referred to as transform domain frame portions hereinafter. The alternative encoding component 603 may encode the respective transform domain frame portions that together constitute the current frame 613 in the transform domain separately in the following manner.

[0121] The transform domain frame portion may be encoded in intra-frame mode or in inter-frame mode, the intra-frame mode and the inter-frame mode being respectively the same as those described above for Figure 1 The intra-mode and inter-mode discussed in the illustrated video encoder 100 are substantially similar. In the inter-mode, a transform domain frame portion is encoded with reference to a representation 614 of a previously encoded frame in the transform domain. The alternative frame buffer component 604 provides a representation 614 of a previously encoded frame in the transform domain 613. In the intra-mode, a transform domain frame portion is encoded without reference to another frame. In this mode, the transform domain frame portion can be encoded "as is".

[0122] The controller 610 can decide whether the transform domain frame portion is encoded in intra-frame mode or in inter-frame mode. The aforementioned patent disclosure WO2020 / 249790 describes a technique for making such a decision. Basically, the decision is based on two criteria. The first criterion is whether the intra-frame mode or the inter-frame mode is expected to be more advantageous in terms of, for example, data compression rate or image quality or both. When the inter-frame mode is expected to be more advantageous, the second criterion applies. The second criterion relates to each corresponding transform domain frame portion belonging to the immediately preceding frame. The adjective "corresponding" refers to a transform domain frame portion whose position in the previous frame in the transform domain corresponds to the position of the corresponding transform domain frame portion in the current frame 613 in the transform domain.

[0123] The second criterion is that all respective corresponding transform domain frame parts belonging to the immediately preceding frame are coded in inter-mode and their number has reached a threshold value. If the second criterion is not met, the decision is based on the first criterion described above. However, if the second criterion is met, the transform domain parts are coded in intra-mode regardless of the first criterion. That is, if the second criterion is met, intra-mode is forced to be applied even if inter-mode is expected to be more advantageous.

[0124] In fact, the second standard constitutes a refresh mechanism that ensures a reasonable delay between the start of decoding of the encoded video and the start of the production of correctly decoded video. This refresh mechanism can also be applied to Figure 6 In a video encoder other than the illustrated alternative video encoder 600. For example, Figure 1 The controller 610 in the illustrated video encoder 100 may apply a refresh mechanism as described above.

[0125] An intra / inter defined transform domain frame portion 615 applied to the quantization module 607 may be provided instead of the intra / inter mode application module 606. In intra mode, the intra / inter defined transform domain frame portion 615 may simply correspond to a transform domain frame portion forming part of the current frame 613 in the transform domain. In that case, the intra / inter defined transform domain frame portion 615 comprises transform samples, which are also present in the current frame 613 in the transform domain.

[0126] In inter-mode, the intra / inter-defined transform domain frame portion 615 is a differential representation of the transform domain frame portion. The differential representation corresponds to the difference between the transform domain frame portion belonging to the current frame on the one hand and the corresponding portion in the representation 614 of the previously encoded frame in the transform domain on the other hand, which representation 614 is provided by the replaceable frame buffer component 604. Likewise, here, the adjective "corresponding" refers to corresponding in position. The differential representation of the transform domain frame portion includes a residual, which reflects the aforementioned difference in the transform domain.

[0127] The quantization module 607 and the entropy coding module 608 may be considered together to form a data compression module that compresses the data included in the intra / inter-frame defined transform domain frame portion 615. The quantization module 607 and the entropy coding module 608 may be used in the same manner as described above, respectively. Figure 1 108 and the entropy coding module 109 in the video encoder 100 shown in FIG. 10A. The controller 610 may operate in a manner similar to that described above with respect to the quantization module 108 and the entropy coding module 109 in the video encoder 100 shown in FIG. Figure 1 The quantization module 607 and the entropy coding module 608 are controlled in a similar manner to the controller 111 in the illustrated video encoder 100. Thus, the entropy coding module 608 provides a quantized and encoded version 616 of the intra / inter defined transform domain frame portion 615, which may include an amount of data that may approximate a target amount of data.

[0128] The data packaging module 609 can be used in the same manner as described above. Figure 1 The data packaging module 609 may operate in a similar manner to the data packaging module 110 in the video encoder 100 shown in FIG. 1 . Thus, the data packaging module 609 may provide a coded transform domain frame portion comprising a quantized and encoded version 616 of the intra / inter bounded transform domain frame portion 615 obtained as described above. The coded transform domain frame portion may also include a mode indication indicating whether the associated transform domain frame portion is encoded in intra mode or in inter mode. The coded transform domain frame portion may also include information about how the intra / inter bounded transform domain frame portion 615 was generated. For example, the information may include information about the resolution of the quantization applied.

[0129] As mentioned above, Figure 6 The alternative video encoder 600 shown comprises the aforementioned decoding chain 611-612 comprising a dequantization module 611 and an alternative reconstruction module 612. The decoding chain 611-612 allows obtaining a decoded version of a current frame 613 in the transform domain encoded in the alternative video encoder 600. The decoding chain 611-612 operates substantially as follows.

[0130] The dequantization module 611 applies inverse quantization to a quantized version 617 of the intra / inter defined transform domain frame portion 615 provided by the quantization module 607 in the encoding chain 605-610. Inverse quantization is an operation that reverses the quantization in the quantization module 607. Thus, the dequantization module 611 provides a dequantized version 618 of the intra / inter defined transform domain frame portion 615. This dequantized version 618 can be considered as a rough representation of the intra / inter defined transform domain frame portion 615 provided by the alternative intra / inter mode application module 606 in the encoding chain 605-610.

[0131] The alternative reconstruction module 612 provides a decoded version 619 of a transform domain frame portion belonging to the current frame, which transform domain frame portion has been encoded in the encoding chain 605-610 as described above. If the inter mode is applied to the transform domain frame portion, the alternative reconstruction module 612 adds a dequantized version 618 of the intra / inter defined transform domain frame portion 615 to the corresponding portion of the representation 614 of the previously encoded frame in the transform domain, which representation 614 is used as a reference. This addition operation results in a decoded version 619 of the transform domain frame portion, which for convenience will be referred to as the decoded transform domain frame portion 619 in the following. If the intra mode is applied to the transform domain frame portion, the alternative reconstruction module 612 may simply use the dequantized version 618 of the intra / inter defined transform domain frame portion 615 as the decoded transform domain frame portion 619.

[0132] The alternative frame buffer component 604 receives the decoded transform domain frame portion 619 provided by the alternative reconstruction module 612, which is the last in the decoding chain 611-612. Therefore, once the current frame is completely encoded, the alternative frame buffer component 604 receives various decoded transform domain frame portions, which can all be obtained in the manner described above. Therefore, the alternative frame buffer component 604 has received a decoded version of the current frame 613 in the transform domain, which for convenience will be referred to as the decoded current frame in the transform domain. Then, when the subsequent frame is to be encoded at least in part in the inter-frame mode, the alternative frame buffer component 604 can provide a representation of the decoded current frame in the transform domain. This representation can then be used as a reference for encoding one or more transform domain frame portions belonging to the subsequent frame in the inter-frame mode.

[0133] as Figure 1The frame buffer component 104 in the video encoder 100 shown, the alternative frame buffer component 604 applies lossy compression. In the alternative video encoder 600, lossy compression is applied to the decoded current frame in the transform domain. The lossy compression provides a compressed version of the decoded current frame in the transform domain. The alternative frame buffer component 604 stores the compressed version at least temporarily. The compressed version of the decoded current frame in the transform domain can then be decompressed, which provides a decompressed version of the decoded current frame in the transform domain. The decompressed version then constitutes a representation of the current frame 613 in the transform domain, which can be used as a reference for encoding one or more transform domain frame portions belonging to subsequent frames in an inter-frame mode.

[0134] as Figure 1 The frame buffer component in the video encoder 100 shown may provide an indication 620 of the applied lossy compression in place of the frame buffer component 604. The data packaging module 609 may include the indication 620 in a data set including the encoded current frame and other encoded frames forming the encoded video.

[0135] Figure 7 An alternative frame buffer component 604 in an alternative video encoder 600 is shown. Figure 7 A schematic block diagram of an alternative frame buffer component 604 in an alternative video encoder 600 is provided. The alternative frame buffer component 604 includes a data memory 701, a controller 702, and various functional modules 703-707. These functional modules include a delay buffer 703, a quantization module 704, and an entropy coding module 705. The aforementioned functional modules can be regarded as forming a buffer compression chain 703-705. The buffer compression chain 703-705 is more Figure 2 The illustrated buffer compression chain in the frame buffer component 104 is simpler because the buffer compression chain 703-705 in the alternative frame buffer component 604 need not include a decorrelating transform module.

[0136] The alternative frame buffer component 604 also includes functional modules that can be considered to constitute a buffer decompression chain 706-707: an entropy decoding module 706 and a dequantization module 707. The buffer decompression chain 706-707 is more Figure 2 The illustrated cache decompression chain in the frame buffer component 104 is simpler because the cache decompression chain 706-707 in the alternative frame buffer component 604 does not need to include an inverse transform module. This is because the alternative video encoder 600 applies the inter-frame mode in the transform domain. Although the alternative frame buffer component 604 is simpler, it can achieve cache compression efficiency similar to that of the frame buffer component 104 in the video encoder 100. The functional modules of the alternative frame buffer component 604 can each be implemented in the form of, for example, dedicated circuits, programmable circuits, or appropriately programmed processors, or any combination thereof.

[0137] For the sake of distinction, the controller 702 in the replaceable frame buffer component 604 will be referred to as the buffer controller 702 hereinafter. For the same reason, the quantization module 704, the entropy encoding module 705, the entropy decoding module 706, and the dequantization module 707 will be referred to as the buffer quantization module 704, the buffer entropy encoding module 705, and the buffer dequantization module 707, respectively, hereinafter. Therefore, the adjective "buffer" is added for the sake of distinction.

[0138] as Figure 2 The alternative frame buffer component shown, the functional module of the alternative frame buffer component 604 can be part of an integrated circuit, and the integrated circuit can also include Figure 6 603 of the alternative video encoder 600 shown. However, the data memory 701 of the alternative cache component 604 can be in the form of a separate circuit, for example, a dynamic random access memory circuit, usually represented by the abbreviation DRAM. Such an embodiment can then include a data write / read channel through which data can be written to the data memory 701 and through which data can be read from the data memory 701. The data write / read channel, which can include a bus, has a bandwidth that limits the amount of data that can be written to the data memory 701 and can be read from the data memory 701 per unit time. That is, the bandwidth defines the maximum data rate for reading and writing.

[0139] The alternative frame buffer component 604 operates substantially as follows. Assume that the alternative frame buffer component 604 receives a decoded transform domain frame portion 619 provided by the alternative reconstruction module 612 as described above. Other decoded frame portions in the transform domain that may belong to the current frame and other frames may be processed in a manner similar to that described below.

[0140] The buffer compression chain 703-705 in the alternative frame buffer component 604 may be implemented in the same manner as described above. Figure 2 The delay buffer 204, the buffered quantization module 205, and the buffered entropy coding module 206 in the compression chain 203-206 in the frame buffer component 104 are shown to operate in a similar manner. Thus, the buffered compression chain 703-705 provides a compressed decoded transform domain frame portion 708. The buffer controller 702 can be used in the same manner as described above with respect to Figure 2 The buffer quantization module 704 is controlled in a manner similar to that of the buffer controller 202 in the frame buffer component 104 shown in FIG. Thus, the buffer controller 702 can provide an indication 620 of the lossy compression applied to the decoded transform domain frame portion 619 to obtain a compressed version of the decoded transform domain frame portion 619.

[0141] Once the current frame is encoded, the cache compression chain 703-705 compresses the individual decoded frame portions in the transform domain, which constitute the decoded current frame in the transform domain. Therefore, the cache compression chain 703-705 produces individual compressed decoded frame portions in the transform domain, which are written to the data memory 701. The individual compressed decoded frame portions constitute a compressed version of the decoded current frame in the transform domain. This compressed version is hereinafter referred to as the compressed decoded current frame in the transform domain. Therefore, the compressed decoded current frame in the transform domain is produced by the lossy compression in the cache compression chain 703-705 described above.

[0142] The cache decompression chain 706-707 in the alternative frame cache component 604 can decompress the compressed decoded current frame in the transform domain to provide a representation of the current frame in the transform domain. The representation can be used as a reference for encoding one or more transform domain frame parts belonging to subsequent frames in inter-frame mode. The cache decompression chain 706-707 can be used in the same manner as described above with Figure 2 The illustrated cached entropy decoding module 207 in the frame buffer component 104 operates in a similar manner with respect to the cached dequantization module 208. Thus, the cached decompression chain 706-707 can provide a decompressed decoded transform domain frame portion 709 based on the compressed decoded transform domain frame portion 708 stored in the data memory 701.

[0143] like Figure 6 As shown, the data packaging module 609 can include an indication 620 of the lossy compression applied by the alternative frame buffer component 604 in a data set that also includes the encoded video. The data set can have a given structure that provides for the inclusion of the indication 620, which for convenience will be referred to as the frame buffer compression indication 620 below. The structure of the data set can be similar to Figure 3 The structure of the data set 300 is shown in FIG. The structure provides one or more specific locations where the frame buffer compression indication 620 may exist so that the video decoder can easily retrieve the indication.

[0144] Figure 8 Shown with Figure 6 An alternative video encoder 600 is shown to be compatible with an alternative video decoder 800 . Figure 8A schematic block diagram of the alternative video decoder 800 is provided. The alternative video decoder 800 includes an alternative decoding component 801 and an alternative frame buffer component 802. For ease of distinction, the alternative frame buffer component 802 will be referred to as an alternative decoder frame buffer component 802 hereinafter. The alternative decoding component 801 includes various functional modules: a data unpacking module 803, an entropy decoding module 804, a dequantization module 805, an alternative reconstruction module 806, and an inverse transform module 807. For ease of distinction, the entropy decoding module 804, the dequantization module 805, and the alternative reconstruction module 806 will be referred to as a decoder entropy decoding module 804, a decoder dequantization module 805, and an alternative decoder reconstruction module 806, respectively, hereinafter. That is, the adjective "decoder" is added to distinguish the aforementioned modules from Figure 6 The corresponding modules in the alternative video encoder 600 are shown separated.

[0145] The alternative video decoder 800 operates substantially as follows. Assume that the alternative video decoder 800 receives a video signal received by Figure 6 The alternative video encoder 600 is shown to generate a sequence of encoded frames 602. In response, the alternative video decoder 800 provides a sequence of decoded frames 808. The sequence of decoded frames 808 includes individual decoded frames, which are decoded frames. Figure 6 The illustrated decoded version of an encoded frame produced by the alternative video encoder 600. When providing a decoded frame, the alternative video decoder 800 also internally produces a decoded frame in the transform domain.

[0146] One or more decoded frames in the transform domain are at least temporarily stored in the alternative decoder frame buffer component 802. The alternative decoder frame buffer component 802 may then provide a representation of the previously decoded frame in the transform domain 809. The representation 809 may be used as a reference for decoding a current coded frame that is at least partially coded in inter mode.

[0147] The alternative video decoder 800 can decode the encoded frame by successively decoding the individual coded transform domain frame portions that make up the encoded frame. The individual coded transform domain frame portions are obtained by decoding the encoded frame as described above with reference to Figure 6 and Figure 7 The described alternative video encoder 600 is generated by encoding various transform domain frame portions.

[0148] The alternative decoding component 801 can decode the coded transform domain frame portion in the following manner. The data unpacking module 803 retrieves various types of data from the coded transform domain frame portion. Therefore, the data unpacking module 803 provides a quantized and encoded version 810 of the intra / inter-frame defined transform domain frame included in the coded transform domain frame portion. The data decompression module 803 further provides a mode indication 811 and a frame buffer compression indication 812. The frame buffer compression indication 812 may correspond to the frame buffer compression indication 620 mentioned above, which corresponds to the frame buffer compression indication 620. Figure 6 The alternative video encoder 600 shown is related to, and more particularly to Figure 7 An alternative frame buffer component 604 is shown.

[0149] The decoder entropy decoding module 804 applies entropy decoding to the quantized and encoded version 810 of the intra / inter-frame defined transform domain frame. Entropy decoding is an inverse operation of the entropy encoding applied by the entropy encoding module 608 in the alternative video encoder 600. Thus, the decoder entropy decoding module 804 reproduces a quantized version 813 of the intra / inter-frame defined transform domain frame. The quantized version 813 can be compared to the quantized version 813 of the intra / inter-frame defined transform domain frame. Figure 6 The decoder dequantization module 805 performs operations corresponding to the operations performed by the dequantization module 611 in the decoding chain 611-612 in the alternative video encoder 600. Thus, the decoder dequantization module 805 provides a dequantized version 814 of the intra / inter-frame defined transform domain frame portion.

[0150] The alternative decoder reconstruction module 806 provides a decoded transform domain frame portion 815 based on the dequantized version 814 of the intra / inter defined transform domain frame portion. If the intra mode is used to generate the encoded transform domain frame portion, the alternative decoder reconstruction module 806 may simply use the dequantized version 814 of the intra / inter defined transform domain frame portion as the decoded transform domain frame portion 815. If the inter mode is used for the transform domain frame portion, the alternative decoder reconstruction module 806 adds the dequantized version 814 of the intra / inter defined transform domain frame portion to the corresponding portion in the representation 809 of the previously decoded frame in the transform domain, which is used as a reference. This addition operation then generates the decoded transform domain frame portion 815.

[0151] Thus, the alternative video decoder 800 generates decoded frames in the transform domain by decoding the respective transform domain frame portions of the encoded frames as described above. The inverse transform module 807 may then apply an inverse transform to the decoded frames in the transform domain in order to obtain a fully decoded frame, which forms part of the sequence of decoded frames 808. The inverse transform corresponds to the inverse operation of the linear transform performed by the linear transform module 605 in the alternative video encoder 600, which has been described above with reference to Figure 6 Furthermore, by decoding each transform domain frame part of the coded frame, each decoded transform domain frame part is obtained. These decoded transform domain frame parts together constitute a decoded frame in the transform domain.

[0152] The alternative decoder frame buffer component 802 can provide a representation of the decoded frame in the transform domain, which is used as a reference later. Ideally, this representation is consistent with the alternative frame buffer component 604 in Figure 6 The representations provided in the alternative video encoder 600 shown correspond. A satisfactory correspondence, or even a perfect correspondence, can be achieved between the aforementioned representations used as references. This is due to the indication 620 of the lossy compression applied by the alternative frame buffer component 604, which is included in one or more frame buffer compression indications in the coded frame sequence 602.

[0153] Fig. 9 An embodiment of an alternative decoder frame buffer component 802 in an alternative video decoder 800 is shown. Fig. 9 A schematic block diagram of this embodiment is provided, and for convenience, this embodiment will be referred to as an alternative decoder frame buffer component 802 hereinafter. The alternative decoder frame buffer component 802 includes a data memory 901 and a controller 902, and for ease of distinction, the data memory 901 and the controller 902 will be referred to as a decoder buffer data memory 901 and a decoder buffer controller 902 hereinafter, respectively.

[0154] Like the alternative frame buffer component 604 in the alternative video encoder 600, the alternative decoder frame buffer component 802 includes various functional modules that can be considered to constitute a buffer compression chain 903-904. These functional modules include a quantization module 903 and an entropy coding module 904. For ease of distinction, the quantization module 903 and the entropy coding module 904 will be referred to as a decoder buffer quantization module 903 and a decoder buffer entropy coding module 904, respectively, hereinafter. For the same reason, the buffer compression chain 903-904 in the alternative decoder frame buffer component 802 will be referred to as a decoder buffer compression chain 903-904, hereinafter.

[0155] The alternative decoder frame buffer component 802 also includes functional modules that can be considered to constitute a cache decompression chain 905-906: a decoder entropy decoding module 905 and a dequantization module 906. For ease of distinction, the decoder entropy decoding module 905 and the dequantization module 906 will be referred to as a decoder cache entropy decoding module 905 and a decoder cache dequantization module 905, respectively, hereinafter. For the same reason, the cache decompression chain 905-906 in the alternative decoder frame buffer component 802 will be referred to as a decoder cache decompression chain 905-906 hereinafter. The functional modules in the decoder cache compression chain 903-904 and the functional modules in the decoder cache decompression chain 905-906 can each be implemented in the form of, for example, dedicated circuits, programmable circuits or appropriately programmed processors, or any combination thereof.

[0156] The alternative decoder frame buffer component 802 operates substantially as follows. Assume that the alternative decoder frame buffer component 802 receives the frame buffer as described above with reference to Figure 8 The depicted alternative decoder reconstruction module 806 provides a decoded transform domain frame portion 815. Other decoded transform domain frame portions (resulting from decoding other encoded transform domain frame portions) may be processed in a manner similar to that described below.

[0157] The decoder buffer controller 902 receives the frame buffer compression indication 812 included in the coded transform domain frame portion as mentioned above. The decoder buffer controller 902 controls the decoder buffer quantization module 903 so that the module applies the same Figure 7 The illustrated quantization is similar to the quantization in the buffer quantization module 704 of the alternative frame buffer component 604. The control may be similar to the control described above in connection with the buffer controller 702 in the alternative frame buffer component 604.

[0158] The decoder buffer entropy coding module 904 applies entropy coding to the quantized compressed version 907 of the decoded transform domain frame portion 815 provided by the decoder buffer quantization module 903. This entropy coding may be similar to that provided by Figure 5 The entropy coding applied by the cache entropy coding module 705 in the alternative frame buffer component 604 is shown. Different entropy coding can also be applied. Therefore, the decoder cache entropy coding module 904 provides a quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 received by the alternative decoder frame buffer component 802.

[0159] The quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 provided by the decoder buffer compression chain 903-904 corresponds approximately or even exactly to the compressed decoded transform domain frame portion 708 provided by the buffer compression chain 703-705 in the alternative frame buffer component 604 in the alternative video encoder 600. This is due to the frame buffer compression indication 812, which enables the quantization in the decoder buffer compression chain 903-904 to correspond approximately or even exactly to the quantization applied by the buffer compression chain 703-705 in the alternative frame buffer component 604 in the alternative video encoder 600. Therefore, the quantized and encoded compressed version 908 of the decoded transform domain frame portion 815 provided by the decoder buffer compression chain 903-904 will be referred to as the approximately compressed decoded transform domain frame portion 908 in the following. The approximately compressed decoded transform domain frame portion 908 is sent to the decoder buffer data memory 901 for storage in the decoder buffer data memory 901.

[0160] Fig. 9 The decoder buffer decompression chain 905-906 in the alternative decoder frame buffer component 802 shown may perform the same operations as those performed by Figure 7 604 in the alternative frame buffer component 604. Thus, the alternative decoder frame buffer component 802 can provide an approximately decompressed decoded transform domain frame portion 909 based on the approximately compressed decoded transform domain frame portion 908 stored in the data memory 901. The approximately decompressed decoded transform domain frame portion 909 can be stored in the data memory 901. Figure 7 The decompressed decoded transform domain frame portion 709 provided by the alternative frame buffer component 604 shown corresponds approximately or even exactly. This is also due to the frame buffer compression indication 812, which enables the quantization and inverse quantization in the alternative decoder frame buffer component 802 to correspond approximately or even exactly to the quantization and inverse quantization in the alternative frame buffer component 604 in the alternative video encoder 600.

[0161] Thus, the alternative decoder frame buffer component 802 can provide respective approximately decompressed decoded transform domain frame portions, which together constitute a decompressed version of a previously decoded current frame in the transform domain. The decompressed version constitutes a representation 809 of the previously decoded frame in the transform domain, which representation 809 can be used at least in part as a reference for decoding a currently encoded frame, which is at least in part encoded in an inter-frame mode. The reference can approximately correspond or even exactly correspond to the reference used in the alternative video encoder 600 to generate the currently encoded frame.

[0162] The embodiment described above is another example of a frame buffer compression indication 812 that causes the reference in the alternative video decoder 800 to correspond to the reference used in the alternative video encoder 600. As such, at least in the alternative video encoder 600, the reference can be compressed to a relatively large extent while ensuring satisfactory image quality. This, in turn, allows a reduction in the amount of data that needs to be temporarily stored, as well as a reduction in the bandwidth required to write and read the temporarily stored data. This, in turn, reduces costs and power consumption.

[0163] Notes

[0164] The embodiments described above with reference to the accompanying drawings are presented by way of example. The present invention can be implemented in many different ways. In order to illustrate this, some alternatives will be briefly pointed out.

[0165] According to the present invention, there are many different ways to implement lossy compression in the frame buffer component in the video encoder. In the embodiment presented above, lossy compression is complementary to entropy coding and entropy decoding. In other embodiments, entropy coding and entropy decoding can be omitted. In the embodiment presented above, the coding chain in the frame buffer component includes a delay buffer. In other embodiments, the coding chain may not include a delay buffer. In addition, there are many different ways to implement lossy compression. For example, lossy compression can be implemented by deleting one or more least significant bits. Such deletion, truncation can be included in, for example, GCLI encoding. In addition, lossy compression can be applied to pixel blocks or transform samples, which do not need to be consistent in size with the pixel blocks or transform samples used for video encoding and decoding.

[0166] There are many different ways to provide an indication of the lossy compression applied in the frame buffer component. In a very basic embodiment, the indication may be in the form of, for example, a flag indicating whether lossy compression is applied. The lossy compression may be predefined so that the flag may enable the decoder to apply the predefined lossy compression or an equivalent lossy compression. In another embodiment, the quantization includes truncation of binary sample values, and the indication of the lossy compression may include the respective truncation levels of the respective parts in the decoded version of the previously encoded frame. In yet another embodiment, the indication may refer to information about the quantization used to encode the frame or frame portion. That is, the lossy compression in the frame buffer component may be associated with the aforementioned quantization according to, for example, a predefined scheme or rule. In other embodiments, the indication may be in the form of, for example, a maximum storage bandwidth for frame buffer compression, or a profile defining one or more storage bandwidths for frame buffer compression. Such a profile may be predefined, allowing relatively simple indications.

[0167] According to the present invention, there are many different ways to implement a decoder compatible with a video encoder. Such a decoder does not necessarily have to store decoded frames used as references in compressed form. Decoded frames can be stored "as is" without compression. When a decoded frame is used as a reference, the decoded frame is processed to imitate the lossy compression applied to the decoded frame in the video encoder. For example, the video decoder can be implemented by a software program running on a computer, and the computer has a relatively large storage capacity and a relatively large storage bandwidth to allow the storage of uncompressed decoded frames. In this embodiment, the software program can provide the above-mentioned processing based on the indication of the lossy compression applied in the video encoder, thereby ensuring that the reference used in the video encoder and the video decoder is the same, or at least similar enough.

[0168] There are many different ways to encode frames or frame portions in inter-frame mode. For example, several previously encoded frames can be used as references. In one embodiment, different frame portions of the same frame can be encoded with reference to frame portions in different frames in a set of previously encoded frames, which can each be used as a reference. In another embodiment, interpolation between two or more previously encoded frames can be used as a reference.

[0169] Generally, according to the present invention, there are many different ways to implement video encoding and video decoding. Any module in the presented embodiments can be implemented by a circuit (which can be a dedicated circuit or a programmable circuit) or a properly programmed processor or a combination thereof. A computer program can define one or more operations described with reference to the presented embodiments. To this end, Figure 1 , Figure 2 , Figure 4-Figure 9 The schematic block diagrams of can also each be regarded at least in part as a flow chart representing such a computer program, and representing a method that a processor can perform when running the computer program. Figure 2 The cache control module 202 in the block diagram of can be regarded as representing a cache control step. Similarly, other modules can be regarded as representing steps of the method.

[0170] The description made above shows that the embodiments described with reference to the accompanying drawings illustrate the invention rather than limiting it. The invention can be implemented in many alternative ways within the scope of the appended claims. All changes within the equivalent meaning and equivalent range of the claims will be included in the scope of the claims. Any reference signs in the claims should not be interpreted as limiting the claims. The verb "comprise" in the claims does not exclude the presence of other elements or steps than those listed in the claims. The same applies to similar verbs such as "comprise" and "contain". When a singular element is mentioned in a claim related to a product, it is not excluded that the product may include multiple such elements. Similarly, when a singular step is mentioned in a claim related to a method, it is not excluded that the method may include multiple such steps. The fact that each dependent claim defines its own additional features does not in itself exclude a combination of additional features other than those reflected in the claim.

Claims

1. A video encoder (100, 600) adapted to encode a frame sequence (101, 601) in order to obtain an encoded frame sequence (102, 602), the video encoder comprising: - a frame buffer component (104, 604) adapted to provide a representation (115, 614) of a previously coded frame for a current frame to be coded, said representation resulting from a lossy compression of a decoded version of said previously coded frame, said lossy compression comprising at least one of the following operations: quantization and truncation of samples of said decoded version of said previously coded frame; as well as an encoding component (103, 603) adapted to encode said current frame with reference at least in part to said representation of said previously encoded frame; - the video encoder is arranged to generate a data set constituting an encoded video, the data set comprising the sequence of encoded frames and comprising an indication (124, 320, 620) of the lossy compression associated with the previously encoded frame, the lossy compression being applied by the frame buffer component to provide the representation of the previously encoded frame, the representation of the previously encoded frame being used at least in part as a reference for encoding the current frame.

2. The video encoder according to claim 1, wherein: The lossy compression is adaptive such that one portion of the decoded version of the previously encoded frame is compressed to a greater extent than another portion of the decoded version of the previously encoded frame.

3. The video encoder according to any one of claims 1 and 2, wherein: The indication (124, 320, 620) of the lossy compression comprises an indication of a target amount of data to be obtained by the lossy compression of the decoded version of the previously encoded frame.

4. The video encoder according to claim 3, wherein: The target amount of data is related to a fixed-size portion of the representation of the previously encoded frame.

5. The video encoder according to any one of claims 1 and 2, wherein: The indication (124, 320, 620) of the lossy compression includes at least one parameter defining a quantization of samples in the decoded version of the previously encoded frame.

6. The video encoder according to claim 5, wherein: The indication (124, 320, 620) of the lossy compression includes a quantization step size.

7. The video encoder according to claim 6, wherein: The indication (124, 320, 620) of the lossy compression includes respective quantization step sizes for respective portions of the decoded version of the previously encoded frame.

8. The video encoder according to any one of claims 5 to 7, wherein: The quantization of the samples includes truncation of binary values, and the indication (124, 320, 620) of the lossy compression includes a truncation level indicating a number of least significant bits that are discarded in the truncation.

9. The video encoder (600) according to any one of claims 1 to 8, wherein: The encoding component (602) includes: - a linear transformation module (605) adapted to apply a linear transformation to said current frame so as to provide said current frame in a transformed domain; - an inter-mode application module (606) adapted to provide a differential representation of a portion of said current frame in said transformed domain, said differential representation corresponding to the difference between said portion of said current frame in said transformed domain on the one hand and a corresponding portion in said representation of said previously coded frame in said transformed domain on the other hand; and - a data compression module (607) adapted to apply data compression to said differential representation of said portion of said current frame in said transform domain; - said frame buffer component (604) being adapted to apply said lossy compression to said decoded version of said previously encoded frame in said transform domain.

10. A video encoder according to claim 9, arranged to encode the further part of the current frame in the transform domain by directly applying the data compression to the further part of the current frame in the transform domain.

11. A method for encoding a frame sequence (101, 601) to obtain an encoded frame sequence (102, 602), the method comprising: - providing a representation of a previously coded frame for a current frame to be coded by lossy compression of a decoded version of the previously coded frame (115, 614), the lossy compression comprising at least one of the following operations: quantization and truncation of samples of the decoded version of the previously coded frame; as well as encoding said current frame with reference at least in part to said representation of said previously encoded frame; - generating a data set constituting an encoded video, the data set comprising the sequence of encoded frames and comprising an indication (124, 320, 620) of the lossy compression associated with the previously encoded frame, the lossy compression being applied to provide the representation of the previously encoded frame, the representation of the previously encoded frame being used at least in part as a reference for encoding the current frame.

12. A computer program for a video encoder, the computer program comprising a set of instructions enabling the video encoder to perform the method according to claim 11.

13. A data set (300) constituting a coded video, the data set comprising: - a sequence of coded frames (102) comprising coded frames (304, 305) coded at least partly with reference to a representation of another previously coded frame (303), said representation being produced by lossy compression of a decoded version of said another previously coded frame by a frame buffer component (104), said lossy compression comprising at least one of the following operations: quantization and truncation of samples of said decoded version of said previously coded frame; as well as - an indication (320) associated with said further previously coded frame regarding said lossy compression applied by said frame buffer component (104) to provide said representation of said further previously coded frame, said representation of said further previously coded frame being used at least in part as a reference for encoding said coded frame.

14. A video decoder (400, 800) adapted to decode a sequence of coded frames to obtain a sequence of decoded frames, the video decoder comprising: - a frame buffer component (402, 802) adapted to provide a representation of previously decoded frames for a coded frame to be decoded; as well as - a decoding component (401, 801) adapted to decode said coded frame at least partly with reference to said representation of said previously decoded frame; - the video decoder is arranged to retrieve from a data set constituting the encoded video an indication of lossy compression contained in the data set (124, 320, 620, 812), the data set comprising the sequence of encoded frames, the lossy compression being applied by a frame buffer component in a video encoder producing the sequence of encoded frames, the lossy compression comprising at least one of the following operations: quantization and truncation of samples of a decoded version of a previously encoded frame; as well as - the frame buffer component of the decoder is suitable for processing the previously decoded frame in a manner equivalent to the lossy compression applied by the frame buffer component in the video encoder, using the indication retrieved from the data set, so as to obtain the representation used at least in part as a reference for decoding the current frame.

15. The decoder according to claim 14, wherein: The decoder is adapted to set at least one parameter of the processing applied to the previously decoded frame in dependence upon the indication (124, 320, 620, 812) of the lossy compression applied by the frame buffer component in the video encoder.

Images