Flexible block allocation structure for image / video compression and processing

Abstract

Techniques for encoding and decoding video may include predicting picture areas defined by time-varying pattern mosaics and / or by spatially varying pattern mosaics within a picture. These techniques improve decoded video quality, for example, by reducing block-based visual artifacts. The pattern mosaic pattern may be spatially irregular to prevent alignment of some prediction region boundaries within a picture. The pattern mosaic pattern may vary over time based on a spatial offset value, and the spatial offset value may be determined by a modulus function. The pattern mosaic pattern may include overlapping shapes, for example, when used in conjunction with overlapping block motion compensation.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 USC 119(e) to U.S. Provisional Application No. 62 / 929,045, filed on October 31, 2019. Background Art

[0003] The present disclosure relates to video coding, and more particularly, to block-based predictive coding.

[0004] Conventional image and video coding schemes typically employ prediction, computation of a residual from the prediction, transformation and quantization of the residual, loop filtering, and entropy coding to produce a compressed bitstream.

[0005] In prediction, a region of the image to be encoded (e.g., a block region of size MxN) is predicted using previously coded samples (spatially and / or temporally) if available or using some "predefined" prediction method or value. Such regions are typically placed on a fixed grid that does not change over time, with large fixed-size super partitions defining the boundaries of the grid, such as macroblocks (MBs) or coding tree units (CTUs). However, it is typically allowed that such super partitions can be further subdivided into sub-partitions of smaller size and different shape than the super partitions. Smaller sub-partitions may include, for example, coding units (CUs), prediction units (PUs), sub-macroblocks, transform units (TUs), etc.

[0006] Any residual remaining after prediction is typically calculated by comparing the prediction with the source video signal (possibly after pre-filtering) corresponding to each individual prediction region.

[0007] This residual may then be transformed using a particular transform, such as a discrete cosine transform (DCT), and then quantized to remove information that is typically perceived as less important to the visualization of the image.

[0008] In-loop filtering (such as deblocking, sample adaptive offset processing, adaptive loop filtering, etc.) may also be applied on such blocks and on their neighboring blocks to reduce possible coding artifacts that may have been introduced by the encoding process.

[0009] Any information needed to construct this block, such as block allocation, prediction mode and its related information (e.g. motion vectors and reference indices), the type of transform or transforms used, quantization parameters, and any remaining quantization coefficients, is then entropy coded and added to the compressed bitstream.

[0010] Decoding the compressed bitstream of a block will reverse the above process to some extent. The decoder can first entropy decode the compressed bitstream to derive the coding information (partitions, modes, motion vectors (MVs), quantization coefficients, etc.), and the entropy decoding can be followed by a prediction process. The inverse quantization and transform steps can be performed in parallel, and then the reconstructed residual is added to the prediction information. Given an appropriate in-loop filtering mechanism supported and used by the underlying codec, the reconstructed block can then be in-loop filtered. Summary of the invention

[0011] Techniques for encoding and decoding video may include predicting picture regions defined by time-varying mosaics and / or by spatially varying mosaics within the picture. These techniques improve decoded video quality, for example, by reducing block-based visual artifacts.

[0012] In one aspect, a coding technique may include determining a first spatial allocation; predictively encoding a first frame of a sequence of source video frames, including predicting a partition of the first frame based on the first spatial allocation; determining a second spatial allocation different from the first spatial allocation; predictively encoding a second frame of the sequence of source video frames, including predicting a partition of the second frame based on the second spatial allocation; and sending an encoded video sequence including the first encoded frame, the second encoded frame, and indications of the first spatial allocation and the second spatial allocation.

[0013] On the other hand, a decoding technique may include a method for decoding a video, the method comprising: extracting an indication of a first spatial allocation from a coded video sequence; predictively decoding a first frame of the coded video sequence, including predicting a partition of the first frame based on the first spatial allocation; extracting an indication of a second spatial allocation different from the first spatial allocation from the coded video sequence; predictively decoding a second frame of the coded video sequence, including predicting a partition of the second frame based on the second spatial allocation; and sending a coded video sequence including the first coded frame, the second coded frame, and indications of the first spatial allocation and the second spatial allocation.

[0014] In another aspect, an encoding technique may include determining a pattern mosaic for each corresponding portion of a source video, wherein the pattern mosaic defines a prediction region for the corresponding portion and wherein the pattern mosaic varies between the portions; predictively encoding the portions including predicting the prediction region defined by the corresponding pattern mosaic of the portions; and storing an encoded video including the encoded portions and an indication of the varied pattern mosaic.

[0015] On the other hand, a decoding technique may include extracting from an encoded video stream an indication of a prediction region for each corresponding portion of a video in the encoded video stream, wherein the pattern mosaic varies between the portions; decoding the portion using a predictive decoding technique, including predicting the prediction region defined by the corresponding pattern mosaic of the portion; and outputting the decoded portion.

[0016] As used herein, "tessellation" may refer to overlapping or non-overlapping shapes covering an area being tessellated, such as in conjunction with overlapping block motion compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 An exemplary simplified block diagram of a video delivery system is shown.

[0018] Figure 2 is a functional block diagram illustrating components of an exemplary encoding terminal.

[0019] Figure 3 is a functional block diagram illustrating components of an exemplary decoding terminal.

[0020] Figure 4A is an exemplary encoding flow chart 400 .

[0021] Figure 4B is an exemplary decoding flow chart 450 .

[0022] Figure 5A An exemplary conventional allocation grid is shown.

[0023] Figure 5B An exemplary horizontally shifted allocation grid is shown.

[0024] Figure 5C An exemplary vertically and horizontally shifted allocation grid is shown.

[0025] Figure 5D An exemplary allocation grid with variable-sized partitions is shown.

[0026] Figure 5E An exemplary allocation grid is shown.

[0027] Fig. 5F An exemplary allocation grid is shown.

[0028] Figure 5G An exemplary allocation grid is shown.

[0029] Figure 5H An exemplary allocation grid is shown.

[0030] Figure 6 An exemplary allocation grid is shown.

[0031] Figure 7 A timeline showing the distribution changes over time.

[0032] Fig. 8A An exemplary allocation grid for use with overlapped block motion compensation (OBMC) is shown.

[0033] Figure 8B An exemplary allocation grid for use with OBMC is shown.

[0034] Figure 8C An exemplary allocation grid for use with OBMC is shown.

[0035] Fig. 9 is an exemplary functional block diagram of an exemplary encoding system 900 .

[0036] Fig.10 is an exemplary functional block diagram of an exemplary decoding system 1000 . DETAILED DESCRIPTION

[0037] Figure 1 A simplified block diagram of a video delivery system 100 according to aspects of the present disclosure is shown. The system 100 may include a plurality of terminals 110, 120 interconnected via a network. The terminals 110, 120 may encode video data for transmission to their corresponding terminals via the network. Thus, a first terminal 110 may capture video data locally, encode the video data, and transmit the encoded video data to a corresponding terminal 120 via a channel. The receiving terminal 120 may receive the encoded video data, decode it, and present it locally, such as on a display at the terminal 120. If the terminals are engaged in a two-way exchange of video data, the terminal 120 may capture video data locally, encode the video data, and transmit the encoded video data to the corresponding terminal 110 via another channel. The receiving terminal 110 may receive the encoded video data transmitted from the terminal 120, decode it, and present it locally, such as on its own display. The processes described herein may operate both frame picture and interlaced field picture encoding, but for simplicity, this discussion will describe the techniques in the context of whole frames.

[0038] The video encoding system 100 can be used in a variety of applications. In a first application, the terminals 110, 120 can support real-time bidirectional exchange of encoded video to establish a video conferencing session between them. In another application, the terminal 110 can encode pre-generated video (e.g., a television or movie program) and store the encoded video for delivery to one or often to multiple clients (e.g., terminal 120) for downloading. Therefore, the video being encoded can be real-time or pre-generated, and the terminal 110 can act as a media server to deliver the encoded video according to a one-to-one or one-to-many distribution model. For the purpose of this discussion, unless otherwise specified, the type of video and the video publishing scheme are irrelevant.

[0039] exist Figure 1 In the embodiment, terminals 110 and 120 are respectively illustrated as personal computers and smart phones, but the principles of the present disclosure are not limited thereto. Aspects of the present disclosure are also applied to various types of computers (desktop computers, laptop computers, and tablet computers), computer servers, media players, dedicated video conferencing equipment, and / or dedicated video encoding equipment.

[0040] Network 130 represents any number of networks that communicate encoded video data between terminals 110, 120, including, for example, wired communication networks and / or wireless communication networks. Communication networks can exchange data in circuit switching channels and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, unless otherwise indicated, the architecture and topology of the network are immaterial to the operation of the present invention.

[0041] Figure 2 2 is a functional block diagram illustrating components of an encoding terminal 200 according to one aspect of the present disclosure. The encoding terminal may include a video source 210, an image processor 220, an encoding system 230, and a transmitter 240. The video source 210 may supply a video to be encoded. The video source 210 may be provided as a camera that captures image data of a local environment, a storage device that stores video from some other source, or a network connection by which source video data is received. The image processor 220 may perform signal conditioning operations on the video to be encoded to prepare the video data for encoding. For example, the preprocessor 220 changes the frame rate, frame resolution, and / or other characteristics of the source video. The image processor 220 may also perform filtering operations on the source video.

[0042] The encoding system 230 may perform encoding operations on the video to reduce its bandwidth. Typically, the encoding system 230 exploits temporal and / or spatial redundancy within the source video. For example, the encoding system 230 may perform motion compensated predictive encoding, in which a video frame or field is parsed into sub-units (referred to as "pixel blocks" for convenience), and each pixel block is differentially encoded relative to a predicted pixel block, which is derived from previously encoded video data. A given pixel block may be encoded according to any of a variety of predictive coding modes, such as:

[0043] Intra-frame coding, where an input pixel block is differentially encoded relative to previously encoded / decoded data of a common frame;

[0044] · uni-predictive inter-frame coding, where the input pixel block is differentially encoded with respect to data from previously encoded / decoded frames; and

[0045] • Multi-hypothesis motion compensated predictive coding, where an input pixel block is predictively encoded via temporal or spatial prediction using decoded data from two or more sources.

[0046] The predictive coding mode can be used in cooperation with other coding techniques such as transform skip coding, reduced resolution update (RRU) coding, scaling of prediction sources, palette coding, etc.

[0047] The coding system 230 may include a forward encoder 232, a decoder 233, a loop filter 234, a picture buffer 235, and a predictor 236. The encoder 232 may apply a differential coding technique to an input pixel block using the predicted pixel block data provided by the predictor 236. The decoder 233 may invert the differential coding technique applied by the encoder 232 to a subset of the coded frames designated as reference frames. The loop filter 234 may apply a filtering technique to a reconstructed reference frame generated by the decoder 233. The picture repository 235 may store the reconstructed reference frame for prediction operations. The predictor 236 may predict the data of the input pixel block from within the reference frame stored in the picture repository.

[0048] The transmitter 240 may transmit the encoded video data to a decoding terminal through a channel.

[0049] Figure 3 3 is a functional block diagram illustrating components of a decoding terminal 300 according to one aspect of the present disclosure. The decoding terminal may include a receiver 310 for receiving encoded video data from a channel, a video decoding system 320 for decoding the encoded data, a post-processor 330, and a video sink 340 for consuming the output video data.

[0050] Receiver 310 may receive a data stream from a network and may route components of the data stream to appropriate units within terminal 300. Figure 2 and Figure 3 The functional units for video encoding and decoding are shown, but the terminals 110, 120 ( Figure 1 ) will typically include an encoding / decoding system for audio data associated with the video, and may include other processing units (not shown). Thus, receiver 310 can parse the encoded video data from other elements of the data stream and route it to video decoder 320.

[0051] The video decoding system 320 may perform a decoding operation that reverses the encoding operation performed by the encoding system 230. The video decoder may include a decoder 322, a loop filter 324, a picture buffer 326, and a predictor 328. The decoder 322 may reverse the differential encoding technique applied by the encoder 232 into an encoded frame. The loop filter 324 may apply filtering techniques to the reconstructed frame data generated by the decoder 322. For example, the loop filter 324 may perform various filtering operations (e.g., deblocking, de-ringing filtering, sample adaptive offset processing, etc.). The filtered frame data may be output from the decoding system. The picture repository 326 may store reconstructed reference frames for prediction operations. The predictor 328 may predict data of an input pixel block from a reference frame stored by the picture repository based on the prediction reference data provided in the encoded video data.

[0052] The post-processor 330 may perform operations to adjust the reconstructed video data for display. For example, the post-processor 330 may perform various filtering operations (e.g., deblocking, de-ringing filtering, etc.), which may blur visual artifacts in the output video generated by the encoding / decoding process. The post-processor 330 may also change the resolution, frame rate, color space, etc. of the reconstructed video to conform to the requirements of the video sink 340.

[0053] The video sink 340 represents various hardware and / or software components in the decoding terminal that can consume the reconstructed video. The video sink 340 may generally include one or more display devices that can present the reconstructed video. Alternatively, the video sink 340 may be represented by a memory system that stores the reconstructed video for later use. The video sink 340 may also include one or more applications that process the reconstructed video data according to controls provided in the application. In some aspects, the video sink may represent a transmission system that transmits the reconstructed video to a display on another device separate from the decoding terminal; for example, the reconstructed video generated by a notebook computer may be transmitted to a large flat panel display for viewing.

[0054] For encoding terminal and decoding terminal ( Figure 2 and Figure 3 ) shows the operations performed to encode and decode video data in a single direction between terminals, such as from terminal 110 to terminal 120 ( Figure 1In applications where a bidirectional exchange of video is to be performed between the terminals 110, 120, each terminal 110, 120 will have a connection to the encoding terminal ( Figure 2 ) and each terminal 110, 120 will have a functional unit associated with the decoding terminal ( Figure 3 ) associated functional units. In fact, in some applications, the terminals 110, 120 may exchange multiple streams of encoded video in a single direction, in which case a single terminal (such as terminal 110) will have an encoding terminal ( Figure 2 Such specific implementations are fully consistent with this discussion.

[0055] Figure 4A is an exemplary encoding flow chart 400 . Figure 4A An initial mosaic is included that determines a prediction region (block 412). Next, for each portion of the video to be encoded, the image region defined by the mosaic is predictively encoded (block 414), and the encoded video portion is stored or transmitted along with an indication of the mosaic. If there are more video portions to be encoded (block 418), the mosaic may be changed (block 422), and the partial encoding process repeated.

[0056] A mosaic (as determined in blocks 412 and 422) may be an arrangement of two-dimensional shapes that fit closely together and that cover an area of ​​a frame (or other portion of a video), for example, the shapes may be polygonal and the arrangement may be a repeating pattern of one or more polygons without any gaps. A mosaic may define the allocation of image data. As used herein, a "mosaic" may refer to overlapping or non-overlapping shapes that cover an area being mosaicked. For example, overlapping mosaics may be used with overlapped block motion compensation (OBMC). A mosaic may be based on a regular or uniform block structure where all blocks are the same shape and size, or may be irregular where block size or shape varies within a mosaic pattern of a frame (or another portion of a video that is smaller than a frame). An irregular spatial pattern may allow some block boundaries of adjacent blocks to be misaligned. Such spatial irregularity may reduce visual blocking artifacts. Examples of spatial pattern mosaics for allocating prediction areas are described below with respect to FIG. 5A to FIG. 5H , Figure 6 and FIG. 8A to FIG. 8C To give.

[0057] Predictive encoding of a portion of a video (block 414) may include predicting pixels in each spatial region defined by a mosaic according to a selected prediction mode (such as an inter-frame or intra-frame prediction mode). An indication of the mosaic may be stored (block 416) with the encoded video portion in a single encoded bitstream that includes the mosaic and, for example, other prediction parameters (such as motion vectors and coding residuals). The mosaic may be altered for separate portions of the video (block 422) such that the mosaic pattern may shift over time or may otherwise vary over time. In some aspects, the change in the mosaic may be selected to reduce or minimize the number of overlapping or coinciding region boundaries between portions of the video. Such shifts or other mosaic pattern changes that occur over time may reduce blocking artifacts.

[0058] In an optional aspect, the tessellation can be based on a spatial offset value, and the tessellation can be changed over time by changing the offset value. In optional block 410, an initial one-dimensional or two-dimensional offset can be determined, which is then used to determine the initial tessellation in block 412. In optional block 420, a new offset can be determined, which is then used to determine the changed tessellation in block 422.

[0059] Figure 4B is an exemplary decoding flow chart 450 . Figure 4B An initial mosaic of patterns is included to determine the prediction area (block 464). Then, for each portion of the video to be decoded, the image area defined by the mosaic is predicted to decode the video portion (block 466), and the decoded video portion is output (block 468). If more video portions are to be decoded (block 470), the mosaic may be changed (block 474) and the partial decoding process repeated.

[0060] The pattern mosaic determined in blocks 412 and 422 may be similar to the one described above with respect to Figure 4A The prediction decoding of block 466 may include the above description of Figure 4A The corresponding part of the predictive coding described in the encoding.

[0061] In an optional aspect, one or more mosaic parameters may be extracted from the encoded video stream (block 469). An initial one-dimensional or two-dimensional offset may be determined (optional block 462), which may then be used to determine the initial mosaic in block 464. A new offset may be determined (optional block 420), which may then be used to determine the altered mosaic in block 422. Examples of mosaic parameters may be an initial offset value, a delta offset value for altering the mosaic between portions, or an indication of a function for determining an altered offset value.

[0062] Non-static allocation

[0063] As discussed above, conventional block-based coding systems can operate on blocks of a shape and size defined by a static grid. Unfortunately, the fact that the coding area is placed on a fixed grid may have an adverse effect on the coding quality. Specifically, such grids can cause compression artifacts, and more specifically, blockiness / blocking artifacts that can become quite scattered and visible, especially at certain quality / compression levels. Since such grids do not change over time, blockiness artifacts may accumulate and deteriorate over time. Some methods for mitigating such artifacts can be adopted from many codecs, including the use of in-loop or out-of-loop deblocking, adaptive loop filtering (ALF), and even overlapping block prediction. In the latter case, multiple blocks are allowed to overlap, and prediction of the region is performed by weighted averaging the predictions contributed from each block. This can help reduce some of the blockiness artifacts in the blockiness artifacts. However, such methods can also add considerable complexity to both the encoder and the decoder. Other methods can also mitigate some of the artifacts in the artifacts, but are not always effective because they are controlled by certain characteristics of the encoding process, which do not guarantee that all blockiness artifacts will be identified and therefore resolved by such schemes. It should be noted that the techniques described herein are applicable to both intra and inter prediction methods, including directional intra prediction, palette mode, intra block copy prediction, fractal prediction, translational or affine prediction, weighted prediction, warped prediction, and the like.

[0064] Improved coding schemes can further mitigate such coding artifacts by better distributing block partitions in time and thus reducing blockiness propagation. Figure 5A An exemplary conventional allocation grid 510 for allocating image frames into coding regions is shown. As mentioned previously, existing schemes use fixed block partitioning schemes such as Figure 5A In one aspect of the present disclosure, an adaptive / non-fixed block partitioning scheme is used, the goal of which is to reduce block propagation around edges. In this scheme, and for each different frame, picture, slice or tile of a picture, a different allocation scheme is basically expected or indicated. In particular, in existing schemes, the allocation of images or slices / tiles is usually performed using square partitions of equal size (e.g., MxM pixels), such as Figure 5A As shown, starting from the top left corner of the image or slice / tile to be encoded. If the resolution of the image or slice / tile is not a multiple of M, appropriate padding can be applied on the right or bottom border of the partition. Unlike such conventional methods, the improved technique allows the use of partitions of different sizes within the same area, and the partitions of different sizes can be spatially arranged in such a way that the arrangement covers the entire image. An example of such an allocation is shown in Figure 5B and Figure 5C Among them, Figure 5B In , the allocation grid 520 is shifted toward the left, while in Figure 5C , the allocation grid 530 is shifted toward both the left side and the top side. Figure 5B and Figure 5C The allocation scheme uses a regular rectangular grid, however this grid includes horizontal and / or vertical offsets that control its starting position.

[0065] A single allocation with a single regular grid of horizontally and / or vertically offset starting positions (such as in Figure 5B and Figure 5C In some aspects, the allocation for prediction may be independent of the allocation for block-based residual coding (which may include block-based transform and quantization). Such artifacts may be reduced if the prediction allocation varies sufficiently in time and avoids temporal alignment of most (if not all) residual encoder super partitions (e.g., MB or CTU level) and sub-partitions (sub-macroblocks, CUs, PUs, etc.). For example, super partitions are typically subdivided into sub-partitions in a hierarchical manner and are primarily subdivided using powers of two sub-partitions. For example, MxM partitions are typically subdivided into partitions of size (M / 2)xM, Mx(M / 2), (M / 2)x(M / 2), (M / 4)x(M / 2), (M / 4)x(M / 4), etc. It may be advantageous to use horizontal and / or vertical offsets of prediction partitions, which will result in as little conflict as possible with prediction partitions used in previously encoded frames. It should be noted that odd shift / offset values ​​may be undesirable since the most commonly used image format is the 4:2:0 YUV format, which specifies that the chrominance / color components are downsampled in both the vertical and horizontal directions before encoding. In this case, an odd value offset will result in an improper / undesirable allocation of the chrominance components, i.e., at sub-pixel locations, which may affect the operation of the codec. Therefore, in a preferred embodiment, only even-valued assignments to the offset are allowed.

[0066] The time-varying allocation can be determined by the spatial offset. In one aspect, the offset value can be implicit, such as based on the coding or temporal order, and no explicit signaling is required. For example, for a frame at display or coding order t and assuming only horizontal offsets, the offset can be calculated as 2*modulo(t,N), where N is a predetermined number known to both the encoder and the decoder. If for two dimensions, we can use different functions to calculate the offset for the horizontal offset x_offset and the vertical offset y_offset, as follows:

[0067] x_offset=f0(t,N)

[0068] y_offset=f1(t,M)

[0069] Where f0() and f1() are two functions, and M and N may be some constants. f0() and f1() may be modulo operations or some other functions. Other control parameters may also be used to determine the function of the allocation offset.

[0070] The allocation offset function and control parameters (such as M and N) can be fixed or can be signaled in the bitstream. In one aspect, signaling may be sufficient in the sequence parameter set (SPS) of the stream, i.e., constant for the entire stream. Alternatively, they can also change per frame. In this case, it may not be guaranteed that the offset changes from one frame to the next, but it may still be desirable in some applications. However, on the other hand, the allocation offset can be directly signaled per frame, per slice, or per tile of the picture without the need for the decoder to determine the offset by an offset function. This explicit signaling may be more efficient for compression efficiency and decoder calculations. In such cases, the encoder can intelligently decide what offset to use for each frame (or slice or tile) based on the encoding target quality, the characteristics of the region in the image, complexity, and other factors. For example, if the image is quite static and the encoding quality is quite high, it may be preferable not to change the allocation scheme from one frame to another because there will be a large blocking effect in the scene.

[0071] On the other hand, the allocation can be based on image content analysis. For example, the prediction allocation can be aligned based on the presence and characteristics of edges in the content, the characteristics of textures, and other important features within the image being encoded (including motion, color, and object shape, etc.). Keeping the allocation consistent in this case will make the encoder operation easier because information from previously encoded frames can be reused. However, if the image or encoding quality degrades, it may be preferable to adjust the offsets used from one frame to the next.

[0072] Uneven distribution of space

[0073] Discussed above Figure 5B and 5C Examples of allocations include a regular but shifted rectangular grid where all rectangles in the grid have uniform shape and size (after padding the partition areas at the edges of the image). On the other hand, the allocation may not be regular and may not be based on uniform block sizes and shapes. For example, in a particular case, in one frame, all CTUs at even vertical index positions may have size MxM, while all odd vertical index positions may have size NxN. In another frame, the reverse order may be used. In another case, partitions within the same row may have different sizes or shapes, and the arrangement may vary from one row to the next and from one frame, slice, or tile of the picture to another. Such an exemplary allocation 540 is shown in Figure 5D middle.

[0074] Figure 5E and Fig. 5F Exemplary allocation grids 550 and 560 are shown, respectively. These figures show allocation schemes that combine two super partitions of size MxM and (M / 2)x(M / 2). Such allocations reduce some of the partition edges and may be friendly to hardware implementations. Allocation grids 550 and 560 include square partition area shapes with various partition area sizes. Figure 5E Grid 550 includes a mixed allocation of MxM and (M / 2)x(M / 2) super partitions with MxM starting offsets. Fig. 5F Grid 560 includes a mixed allocation of MxM and (M / 2)x(M / 2) super partitions with (M / 2)x(M / 2) starting offsets.

[0075] Figure 5G and Figure 5H Exemplary allocation grids 570 and 580 are shown and are Figure 5E and Fig. 5F A variant form of . Figure 5G A mixed allocation of MxM and (M / 2)x(M / 2) super partitions is shown. Figure 5H shows a mixed allocation of MxM and (M / 2)x(M / 2) super partitions, but has the same Figure 5G The super partitions in different orders.

[0076] In one aspect, for example, as described above with respect to Figure 5B and Figure 5C As described, it is possible to combine time-varying spatial offsets using e.g. FIG. 5D to FIG. 5H The space is unevenly distributed.

[0077] In other aspects, "block-based" allocation need not be limited to rectangular partition areas. Diamond, triangular, trapezoidal or other polygonal partitions may also be used in codecs that support such partition types. If such allocation patterns are utilized, according to the present invention, it will be desirable to vary the allocation pattern from one frame to the next in order to avoid accumulating errors at the edges of each partition.

[0078] Figure 6An exemplary allocation grid 600 is shown. The allocation grid 600 includes spatial variations in both the shape (square and non-square rectangles) and size of the partitioned areas. The partitioned areas can be equivalent to the CTU concept in HEVC / VVC, and can be further divided into sub-partitions. In this regard, the maximum partition is limited. It should also be noted that large partitions can still be "split" in a way that still creates a single long line (e.g., a large square (green) partition can be split into two vertical partitions in the middle), but such splitting can be decided at the encoder, and it is hoped that the encoder will avoid (or be opposed to) using such allocations during mode decision.

[0079] In one aspect, a spatial allocation grid in which partition boundaries are discontinuous across an image (or across a slice or other portion of a video) can reduce visual blocking artifacts perceived by humans. For example, structures of discontinuous blocking artifacts can have less visual structure and, therefore, may be less likely to be perceived. Figure 6 In the example of , the vertical partition boundaries on the right and left sides of each (green) large square partition do not intersect with the vertical partition boundaries of the vertically adjacent large square partitions. In this way, the vertical partition boundaries of the adjacent large square partitions are discontinuous.

[0080] Figure 7 A timeline 700 showing changes in allocation over time is shown. For example, time may be measured in frames in encoding or presentation order. Alternatively, Figure 7 The time in may be a sequence of slices or other parts of a video sequence.

[0081] In one aspect, a fixed set of known allocations may be alternating over time (or across slices or other portions of a video stream). For example, a modulus function based on an input frame number may determine which fixed set of known allocations to use. Figure 7 As shown in FIG. 1 and assuming that the allocation changes every frame, the modulus function for the first frame may determine to use the known allocation 0, and then for the following five frames, the known allocations are used in the order 1, 2, 3, 0, 4. A set of such predetermined known allocations may be determined arbitrarily. Alternatively, the predetermined known allocations may be related to each other, for example they may all be derived from one spatial allocation with a varying spatial offset.

[0082] Overlapped Block Motion Compensation

[0083] Fig. 8A , Figure 8B and Figure 8C 8 and 9 show exemplary allocation grids 810, 820, and 830 for use with overlapped block motion compensation (OBMC), respectively. The various allocation schemes described above can be generalized by combining these schemes with concepts such as OBMC. In this regard, the partitioned regions can overlap ( Fig. 8A). In the case of overlapping prediction partitions, each predicted pixel may be determined as a weighted combination of corresponding pixels predicted by each adjacent prediction region. Fig. 8A , Figure 8B , Figure 8C Region A in may correspond to a region that can be predicted from a single reference, while region B may be predicted from a weighted combination of references used in two neighboring regions A, and region C may be predicted from a weighted combination of references used in four neighboring regions A. In one aspect, only the size of the prediction partitions may be adjusted while maintaining the size of the overlap constant ( Figure 8B ). Grid 820 includes OBMCs with different CTU sizes at even and odd horizontal positions. On the other hand, the amount of overlap between adjacent partition regions may also vary from one partition region to the next ( Figure 8C ). In grid 830, an OBMC with partitions utilizing weighted regions of different sizes (only horizontal variations) is shown.

[0084] The allocation may be signaled in the compressed bitstream. Likewise, there may be signaling in the bitstream about how such partitions are arranged. Such signaling may be present at various levels of the bitstream, e.g., sequence parameter set (SPS), picture parameter set (PPS), slice or tile header, etc., depending on the control that may be desired over the allocation or the resulting overhead (SPS and PPS information is signaled less frequently and shared across multiple pictures; slice and / or tile headers are typically signaled for each slice or tile in the stream).

[0085] In other aspects, various encoding operations can be adjusted based on the various partitioning schemes described herein. For example, deblocking and adaptive loop filtering can be adjusted based on the allocation scheme used for each different frame (or other part of the video). This can be beneficial for the encoding operation, having the effect of reducing the blocking effect caused by the allocation. In these aspects, the type or location of these operations (such as deblocking) can be determined in part based on the allocation.

[0086] Any complexity caused by handling partitions of different sizes can be alleviated by limiting the size. Prediction and transform blocks can be limited to certain sizes that may be supported by the system, such as 4x4, 8x8, 16x16, 32x32, 64x64, etc. Phantom partition areas created by padding non-supported partition sizes to supported sizes can reduce complexity. For example, more flexible allocations can be handled by allowing boundaries (the first and last partitions on each row and column of an image, slice, or tile) to adopt more flexible sizes. Those can be explicitly encoded by an encoder that supports the partition size, or alternatively, those partitions can be handled by padding the non-supported size to the nearest supported size (such as the coding tree unit size). For example, if an offset of 10 is used in the horizontal direction, and assuming a CTU size of 64, it is not necessary to process a 10x64 block partition on the left side of the image, but the partition can be padded to a resolution of 16x64 or 64x64 by adding samples on the left side of the partition, and then encoded using conventional prediction and transform methods. During decoding, the block is first decoded at the padded resolution (i.e. 16x64 or 64x64) and then trimmed to the desired size by discarding the padded samples on the left side. A similar approach can be done on non-CTU size partitions on the right, top or bottom side of a picture, slice or tile group.

[0087] In one aspect, block-based methods are also used for video processing, such as de-interlacing and motion compensated denoising (e.g., motion compensated temporal filtering / MCTF). The allocation methods discussed herein can also be applied in the context of such operations, since these block-based methods may result in blocking artifacts if the operation grid remains fixed. Adjusting the operation grid per region or frame for performing such operations, similar to that described herein for image / video compression systems, can help reduce blocking artifacts.

[0088] Encoding using pattern mosaic

[0089] Fig. 9 9 is a functional block diagram of an encoding system 900 according to one aspect of the present disclosure. The system 900 may include a pixel block encoder 910, a pixel block decoder 920, a picture store 930, a loop filter system 930, a reference frame store 950, a predictor 960, a controller 970, and a syntax unit 980. The predictor 960 may generate a prediction block for use by the pixel block encoder 910 and the pixel block decoder 920. Prediction Block can be a prediction of a newly presented block s of the input source video, and the prediction block The pixel block encoder 910 may encode the new pixel block by predictive coding techniques and present the encoded pixel block data to the syntax unit 980. The pixel block decoder 920 may decode the encoded pixel block data, thereby generating decoded pixel block data. The loop filter 940 may perform one or more filtering operations on the reconstructed frame. For example, the loop filter 940 may perform deblocking filtering, sample adaptive offset (SAO) filtering, adaptive loop filtering (ALF), filtering schemes based on maximum likelihood (ML), de-ringing, band removal, sharpening, resolution scaling, etc. The reference frame repository 950 may store filtered frames, which may be used as a prediction source for pixel blocks received later. The syntax unit 980 may combine data streams from encoded pixel block data that conform to the management coding protocol.

[0090] The pixel block encoder 910 may include a subtractor 912, a transform unit 914, a quantizer 916, and an entropy encoder 918. The pixel block encoder 910 may accept a pixel block s of input data at the subtractor 912. The subtractor 912 may receive a predicted pixel block from the predictor 960. And thus generate the input pixel block s and the predicted pixel block 914 may apply a transform to the sample data output from the subtractor 912 to convert the data from the pixel domain to the transform coefficient domain. A quantizer 916 may perform quantization on the transform coefficients output by the transform unit 914. The quantizer 916 may be a uniform or non-uniform quantizer. An entropy encoder 918 may reduce the output bandwidth of the coefficient quantizer by, for example, losslessly encoding the output with a variable length codeword or using a context adaptive binary arithmetic encoder.

[0091] The transform unit 914 may operate in a variety of transform modes determined by the controller 970. For example, the transform unit 914 may apply a discrete cosine transform (DCT), a discrete sine transform (DST), a Walsh-Hadamard transform, a Haar transform, a Daubechies wavelet transform, etc. In one aspect, the controller 970 may select a coding mode M to be applied by the transform unit 915, may configure the transform unit 915 accordingly, and may signal the coding mode M explicitly or implicitly in the encoded video data.

[0092] The quantizer 916 may operate according to a quantization parameter QP provided by the controller 970. In one aspect, the quantization parameter QP may be applied to the transform coefficients as a multi-valued quantization parameter that may vary, for example, at different coefficient positions within a transform domain pixel block. Thus, the quantization parameter QP may be provided as a quantization parameter array.

[0093] The entropy encoder 918, as its name implies, may perform entropy encoding of the data output from the quantizer 916. For example, the entropy encoder 918 may perform run length encoding, Huffman encoding, Golomb encoding, context adaptive binary arithmetic coding, etc. The entropy encoder 918 may also entropy encode various encoding parameters such as a quantization parameter QP or an indication of a frame pattern mosaic for prediction.

[0094] The pixel block decoder 920 may reverse the encoding operation of the pixel block encoder 910 and may operate on blocks of the same shape and size as the pixel block encoder 910. For example, the pixel block decoder 920 may include a dequantizer 922, an inverse transform unit 924, and an adder 926. The pixel block decoder 920 may obtain its input data from the output of the quantizer 916. Although allowed, the pixel block decoder 920 does not have to perform entropy decoding of the entropy encoded data because entropy encoding is a lossless event. The dequantizer 922 may reverse the operation of the quantizer 916 of the pixel block encoder 910. The dequantizer 922 may perform uniform or non-uniform dequantization. Similarly, the inverse transform unit 924 may reverse the operation of the transform unit 914. The dequantizer 922 and the inverse transform unit 924 may use the same quantization parameter QP and transform mode M as their corresponding components in the pixel block encoder 910. The quantization operation will likely truncate the data at various points, and therefore the data recovered by the dequantizer 922 will likely have encoding errors when compared to the data presented to the quantizer 916 in the pixel block encoder 910 .

[0095] Adder 926 may invert the operation performed by subtractor 912. It may receive the same predicted pixel block from predictor 960 as subtractor 912 used in generating the residual signal. The adder 926 may add the predicted pixel block to the reconstructed residual value output by the inverse transform unit 924 and may output the reconstructed pixel block data.

[0096] The loop filter 930 may perform various filtering operations on the restored pixel block data. For example, the loop filter 940 may include a deblocking filter 932 and a sample adaptive offset ("SAO") filter 934 and / or other types of loop filters (not shown).

[0097] Reference frame repository 950 can store filtered frame data for use in subsequent predictions of other pixel blocks. Different types of prediction data are made available to predictor 960 for use in different prediction modes. For example, for an input pixel block, intra-frame predictor 963 obtains a prediction reference from decoded data of the same frame in which the input pixel block is positioned. Therefore, reference frame repository 950 can store decoded pixel block data of each frame during encoding. For the same input pixel block, inter-frame predictor 962 can use a prediction reference from a previously encoded and decoded frame, which is designated as a reference frame. Therefore, reference frame repository 950 can store these decoded reference frames.

[0098] In one aspect, the predictor 960 may operate on blocks of different types than the block encoder 910 and the block encoder 920. A prediction picture repository (not shown) may collect a plurality of prediction blocks output by the predictor 960 for use by the pixel block encoder 910 and the decoder 920. This may enable the encoder 910 and the decoder 920 to operate on blocks of different shapes, sizes, or offsets within a frame than those predicted by the predictor 960. In one aspect, the predictor 660 may predict blocks of pixel data, wherein the block shape is determined by a mosaic of a video frame. The predicted block shape may vary within a frame according to the mosaic, and the mosaic may vary over time between frames. Likewise, the prediction block shape need not be a rectangular block, and may include any block shape for mosaicking a two-dimensional image. For example, in addition to square and non-square rectangular blocks, the prediction block shape may also include hexagonal, octagonal, triangular, and L-shaped blocks.

[0099] On the other hand, the loop filter 930 can operate on multiple pixel blocks at a time. A decoded picture storage library (not shown) can collect multiple decoded pixel blocks output from the pixel block decoder 920 for use by the loop filter 930. This can enable, for example, the loop filter 930 to filter across pixel block boundaries.

[0100] The controller 970 can control the overall operation of the encoding system 900. The controller 970 can select operating parameters of the pixel block encoder 910 and the predictor 960 based on analysis of the input pixel block and also external constraints such as encoding bit rate targets and other operating parameters. Relevant to the present discussion, when selecting the quantization parameter QP, the use of uniform or non-uniform quantization processors, the transform mode M, and / or the frame pattern mosaic for prediction, those parameters can be provided to the entropy encoder 918 or the syntax unit 980, which can include data representing those parameters in the data stream of the encoded video data output by the system 900. The controller 970 can also select between different operating modes that the system can use to generate reference images, and can include metadata identifying the mode selected for each portion of the encoded data. In addition, the controller 970 can control the operation of the loop filter 930 and the prediction unit 960. For the prediction unit 960, such control may include frame pattern mosaicking, mode selection (λ, mode to be tested, search window, distortion strategy, etc.), and for the loop filter 940, selection of filter parameters, reordering parameters, weighted prediction, etc.

[0101] During operation, the controller 970 may modify operating parameters (such as QP, M, and frame pattern mosaics for prediction) at different granularities of the image data, either on a per-pixel block basis or at a larger granularity (e.g., per frame, per slice, per largest coding unit ("LCU") or coding tree unit (CTU), or another region). In one aspect, the quantization parameter may be modified on a per-pixel basis within an encoded frame. Similarly, the compressed bitstream output by the syntax unit 980 may include indications of operating parameters at different levels in a syntax hierarchy, such as a syntax level corresponding to a pixel block, frame, slice, LCU, CTU, CU, or other portion of the compressed bitstream.

[0102] Decoding using pattern mosaic

[0103] Fig.10 1 is a functional block diagram of a decoding system 1000 according to one aspect of the present disclosure. The decoding system 1000 may include a syntax unit 1010, a pixel block decoder 1020, a loop filter 1030, a reference frame repository 1050, a predictor 1060, and a controller 1070. Fig. 9 ), a frame repository (not shown) between the predictor 1060 and the pixel block decoder 1020 may allow for different block shapes, sizes, or offsets between prediction and encoding.

[0104] The syntax unit 1010 may receive a coded video data stream and may parse the coded data into its component parts. Data representing the coding parameters may be provided to the controller 1070, while data representing the coded residual (by Fig. 9 The data output by the pixel block encoder 910 may be provided to its corresponding pixel block decoder 1020. The predictor 1060 may generate a prediction block from the reference data available in the reference frame repository 1050 according to the encoding parameter data provided in the encoded video data. The pixel block decoder 1020 may invert the encoding operation applied by the pixel block encoder 910 ( Fig. 9 ). The loop filter 1040 may filter the reconstructed frame data. The filtered image data may be a reconstructed approximation s' of the source video data and may be output from the decoding system 1000 as output video. The filtered frames designated for use as reference frames may also be stored in the reference frame store 1050 for later use by the predictor 1060.

[0105] The pixel block decoder 1020 may include an entropy decoder 1022, a dequantizer 1024, an inverse transform unit 1026, and an adder 1028. The entropy decoder 1022 may perform entropy decoding to reverse the process performed by the entropy encoder 918 ( Fig. 9 ). The dequantizer 1024 may invert the operation of the quantizer 1016 of the pixel block encoder 910 ( Fig. 9 ). Similarly, the inverse transform unit 1026 may invert the operation of the transform unit 914 ( Fig. 9 ). They can use the quantization parameter QP and transform mode M provided in the encoded video data stream. Because quantization may truncate data, the pixel block s' restored by the dequantizer 1024 is likely to have coding errors ( Fig. 9 ).

[0106] Adder 1028 may invert the operation performed by subtractor 910 ( Fig. 9 ). It may receive a predicted pixel block from the predictor 1060, which is determined by a prediction reference in the coded video data stream. The adder 1028 may add the predicted pixel block to the reconstructed residual value output by the inverse transform unit 1026, and may output the reconstructed pixel block data.

[0107] The loop filter 1040 may perform various filtering operations on the restored pixel block data identified by the encoded video data. For example, the loop filter 1040 may include a deblocking filter 1032, a sample adaptive offset (SAO) filter 1034, and / or other types of loop filters. In this way, the operation of the loop filter 1030 simulates the operation of the corresponding loop filter 930 of the encoder 900 ( Fig. 9 ).

[0108] Reference frame store 1050 may store filtered frame data for use in subsequent prediction of other pixel blocks. Reference frame store 1050 may store entire decoded reference frames, and it may store partially decoded frames as it is decoded for intra prediction.

[0109] In one aspect, as with the encoder, the predictor 1060 may operate on blocks of a different type than the block type used by the block decoder 1020. A prediction picture repository (not shown) may collect a plurality of prediction blocks output by the predictor 1060 for use by the pixel block decoder 1020. This may enable the block decoder 1020 to operate on blocks of different shapes, sizes, or offsets within a frame than those predicted by the predictor 1060. In one aspect, the predictor 1060 may predict a block of pixel data, wherein the block shape of the prediction block is determined by a mosaic of the video frame. The predicted block shape may vary within a frame according to the mosaic, and the mosaic may vary over time between frames. Likewise, the prediction block shape need not be a rectangular block, and may include any block shape for mosaicking a two-dimensional image. For example, in addition to square and non-square rectangular blocks, the prediction block shape may also include hexagonal, octagonal, triangular, and L-shaped blocks.

[0110] On the other hand, the loop filter 1030 can operate on multiple pixel blocks at a time. A decoded picture storage library (not shown) can collect multiple decoded pixel blocks output from the pixel block decoder 1020 for use by the loop filter 1030. This can enable, for example, the loop filter 930 to filter across pixel block boundaries.

[0111] The controller 1070 may control the overall operation of the encoding system 1000. The controller 1070 may set operating parameters of the pixel block decoder 1020 and the predictor 1060 based on parameters received in the encoded video data stream. Relevant to the present discussion, these operating parameters may include pattern mosaic parameters for the prediction 1060, quantization parameter QP of the dequantizer 1024, and transform mode M of the inverse transform unit 1010. As discussed, the received parameters may vary at various granularities of the image data and may be transmitted at various levels of compressed bitstream syntax, for example, on a per-pixel block, per-frame, per-slice, per-LCU / CTU, or other types of based portions of the video.

[0112] The foregoing discussion has described the operation of various aspects of the present disclosure in the context of video encoders and decoders. These components are often provided as electronic devices. Video decoders and / or controllers can be embedded in integrated circuits, such as application specific integrated circuits, field programmable gate arrays, and / or digital signal processors. Alternatively, they can be embedded in computer programs executed on camera devices, personal computers, laptops, tablet computers, smart phones, or computer servers. Such computer programs typically include instructions stored in physical storage media such as electronic, magnetic, and / or optical-based storage devices, where they are read and executed by processors. Decoders are often packaged in consumer electronic devices, such as smart phones, tablet computers, game systems, DVD players, portable media players, etc.; and they can also be packaged in consumer software applications, such as video games, media players, media editors, etc. And, of course, these components can be provided as a hybrid system that distributes functions on dedicated hardware components and programmed general-purpose processors as needed.

[0113] Video encoders and decoders can exchange video over a channel in a variety of ways. They can communicate with each other via communications and / or computer networks, such as Figure 1 In other applications, the video encoder may output the video data to a storage device, such as an electrical, magnetic, and / or optical storage medium, which may be provided to a decoder at some later time. In such applications, the decoder may retrieve the encoded video data from the storage device and decode it.

[0114] Several embodiments of the present invention are specifically shown and / or described herein. However, it should be understood that modifications and variations of the present invention are covered by the above teachings and are within the scope of the appended claims without departing from the spirit and intended scope of the invention.

Claims

1. A method for encoding a video, comprising: determining a predetermined plurality of spatial allocation schemes based on spatial offsets derived from temporal positions of corresponding input frames; determining, for each of a plurality of input frames of a video to be encoded, a spatial allocation scheme from the predetermined plurality of spatial allocation schemes, wherein partitions of the spatial allocation scheme are spatially offset relative to partitions of other spatial allocation schemes, and determining the spatial allocation scheme is based on respective temporal positions of the input frames within the video; For the first input frame: partitioning the first input frame into allocation units according to a spatial allocation scheme determined for the first input frame; encoding allocation units of a first input frame according to a prediction algorithm; For a second input frame that is offset in time relative to the first input frame: partitioning the second input frame into allocation units according to the spatial allocation scheme determined for the second input frame; and encoding allocation units of a second input frame according to a prediction algorithm; and An encoded video sequence is sent that includes a predictively encoded first input frame, a predictively encoded second input frame, and an indication of a spatial allocation scheme determined for the predictively encoded first input frame and the predictively encoded second input frame, the indication including an indication of corresponding spatial offsets.

2. The method according to claim 1, wherein The spatial allocation scheme determined for the first input frame is spatially offset relative to the initial spatial allocation scheme based on the temporal position of the first input frame, and The spatial allocation scheme determined for the second input frame is spatially offset relative to the initial spatial allocation scheme based on the temporal position of the second input frame.

3. The method of claim 1, wherein the spatial offset is predetermined based on a selected offset function with respect to time.

4. The method of claim 1, wherein the indication of the spatial offset comprises an indication of a selected offset function with respect to time.

5. The method according to claim 1, wherein the method further comprises: Prior to encoding the first input frame or the second input frame, the predetermined plurality of spatial allocation schemes are determined by deriving the spatial offset from a function 2*modulus(t,N), where t is a corresponding temporal position of the input frame and N is a predetermined constant.

6. The method of claim 1, wherein the spatial offset comprises a horizontal offset specified by a first selected function based on a first constant and a vertical offset specified by a second selected function based on a second constant.

7. The method of claim 1, wherein the plurality of spatial allocation schemes comprises irregular allocations of partitions having varying shapes within an allocation scheme of an input frame.

8. A non-transitory computer-readable medium comprising instructions for encoding a video, the instructions, when executed by a processor, causing the processor to perform the method according to any one of claims 1-7.

9. A video encoder comprising: Frame buffer, a predictor for predicting from a reference picture in the picture buffer, and A controller configured to execute instructions, wherein the instructions, when executed by the controller, cause the video encoder to perform the method according to any one of claims 1-7.

10. A video decoding method, comprising: extracting, from a coded video sequence, an indication of a spatial allocation scheme for a corresponding plurality of coded frames, wherein partitions of the spatial allocation scheme are spatially offset relative to partitions of other spatial allocation schemes, and the indication of the spatial allocation scheme comprises an indication of the spatial offsets derived from temporal positions of the corresponding coded frames; For the first coded frame: partitioning the first coded frame into allocation units according to a spatial allocation scheme indicated for the first coded frame; decoding allocation units of a first coded frame according to a prediction algorithm; as well as For a second coded frame that is offset in time relative to the first coded frame: partitioning the second coded frame into allocation units according to the spatial allocation scheme indicated for the second coded frame; and The allocation units of the second coded frame are decoded according to the prediction algorithm.

11. The method according to claim 10, wherein The spatial allocation scheme indicated for the first coded frame is spatially offset relative to the initial spatial allocation scheme based on a temporal position of the first coded frame, and The spatial allocation scheme indicated for the second encoded frame is spatially offset relative to the initial spatial allocation scheme based on the temporal position of the second encoded frame.

12. The method of claim 10, wherein the spatial offset is predetermined based on a selected offset function with respect to time.

13. The method of claim 10, wherein the indication of the spatial offset comprises an indication of a selected offset function with respect to time.

14. The method according to claim 10, wherein the method further comprises: Prior to decoding the first coded frame or the second coded frame, the spatial allocation scheme is determined by deriving the spatial offset from the function 2*modulus(t,N), where t is the corresponding temporal position of the coded frame and N is a predetermined constant.

15. The method of claim 10, wherein the spatial offset comprises a horizontal offset specified by a first selected function based on a first constant and a vertical offset specified by a second selected function based on a second constant.

16. The method of claim 10, wherein the plurality of spatial allocation schemes comprises irregular allocations of partitions having varying shapes within an allocation scheme for a coding frame.

17. A non-transitory computer readable medium comprising instructions for decoding a video, the instructions when executed by a processor causing the processor to perform the method according to any one of claims 10-16.

18. A video decoder comprising: Decoded Picture Buffer (DPB), a predictor for predicting from reference pictures in the DPB, and A controller configured to execute instructions, wherein the instructions, when executed by the controller, cause the video decoder to perform the method according to any one of claims 10-16.

Images