Video block encoding or decoding method and apparatus using current picture reference encoding mode

Abstract

A video processing method includes receiving input data for a current block in a current picture, constructing a candidate list for the current block by including one or more history-based motion vector prediction (HMVP) candidates, selecting a candidate from the candidate list, locating a reference block based on motion information of the selected candidate, and predicting the current block by using the reference block for encoding or decoding the current block. If the current block is to be processed by a general inter prediction mode, the one or more HMVP candidates are obtained from a general HMVP table, and if the current block is to be processed by a current picture reference (CPR) mode, the one or more HMVP candidates are obtained from a CPR HMVP table. The two HMVP tables are maintained and updated separately.

Description

[0001] cross-application

[0002] This application is a divisional application of patent application number 201980072894.6, entitled "A Method and Apparatus for Encoding or Decoding Video Blocks Using Current Screen Reference Encoding Method". Technical Field

[0003] This invention relates to video encoding and decoding using current-frame reference encoding / decoding. In particular, this invention relates to using one or more encoding / decoding tools to perform current-frame reference encoding / decoding. Background Technology

[0004] High Efficiency Video Coding (HEVC) is a new video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) of the ITU-T Study Group. HEVC relies on a block-based coding structure, dividing each slice into multiple Coding Tree Units (CTUs). The encoded frame is represented by a slice, or a whole slice, each slice containing an integer number of CTUs. Individual CTUs within a slice are processed according to a raster scanning sequence. In the HEVC main profile, the minimum and maximum sizes of CTUs are specified by syntax elements in the Sequence Parameter Set (SPS), in sizes of 8x8, 16x16, 32x32, or 64x64. In a bi-predictive (B) slice, the sample values ​​of each block are decoded using intra-predictive or inter-predictive methods with at most two motion vectors and a reference index. In a predictive (P) slice, the sample values ​​of each block are decoded using intra-predictive or inter-predictive methods with only one motion vector and one reference index. Intra (I) slices are decoded using only intra-predictive methods. Following a quadtree (QT) partitioning process, each CTU is further recursively partitioned into one or more coding units (CUs) to accommodate various local motion and texture characteristics. The CTU size is M x M, where M is one of the values ​​64, 32, and 16. Each CTU can be a single CU or partitioned into four smaller units of the same size M / 2 x M / 2, which are called nodes in the code-decode tree. If a node is not further partitioned, it is a leaf node in the code-decode tree, and each leaf node is a CU. The quadtree partitioning process can be iterated until the node size reaches the minimum allowed CU size specified in the Sequence Parameter Set (SPS). An example of a quadtree block partitioning structure used to partition a CTU into several CUs is illustrated in [illustration missing]. Figure 1 In the diagram, the solid line represents the boundary of the CU in CTU100.

[0005] Prediction decisions are made at the CU layer, where each CU is encoded or decoded using either inter-frame (temporal) prediction or intra-frame (spatial) prediction. Since the minimum CU size can be 8x8, the minimum interval size for switching between different basic prediction types is also 8x8. Once the CU hierarchical tree is segmented, each CU can be further segmented into one or more Prediction Units (PUs) based on the PU segmentation type used for prediction. Figure 2 This displays the eight PU segmentation types defined in the HEVC standard. (Based on the information displayed...) Figure 2 One of the eight PU segmentation types is used, and each CU can be further segmented into one, two, or four PUs. The PU, together with the associated CU syntax, serves as a basic representative block sharing prediction information. The same prediction process is applied to all pixels in the PU, and prediction-related information is conveyed to the decoder based on the PU. The specified prediction process is used to predict the values ​​of relevant pixel samples within the PU. After obtaining the residual signal generated by the prediction process, the residual data belonging to the CU is segmented into one or more Transform Units (TUs) according to the Residual QuadTree (RQT) block segmentation structure to convert the residual data into transformation coefficients as a concise data representation. Figure 1 The dashed lines in the diagram represent TU boundaries. A TU is a basic representative block used to apply transformations and quantizations to the residual signal and transformation coefficients, respectively. For each TU, a transformation matrix of the same size as the TU is applied to the residual signal to generate transformation coefficients, which are then quantized based on the TU and transmitted to the decoder. A transformation unit consists of a transformation block (TB) of 8x8, 16x16, or 32x32 luminance samples and two corresponding chrominance samples encoded and decoded according to the 4:2:0 color sample format. Integer transformations are applied to the transformation blocks, and the level values ​​of the quantized coefficients, along with other side information, are entropy-encoded in the video bitstream.

[0006] These terms—Coding Tree Block (CTB), Coding Block (CB), Prediction Block (PB), and Transform Block (TB)—are defined to specify a two-dimensional array of samples associated with a color component, CTU, CU, PU, ​​and TU, respectively. For example, a CTU consists of a luma CTB, two chroma CTBs, and their associated syntax elements. In HEVC systems, the same quadtree block partitioning structure is typically applied to both the luma and chroma components, unless the minimum size of the chroma block has been reached.

[0007] Binary tree partitioning is an alternative to quadtree partitioning, which recursively divides a block into two smaller blocks. The most efficient and simplest binary tree partitioning types are symmetric horizontal and symmetric vertical partitioning. For a given block of size M x N, a flag is sent to indicate whether the block is to be partitioned into two smaller blocks; if so, another syntax element is sent to indicate which partitioning type is used. When the given block is partitioned using the symmetric horizontal partitioning type, it is divided into two blocks of size M x N / 2; otherwise, it is divided into two blocks of size M / 2 x N. The binary tree partitioning process can be repeated until the block size, width, or height reaches the minimum allowed block size, width, or height defined by the higher-order syntax element. Both the minimum allowed block width and height are specified because there are horizontal and vertical partitioning types in the binary tree partitioning process. Horizontal partitioning is implicitly prohibited when the partitioning would result in a block height less than the specified minimum height. Similarly, vertical partitioning is implicitly disabled when the result of partitioning would cause the block width to be less than the specified minimum width. Binary tree partitioning structures can be used to divide a block into several smaller blocks; for example, it can be used to partition a slice into several CTUs, a CTU into several CUs, a CU into several PUs, or a CU into several TUs.

[0008] Binary tree structures are more flexible than quadtree structures because they can support more segmentation shapes, which is the source of improved encoding and decoding efficiency. However, the encoding and decoding complexity also increases because the encoder needs to decide on the best segmentation shape from among many. A new segmentation structure called Quadtree plus Binary Tree (QTBT) can balance the encoding and decoding efficiency and complexity of quadtree and binary tree segmentation structures. Figure 3AThis demonstrates an exemplary QTBT structure, where the CTU is first segmented using a quadtree structure followed by a binary tree partitioning structure. CUs can be recursively partitioned using quadtree partitioning until the current CU size reaches the minimum allowed quadtree leaf node size. If the size of a leaf quadtree block is not greater than the size of the maximum allowed binary tree root node, then each leaf quadtree block can be further partitioned using binary tree partitioning. Binary tree partitioning can be applied recursively until the current CU size, width, or height reaches the minimum allowed binary tree leaf node size, width, or height, or the binary tree depth reaches the maximum allowed binary tree depth. In QTBT partitioning, only horizontally symmetric partitioning and vertically symmetric partitioning are the two allowed binary tree partitioning types. Leaf CUs generated from QTBT partitioning can be used in prediction and transformation procedures without any further partitioning. Figure 3A Based on the QTBT segmentation structure, an example of a block segmentation structure is drawn, and Figure 3B Draw the corresponding codec tree diagram for use Figure 3A The QTBT segmentation structure is shown. Figure 3A and 3B Solid lines represent quadtree partitions, while dashed lines represent binary tree partitions. In each partition node (i.e., non-leaf) of the binary tree structure, a flag indicates the partition type (horizontal or vertical), with 0 indicating a horizontal symmetrical partition and 1 indicating a vertical symmetrical partition. For a CTU encoded in a type I slice, the QTBT codec tree representation used for the chroma CTB may differ from the QTBT codec tree representation used for the corresponding luma CTB. For CTUs encoded in type P and type B slices, the same QTBT codec tree representation is applied to both the chroma CTB and luma CTB, unless a minimum size has been reached for the chroma block. In the QTBT partition structure, the minimum allowable size of a quadtree leaf node, the maximum allowable size of a binary tree root node, the minimum allowable width and height of a binary tree leaf node, and the maximum allowable depth and height of a binary tree are indicated in higher-order syntax, such as SPS.

[0009] QTBT partitioning structures can be used to divide a block into multiple smaller blocks. For example, a QTBT partitioning structure can be used to divide a CTU into multiple CUs, which are then further processed by prediction and transformation codecs. In other words, the leaf nodes of a QTBT partitioning structure are the basic units used for both prediction and transformation codecs. In one example, the CTU size is 128x128, the minimum allowed quadtree leaf node size is 16x16, the maximum allowed binary tree root node size is 64x64, the minimum allowed binary tree leaf node width and height are both 4, and the maximum allowed binary tree depth is 4. In this example, each CTU is partitioned using a quadtree partitioning structure, so the size of the leaf quadtree unit can range from 16x16 (the minimum allowed quadtree leaf node size) to 128x128 (the size of the CTU). If a leaf quadtree unit is 128×128, it cannot be further segmented by binary tree partitioning because its size exceeds the maximum allowed binary root node size of 64x64. Otherwise, leaf quadtree units with a size less than or equal to 64x64 can be further segmented by binary tree partitioning. When a leaf quadtree unit is partitioned by binary tree partitioning, this leaf quadtree unit also serves as the root binary tree unit, with a binary tree depth of 0. When the binary tree depth reaches 4 (i.e., the specified maximum depth), partitioning is implicitly prohibited. Similarly, when the width or height of a binary tree node is equal to 4, horizontal or vertical partitioning is implicitly prohibited, respectively. Leaf nodes of the QTBT partitioning structure are further processed through prediction and transformation encoding / decoding.

[0010] The QTBT segmentation structure applies separately to the luma and chroma components of intra-frame (I) slices, and simultaneously to both luma and chroma components of prediction (P) and bidirectional prediction (B) slices, except when certain minimum sizes of the chroma components are reached. Within an intra-frame slice, the luma CTB has its own QTBT segmentation block, while the two chroma CTBs have separate QTBT segmentations, and two corresponding chroma CTBs share another QTBT segmentation block. Alternatively, the two chroma CTBs have their own QTBT segmentation blocks.

[0011] Inter-frame prediction or motion compensation (including inter-frame mode, merge mode, and skip mode) is designed to utilize the temporal correlation between the current frame and one or more reference frames. Motion vectors (MVs) represent the displacement between the current block in the current frame and its corresponding reference block in the reference frame. Typically, it is assumed that a pattern corresponding to an object or background in the current frame will be displaced to form a corresponding object in the next frame, or will be associated with other patterns in the current frame. With an estimate of this displacement from other frames (e.g., using block matching techniques), these patterns can be almost reproduced without re-encoding and decoding. Similarly, block matching can be performed within the same frame to select reference blocks. Similar to the block matching procedure in inter-frame prediction, motion vectors represent the displacement between the current block and its corresponding reference block in the reference frame. The corresponding reference block is a previously reconstructed block within the same frame as the current block. This technique, known as Intra Block Copy (IBC) or Current Picture Referencing (CPR), has been observed to be inefficient when applied to video content captured by a camera. One reason is that text patterns in spatially adjacent regions may resemble the current codec block, but there are usually some gradual spatial variations. Finding a perfect match for a block within the same frame of video captured by the camera is difficult, thus limiting the improvement in codec efficiency. However, by searching for similar patterns within the current frame, CPR is a very effective technique for encoding screen content. For videos typically containing text and graphics, there are often repeated patterns within the same frame. To process the current block in the current frame using CPR, the current block is predicted from previously reconstructed blocks in the current frame. Motion vectors (also known as block vectors, or BV) are used to transmit the relative displacement from the current block position to the reference block position. Transformation, quantization, and entropy encoding are then used to encode prediction errors. Some examples of using CPR compensation in screen content encoding are illustrated in... Figure 4 The reference sample of the reference block derived from MV is the reconstructed sample of the current frame before the in-loop filter operation. The in-loop filter operation consists of both deblocking and Sample Adaptive Offset (SAO) filtering in the HEVC standard.

[0012] The first version of CPR compensation was limited to small local regions, with only one-dimensional MV, and was only used for blocks with a size of 2Nx2N. Later, more advanced CPR designs were developed, incorporating bitstream conformance constraints to regulate the effective CPR MV values ​​of the current PU in the more advanced CPR designs. First, one of equations (1) and (2) must be true.

[0013] BV_x + offsetX + nPbSw + xPbs – xCbs <= 0 (1)

[0014] BV_y + offsetY + nPbSh + yPbs – yCbs <= 0 (2)

[0015] Second, the Wavefront Parallel Processing (WPP) condition in Equation (3) must be true.

[0016] (xPbs+BV_x+offsetX+nPbSw-1) / CtbSizeY–xCbs / CtbSizeY<=yCbs / CtbSizeY-(yPbs+BV_y+offsetY+nPbSh-1) / CtbSizeY(3)

[0017] In equations (1) to (3), (BV_x, BV_y) represents the luma block vector used for the current PU, nPbSw and nPbSh are the width and height of the current PU, (xPbS, yPbs) represents the position of the top left pixel of the current PU relative to the current screen, (xCbs, yCbs) represents the position of the top left pixel of the current CU relative to the current screen, and CtbSizeY is the size of the CTU. The two variables offsetX and offsetY are two adjusted offsets in two dimensions after considering the interpolation of chroma samples used for CPR mode, and are defined in equations (4) and (5).

[0018] offsetX = BVC_x & 0x7? 2 : 0 (4)

[0019] offsetY = BVC_y & 0x7? 2 : 0 (5)

[0020] Where (BVC_x, BVC_y) represents the chroma block vector, which is 1 / 8 pixel resolution in HEVC.

[0021] Third, the reference block used to predict the current PU encoded and decoded in CPR mode must be on the same tile or slice boundary.

[0022] Affine motion compensation uses an affine model to describe the rotation of two-dimensional blocks, as well as the two-dimensional deformation from a cube or cuboid to a parallelogram. This affine model is described in equation (6).

[0023] x'=a0+a1*x+a2*y

[0024] y' = b0 + b1*x + b2*y (6)

[0025] Six parameters a0, a1, a2, b0, b1, and b2 are involved in this affine model. For each pixel (x, y) in the region of interest, the motion vector of this pixel is A'–A=(a0+(a1-1)*x+a2*y, b0+b1*x+(b2-1)*y). The motion vector of each pixel depends on its position. In this affine model, if the motion vectors of three different positions are known, the above six parameters in equation (6) can be solved, so it is equivalent to having six known parameters. Each position with a known motion vector is called a control point. The affine model with six parameters corresponds to the model with three control points.

[0026] In the implementation of affine motion compensation, an affine flag is sent for each 2Nx2N block segment encoded in either merge mode or Advanced Motion Vector Prediction (AMVP) mode. If the flag is true for the current block, the motion vector derivation for that block follows the affine model; if the flag is false, the motion vector derivation follows the traditional translation model. When the affine AMVP mode is used, the motion vectors (MVs) of three control points are sent. At each control point location, the MV is predictively encoded and decoded. Then, the motion vector difference (MVD) of these control points is encoded, decoded, and transmitted.

[0027] The History-based Motion Vector Prediction (HMVP) method stores motion information from several previously encoded / decoded blocks in a table for reference by subsequent blocks. Each piece of motion information in the table is called an HMVP candidate. The table with multiple HMVP candidates is maintained during the encoding or decoding process and is emptied when a new slice is encountered. After encoding or decoding an inter-coded non-affine block, the associated motion information is added to the last field of the table as a new HMVP candidate. The decoding process of the HMVP method is illustrated in [the diagram]. Figure 5 For each block encoded and decoded using inter-frame prediction, an HMVP table with HMVP candidates is loaded by the decoder, and the decoder decodes the block with a candidate list containing one or more HMVP candidates obtained from that HMVP table. The HMVP table is then updated with the decoded motion information of the decoded block. Figure 6 This diagram illustrates an example of updating an HMVP table used to store multiple HMVP candidates. In one example, the table size is set to 6 (L = 6), so a maximum of 6 HMVP candidates can be added to the HMVP table. When a new HMVP candidate is inserted into the HMVP table, a constrained first-in-first-out (FIFO) rule is applied. A redundancy check is first performed to search the HMVP table for any redundant HMVP candidates that are the same as the most recently added HMVP candidate. Figure 6 As shown, HMVP2 is determined to be a redundant HMVP candidate, and it is removed from the HMVP table. After removing the redundant HMVP candidate, all HMVP candidates in the HMVP table that are after the removed candidate are shifted forward by decrementing each index by 1. The HMVP table is then filled with the new HMVP candidate CL-1.

[0028] HMVP candidates stored in the HMVP table can be used in the merge candidate list construction process. The latest few HMVP candidates in the HMVP table are checked sequentially and inserted into the merge candidate list after the Temporal Motion Vector Prediction (TMVP) candidates are inserted. Pruning is applied to the HMVP candidates by checking them against the spatial and temporal candidates in the merge candidate list. For example, sub-block motion candidates like Advanced Temporal Motion Vector Prediction (ATMVP) are not checked with the HMVP candidates. To reduce the number of pruning operations, three simplifications are introduced. In the first simplification, the number of HMVP candidates to be checked (denoted by L) is set according to equation (7).

[0029] L = (N <=4)? M: (8 - N) (7)

[0030] Where N indicates the number of available non-subblock merge candidates, and M indicates the number of available HMVP candidates in the HMVP table.

[0031] The second simplification checks whether the total number of available merge candidates reaches the maximum allowed merge candidates minus 1. If the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list construction process from the HMVP table will be terminated. The third simplification reduces the number of pairs in the merge-type bidirectional predictive merge candidate derivation from 12 to 6.

[0032] Similar to the candidate merging process, HMVP candidates can be used in the AMVP candidate list construction process. The motion vectors of the last K HMVP candidates in the HMVP table are inserted into the AMVP candidate list after the TMVP candidates. Only HMVP candidates with the same reference frame as the AMVP target reference frame are used to construct the AMVP candidate list. Pruning is applied to the HMVP candidates. For example, K is set to 4, and the AMVP list size is set to 2, which is the same as the AMVP list size in the HEVC standard. Summary of the Invention

[0033] A method for processing video data in a video decoding system includes: receiving input data associated with a current frame comprising a plurality of virtual pipeline data units, wherein each virtual pipeline data unit is of a fixed size; performing a decoding procedure to decode codec units in the current virtual pipeline data unit and generate a decoded sample of the current virtual pipeline data unit, wherein each codec unit encoded in a current frame reference mode is decoded by obtaining a decoded sample of a reference block stored in a current frame reference buffer; and updating the current frame reference buffer by storing the decoded sample of the current virtual pipeline data unit when the decoding procedure of the current virtual pipeline data unit is completed, wherein the current frame reference buffer is updated one virtual pipeline data unit after another.

[0034] Other aspects and features of the invention will become apparent to those skilled in the art upon review of the following detailed description of specific embodiments. Attached Figure Description

[0035] Various embodiments of the examples presented in this disclosure will be described in more detail with reference to the following figures, wherein similar numbers are used to refer to similar components, and wherein:

[0036] Figure 1 An exemplary codec tree is drawn according to the quadtree partitioning defined in the HEVC standard, which is used to partition a codec tree unit (CTU) into several codec units (CUs) and each CU into one or more conversion units (TUs).

[0037] Figure 2 Eight different prediction unit (PU) partitioning types are plotted according to the HEVC standard to partition a CU into one or more PUs.

[0038] Figure 3A An exemplary block partitioning structure is illustrated based on the quadtree plus binary tree (QTBT) partitioning structure.

[0039] Figure 3B The drawing corresponds to Figure 3A QTBT segmentation structure of the encoder-decoder tree structure.

[0040] Figure 4 This section showcases some examples of using Current Screen Reference (CPR) technology to encode screen content.

[0041] Figure 5 The decoding process based on the Historical Motion Vector Prediction (HMVP) method is illustrated.

[0042] Figure 6This is an example of a table that is updated for HMVP methods.

[0043] Figure 7 The flowchart shown is an exemplary embodiment of the present invention, illustrating a video processing method that uses two HMVP tables to encode or decode the current block.

[0044] Figure 8 The flowchart illustrates a video processing method for decoding the current VPDU and updating the CPR reference buffer, according to another exemplary embodiment of the present invention.

[0045] Figure 9 An exemplary system block diagram is drawn to illustrate a video coding system for representing a video processing method, according to an embodiment of the present invention.

[0046] Figure 10 An exemplary system block diagram is drawn to illustrate a video decoding system diagram for use in representing a video processing method, according to an embodiment of the present invention. Detailed Implementation

[0047] It is readily understood that the various modules of the invention, as generally described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the invention, as shown in the drawings, is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention.

[0048] References to "embodiments," "some embodiments," or similar language in this specification mean that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. Therefore, the phrases "in embodiments" or "in some embodiments" appearing throughout this specification do not necessarily refer to the same embodiments, which may be implemented alone or in combination with one or more other embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention may be practiced without one or more specific details or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0049] Various embodiments of the present invention improve the integration of Current Picture Referencing (CPR) with other encoding / decoding tools. Some embodiments consider the application of dual-tree block segmentation to separate luma and chroma components under CPR and general inter-frame prediction integration. A flag `CPR_enable_flag` is signaled to indicate whether the current CU or current PU is encoded / decoded in CPR mode. The current CU or PU is presumed to be encoded / decoded in CPR mode when the current picture is included in the reference picture list and there are no inter-frame reference pictures in the reference picture list. For example, the current CU or PU is presumed to be encoded / decoded in CPR mode if all reference pictures in the reference picture list are the current picture, or if there is only one reference picture in the reference picture list and that single reference picture is the current picture. In one example, the current picture is included in the reference picture list and there are no other general inter-frame reference pictures in the reference picture list; when one or more parallel luma blocks are encoded / decoded in CPR mode, the current chroma CU or PU is presumed to be encoded / decoded in CPR mode. In the following description, if the current block is encoded or decoded in CPR mode, or if CPR mode is enabled for the current block, a reference frame pointing to the current frame is used as block prediction.

[0050] CPR based on historical motion vector prediction improves video encoding / decoding efficiency by incorporating one or more HMVP candidates into a merge candidate list or an AMVP candidate list. Blocks encoded or to be encoded using a general inter-frame prediction mode (e.g., skip mode, merge mode, or AMVP mode) can be predicted using reference blocks derived from HMVP candidates. In some examples, a general inter-frame prediction mode is a prediction mode that uses motion vector predictors in a general reference frame. These motion vector predictors are selected from a candidate list and used to generate the predicted pixel values ​​for the current block. The index of the selected motion vector predictor is passed from the encoder to the decoder so that the decoder can derive the selected motion vector predictor after constructing the candidate list using the same method as the encoder. The general reference frame is a frame encoded before the current frame and can be referenced by the current frame. In some embodiments, the HMVP table stores only motion information of CUs encoded using a general inter-frame prediction mode, which includes one or a combination of merge, skip, and AMVP modes. In one embodiment, the HMVP method is disabled if CPR mode is enabled for the current block in the current frame, and this current frame is the only reference frame in the reference frame list. In another embodiment, only MVs pointing to general reference frames are used to update the HMVP table; that is, CPR MVs cannot be inserted into the HMVP table.

[0051] In some embodiments of the invention, the HMVP method is also used to process blocks encoded or to be encoded using CPR mode. Therefore, two separate HMVP tables (or two separate HMVP lists) are maintained: one HMVP table stores HMVP candidates corresponding to the motion information of the encoded blocks previously processed using general inter-frame prediction mode, and is called the general HMVP table; the other HMVP table stores HMVP candidates corresponding to the motion information of the encoded blocks previously processed using CPR mode, and is called the CPR HMVP table. Each HMVP candidate stored in the general HMVP table includes one or two MVs pointing to one or two reference frames in general inter-frame prediction mode from list 0, list 1, or both. The general HMVP table stores the MVs and reference frame indices for the HMVP candidates. Each HMVP candidate stored in the CPR HMVP table includes an MV pointing to the current frame. Only MVs need to be stored in the CPR HMVP table, and the reference frame index is ignored because the reference frame for each HMVP candidate in the CPR HMVP table is always the current frame. In this embodiment, to encode or decode the current block using a general inter-frame prediction mode (e.g., AMVP, merge, or skip mode), the candidate list is constructed by incorporating one or more HMVP candidates obtained from a general HMVP table. If the current block is encoded or decoded using CPR mode, the candidate list is constructed by incorporating one or more HMVP candidates obtained from a CPR HMVP table. For example, the one or more HMVP candidates are inserted into the candidate list after the Temporal Motion Vector Prediction (TMVP) candidates.

[0052] If the current block is encoded / decoded in AMVP mode, only HMVP candidates with the same reference frame as the AMVP target reference frame are used to construct the candidate list. A pruning process is applied to check one or more HMVP candidates against one or more existing candidates in the candidate list to remove any redundant HMVP candidates. Candidates are selected from the candidate list for the current block; for example, the encoder selects candidates by performing rate distortion optimization, and the decoder selects candidates based on indices transmitted in the video bitstream. The current block is then encoded or decoded by locating the reference block in the reference frame based on the motion information of the selected candidates and predicting the current block from the reference block. If the current block is encoded / decoded in CPR mode, the reference frame is the current frame.

[0053] In some embodiments, the validity of a CPR MV is checked before it is added to the merge candidate list or AMVP candidate list from the CPR HMVP table. A range constraint is set to determine the validity of each HMVP candidate obtained from the CPR HMVP table, and the HMVP candidate obtained from the CPR HMVP table is valid and can be added to the candidate list if the reference block pointed to by the HMVP candidate is within the range constraint. For example, if the reference sample in the reference block pointed to by the CPR MV is decoded and stored in the CPR reference buffer, the CPR MV will be marked as valid after the validity check. For example, if the CPR reference buffer stores decoded samples of the current CTU and the left CTU region, a valid CPR MV will point to a reference block in the current CTU or the left CTU region. Invalid CPR MVs will not be added to the merge candidate list or AMVP candidate list.

[0054] In one embodiment employing two HMVP tables in general inter-frame prediction mode and CPR mode, when dual-tree block segmentation is used to segment the luma and chroma components, these two HMVP tables are shared by both the luma and chroma components of the video data. When dual-tree block segmentation is used, the luma and chroma components are segmented using different block segmentation structures; while when shared-tree block segmentation is used, the luma and chroma components are segmented using the same block segmentation structure. In another embodiment, when dual-tree block segmentation is used to segment the luma and chroma components, the candidate list for the current luma block is reused by one or more collocated chroma blocks. In another embodiment, when the current chroma block is processed using CPR mode, the candidate list for the current chroma block includes one or more CPR MVs derived from one or more collocated luma blocks.

[0055] In another embodiment, an HMVP table is used to store previously decoded motion information. Two motion videos (MVs) from the CPR mode and the general inter-frame prediction mode can be used to update the HMVP table, and one or more HMVP candidates are retrieved from the HMVP table and inserted into the merged candidate list or the AMVP candidate list. In one embodiment, when the current frame is selected as the reference frame for predicting the current block, only the CPR MV from the AMVP candidate list can be used.

[0056] In one embodiment, the encoder or decoder inserts only one or more CPR HMVPs from the HMVP table into the merge candidate list or AMVP candidate list. In another embodiment, any HMVP candidate obtained from the HMVP table is rounded up to an integer precision before being added to the merge candidate list or AMVP candidate list. In an embodiment employing dual-tree block segmentation for the luma and chroma components, when the HMVP method is used in conjunction with the CPR mode, two HMVP tables are maintained and used during the encoding or decoding process, one for the luma component and the other for the chroma component. In another embodiment, a single HMVP table is maintained and used for both the luma and chroma components, and motion information for both the luma and chroma components can be used to update the HMVP table. In examples where a single HMVP table is shared for both the luma and chroma components, the chroma component can only use the luma CPR MV and not any adjacent chroma CPR MV.

[0057] The following embodiments of CPR implementation using inter-frame prediction describe various constraints applied when integrating CPR mode with skip mode encoding / decoding, merge mode encoding / decoding, or AMVP mode encoding / decoding. In one embodiment, the encoder or decoder adds CPR MVs to the merge candidate list or AMVP candidate list only from spatial or temporal candidates. In one embodiment, when the current frame is selected as the reference frame for encoding or decoding the current block encoded in AMVP mode, only CPR MVs from the AMVP candidate list can be used. In another embodiment, all MVs from CPR or general inter-frame prediction modes in the merge candidate list can be used. In another embodiment, only MVs from general inter-frame prediction modes in the merge candidate list can be used. In yet another embodiment, an MV from a spatial candidate, temporal candidate, or generated candidate is rounded up to an integer precision before being added to the merge candidate list or AMVP candidate list.

[0058] In one embodiment that uses dual-tree block segmentation to segment the luma and chroma components, the luma and chroma components share the same merge candidate list or AMVP candidate list. For example, a luma block generates a merge candidate list or AMVP candidate list, and a juxtaposed chroma block reuses this merge candidate list or AMVP candidate list. In another embodiment, one or more CPR MVs from one or more juxtaposed luma blocks can be used to construct a merge candidate list or AMVP candidate list for the current chroma block. In some embodiments considering inserting one or more CPR MVs pointing to the current frame in the merge candidate list or AMVP candidate list, the validity of each CPR MV is checked before adding that CPR MV to the merge candidate list or AMVP candidate list.

[0059] In the HEVC standard, CPR is implemented using pairwise averaging of candidates or non-sub-block STMVP candidates. Pairwise averaging of candidates produces pairs of merge candidates from a predefined merge candidate list as replacements of combined candidates. If both lists 0MV are available, the lists 0MV of merge candidates in each predefined pair are averaged to produce a list 0MV for the pairwise averaging candidates, and the list 0MV of the merge candidate with the larger merge index is scaled down to the reference frame of the merge candidate with the smaller merge index. If only list 0MV is available, the list 0MV of the merge candidate is used directly as the list 0MV of the pairwise averaging candidates. If no MV is available in list 0, list 0 of the pairwise averaging candidates is invalid. A similar MV averaging rule is applied to derive list 1MV of the pairwise averaging candidates. The non-subblock Spatial-Temporal Motion Vector Prediction (STMVP) mode is a simplified version of the STMVP mode in the Joint Exploration Test Model (JEM). Non-subblock STMVP offers more encoding / decoding benefits due to its optimized reference location. A non-subblock STMVP candidate is generated by averaging a combination of spatial and temporal candidates. The non-subblock STMVP mode is also beneficial for hardware implementation. Inserting paired averaged candidates or non-subblock STMVP candidates into the merged candidate list or AMVP candidate list improves the encoding / decoding efficiency of skip mode encoding / decoding, merged mode encoding / decoding, or AMVP mode encoding / decoding.

[0060] In one embodiment, if all source candidates used to generate paired average candidates or non-subblock STMVP candidates are CPR MVs, the resulting paired average candidate or non-subblock STMVP candidate is a valid CPR candidate. For example, if two motion candidates from the merged candidate list are both CPR candidates, the paired average candidate generated from these two CPR candidates will also be a valid CPR candidate. In one example, the MV of the paired average candidate is rounded up to an integer MV. In another example, if the MV of the paired average candidate is not an integer MV, this paired average candidate will not be added to the candidate list. In one embodiment, the CPR paired average candidate will not be added to the candidate list. For example, if the reference screen of the CPR paired average candidate points to the current screen, this CPR paired average candidate will not be added to the candidate list.

[0061] In another embodiment, if all available candidates used to generate a non-subblock STMVP candidate are CPRMV, then the non-subblock STMVP candidate is also a valid candidate. In another embodiment, only CPR candidates are considered valid candidates for averaging. In one example, the MV of a non-subblock STMVP candidate is rounded up to an integer MV. In another example, if the MV of a non-subblock STMVP candidate is not an integer MV, then this non-subblock STMVP candidate will not be added to the candidate list. In another embodiment, CPR candidates are considered invalid candidates when used to generate non-subblock STMVP candidates. For example, if the reference screen of a non-subblock STMVP candidate points to the current screen, then this non-subblock STMVP candidate will not be added to the candidate list.

[0062] In recent video codec designs, Merge mode with Motion Vector Difference (MMVD) has been considered for use in skip mode or merge mode codecs. If the merge or skip flag is enabled, the corresponding MMVD syntax elements are signaled. The concept of MMVD is to extend existing video frames (MVs) to increase the diversity of these MVs in the merge candidate list. First, K MVs from the merge candidate list are selected as extensions of MMVD, where K is a positive integer. Only merge candidates with a preset merge type (MRG_TYPE_DEFAULT_N) can be selected as MMVD extensions. Similar to the general merge codec process, only the merge candidate index needs to be signaled, and the decoder can construct MV information from the merge candidate index and MMVD information. For example, the decoder determines the inter-Dir, reference index, and MVP index by referring to adjacent MV information based on the merge candidate index. From the MMVD information, the decoder determines the number of base MVs, the direction of expansion, and the size of the expansion step. In one embodiment of CPR using MMVD, when both CPR and MMVD are enabled, only merge candidates with an IBC merge type (MRG_TYPE_IBC) are saved as valid MMVD expansions. Thus, all expanded MVs must point to the current screen. In another embodiment, the range of expanded MVs can be constrained to CTUs, CTU rows, or M CTU rows, where M is a positive integer. In one example, MVD scaling in MMVD is disabled when the target candidate is a CPR candidate. In another embodiment, MMVD is inferred to be disabled when CPR is enabled. In yet another embodiment, CPR candidates cannot be selected as MMVD expansions.

[0063] In an embodiment employing dual-tree block segmentation, when both MMVD and CPR are enabled, the chroma component can reuse MMVD information from the luma component regardless of scaling. In another embodiment employing dual-tree block segmentation, when both MMVD and CPR are enabled, the chroma component can derive MMVD information using a merging candidate list for the chroma component. In this way, merging candidates for the luma and chroma components are generated independently. In yet another embodiment employing dual-tree block segmentation, when MMVD and CPR are applied together, the chroma component can derive an extended MV using an extension of the luma component's MMVD, and select the optimal extended MV as the resulting chroma component MV. This resulting MV may differ from the luma component's MV. In this way, only one merging candidate is generated for both the luma and chroma components, and this is compatible with recent designs for merge-mode encoding / decoding.

[0064] Implementing CPR using affine motion compensation includes affine merge mode and affine AMVP mode. Several methods will be described below to improve the integration of CPR and affine motion compensation. In one embodiment, affine modes (including affine merge and affine AMVP modes) are disabled when CPR is enabled. In one embodiment, affine AMVP mode is disabled when the current frame is selected as the reference frame for encoding and decoding the current block; therefore, the flag indicating whether affine AMVP mode is enabled can be omitted. In another embodiment, one or more CPR MVs cannot be used in affine merge mode. In another embodiment, the reference frame cannot be the current frame when an affine mode is selected. In another embodiment, the codeword for selecting the current frame as the reference frame is removed from the reference frame index signal when an affine mode is selected.

[0065] In another embodiment, two or three control points of the affine AMVP mode must be constrained, and only the MV of the current frame can be used for the affine AMVP mode. In one embodiment, when the current frame is selected as the reference frame, only a two-control-point affine mode can be used. In another embodiment, when the current frame is selected as the reference frame, only a three-control-point affine mode can be used. In one embodiment, the three control points required in the affine merge mode can be corner-derived MVs or spatially inherited MVs. If CPR is enabled for the affine merge mode, the MVs for both the luma and chroma components should point to the current frame. For example, when CPR is enabled, the first three corner-derived MVs pointing to the current frame will be used for the affine merge mode. In another example, when CPR is enabled, inherited MVs pointing to the current frame will be used for the affine merge mode. In yet another example, when CPR is enabled, if fewer than three of the corner-derived MVs from the current frame are available, the spatially inherited MV from the current frame will be used for the affine merge mode. In one example, the first N spatially inherited MVs from the current frame and the first M corner-derived MVs from the current frame can be used as control points in affine merge mode. Both N and M are integers greater than 0. In another embodiment, when the current frame is selected as the reference frame, the integer MV flag for list 0 is deduced to be true, or imv_idx is 1 or 2. In another embodiment, when the current frame is selected as the reference frame, fractional MVs are allowed in affine mode; an integer MV flag for list 0, list 1, or both lists 0 and 1 is signaled to indicate whether the MV is encoded as an integer. In another embodiment, the MV of each sub-block encoded in affine mode is carried over to an integer MV.

[0066] In some other embodiments, affine merge mode or sub-block merge mode, such as affine mode and Advanced Temporal Motion Vector Prediction (ATMVP) mode, can be signaled using separate candidate lists and separate syntaxes. When CPR is enabled and in an I-slice, or when CPR is enabled and all reference frames are the current frame or only one reference frame is the current frame, affine merge mode or sub-block merge mode is disabled or forced not to be selected.

[0067] In another embodiment applying dual-tree block segmentation, affine motion compensation and CPR are enabled simultaneously, and the CU structures for the luma and chroma components can be different. Affine AMVP encoding / decoding for the luma and chroma components can be applied independently. In another embodiment, the two merge candidate lists for the luma and chroma components are generated independently, so the luma and chroma components can independently apply affine merging modes. In yet another embodiment, the chroma component can reuse the control points of the luma component by scaling the encoding / decoding for the affine merging mode.

[0068] In one embodiment of CPR using sub-block ATMVP, ATMVP is disabled if CPR is enabled. In another embodiment applying dual-tree block segmentation, both ATMVP and CPR are enabled, and the initial MV for the chroma component of the ATMVP can be reused when the luminance component's MV points to the current frame. Chroma ATMVP is disabled when no luminance MV points to the current frame. In one embodiment, a CPR MV from a luminance merging block is considered an ATMVP candidate for the current chroma block. In one embodiment, the merging frame for ATMVP mode cannot be the current frame. When a merging frame for ATMVP mode or for time merging is sent, the current frame is removed from the candidate merging frame list; the current frame cannot be sent, or the current frame is forcibly not sent. In one embodiment, when a pending MV points to the current frame, the validity of the MV is checked before adding the pending candidate to the candidate list. For example, if all reference samples in the reference block pointed to by the CPR MV are decoded and stored in the CPR reference buffer, the CPR MV will be marked as valid after a validity check. For instance, if the CPR reference buffer stores decoded samples of the current CTU and the left CTU region, a valid CPR MV will point to a reference block in that current CTU or left CTU region. Invalid CPR MVs will not be added to the merge candidate list or the AMVP candidate list.

[0069] The update rules for the CPR reference buffer: A Virtual Pipeline Data Unit (VPDU) is defined as the basic unit of pipeline processing. It is a hardware-friendly design that divides the video frame into several fixed-size VPDUs during video encoding or decoding. Each video frame is divided into several non-overlapping VPDUs for pipeline processing, and subsequent VPDUs can be processed simultaneously through multiple pipeline stages. The idea behind defining VPDUs is to constrain the buffer size to a pipeline architecture that decodes different CUs in parallel across different pipeline stages. A VPDU can be a square unit (e.g., each VPDU contains MxM luma pixels or NxN chroma pixels) or a non-square unit. The area of ​​a VPDU can be defined by a region (e.g., 4096 pixels) or a size (e.g., 64x64). In the hardware decoder, each VPDU is processed through one pipeline stage after another until each VPDU has been processed through all video decoding pipeline stages. The size or region of the VPDU is predefined. Different VPDUs are processed simultaneously at different stages of the hardware decoder pipeline. Limiting the size of the VPDU to a certain size limit is an important design issue in hardware implementation because the size of the VPDU is roughly proportional to the buffer size in most stages of the pipeline. One embodiment of the VPDU size is set to be equal to the size of the maximum conversion block (TB), which is 32x32 luma pixels or 16x16 chroma pixels for the HEVC standard 4:2:0 color format, or 64x64 luma pixels or 32x32 chroma pixels for the next-generation video standard 4:2:0 color format.

[0070] In the current design of encoding and decoding the current CU in the current codec tree unit (CTU) using CPR mode, when the current frame is selected as the reference frame, only samples from the current CTU can be referenced by that current CU. Therefore, when processing the first CU within the current CTU, no valid reference samples are available for use in CPR mode. A CPR reference buffer, the same size as the CTU, is needed to store the decoded samples from the current CTU. This CPR reference buffer is reset after decoding the CTU and updated after each CU is decoded. The CPR reference buffer is updated CU-by-CU. Only the decoded samples stored in the CPR reference buffer can be used in CPR mode.

[0071] In some embodiments of the invention, after a VPDU decoding process is completed, the decoded samples stored in the CPR reference buffer are updated; that is, the CPR reference buffer is updated one VPDU after another. A VPDU can be a CU with 64x64 luminance samples, or a VPDU can be set to have the same size as the block with the maximum conversion size. The CPR reference buffer is a first-in-first-out (FIFO) buffer. In one embodiment, the decoded samples stored in the CPR reference buffer can be used to decode the current CU encoded in CPR mode, regardless of whether the decoded samples and the current CU are in the same CTU. For example, one or more decoded samples in the reference block stored in the CPR reference buffer for predicting the current CU are not in the same CTU as the current CU. In an exemplary embodiment, after a VPDU decoding process is completed, the CPR reference buffer is updated, and the decoded samples stored in the CPR reference buffer can be used to decode the current CU through CPR mode, regardless of whether the decoded samples and the current CU are in the same CTU.

[0072] A representative flowchart used in the embodiment is based on an embodiment that applies Current Picture Reference (CPR) using a method based on Historical Motion Vector Prediction (HMVP). Figure 7 This is a flowchart illustrating the video processing method used to process the current block. Figure 7The video processing method illustrated herein can be implemented in a video encoder or a video decoder. In step S702, the video encoder or video decoder receives input data for the current block in the current frame; and the current block is encoded or to be encoded by inter-frame prediction or CPR. The reference frame for the current block encoded or to be encoded by CPR is the current frame. In step S704, it is checked whether the current block is to be processed by a general inter-frame prediction mode or a CPR mode; in step S706, if the current block is processed by a general inter-frame prediction mode such as skip, merge, or AMVP mode, the candidate list for the current block is constructed by incorporating one or more HMVP candidates obtained from a general HMVP table; in step S708, if the current block is processed by a CPR mode, the candidate list for the current block is constructed by incorporating one or more HMVP candidates obtained from a CPR HMVP table. The general HMVP table and the CPR HMVP table are maintained separately. The general HMVP table is updated using motion information of blocks encoded and decoded in the general inter-frame prediction mode, while the CPR HMVP table is updated using motion information of blocks encoded and decoded in the CPR mode. In step S710, a candidate is selected from the candidate list for the current block. For example, the video encoder selects the candidate for the current block based on the Rate Distortion Optimization (ROD) decision, or the video decoder selects the candidate for the current block based on the index parsed and decoded by the entropy decoder. In step S712, a reference block is located in the reference frame based on the motion information of the selected candidate. If the current block is processed using the CPR mode, the reference frame is the current frame. In step S714, the current block is predicted using the reference block, and then encoded or decoded.

[0073] Figure 8 To illustrate a video processing method for processing the current block, based on an embodiment of CPR, a flowchart is drawn. Figure 8The video processing method illustrated can be implemented in a video decoder. In step S802, the video decoder receives input data from the current Virtual Pipeline Data Unit (VPDU) in the current frame, and CPR is enabled in the current frame. A VPDU is defined by a maximum TB size or a predefined size or region. In step S804, a decoding process is executed to decode the CUs in the current VPDU, and a CPR reference buffer is used to decode each CU encoded and decoded in CPR mode. In step S806, the CPR reference buffer is updated by storing the decoded sample of the current VPDU, and subsequent VPDUs are set as the current VPDU. The current VPDU is then processed from steps S802 to S806. According to this embodiment, the CPR reference buffer is updated one VPDU after another.

[0074] A representative block diagram illustrates that any of the video processing methods described above can be implemented on an encoder, a decoder, or both. For example, any of the video processing methods described can be implemented on the CPR prediction module or inter-frame prediction module of the encoder or decoder. Alternatively, any of the video processing methods described can be implemented on the circuitry coupled to the CPR prediction module or inter-frame prediction module of the encoder or decoder to provide the information required by the CPR prediction module or inter-frame prediction module. Figure 9An exemplary system block diagram of a video encoder 900 implementing one or more video processing methods of the present invention is illustrated. The video encoder 900 receives input data for the current block in the current frame. If the current block is to be processed by intra-frame prediction, the intra-frame prediction module 910 provides an intra-frame predictor for the current block based on the intra-frame prediction mode and reconstructed samples of the current frame. If the current block is to be processed by CPR, the CPR prediction module 911 provides a CPR predictor based on the CPR MV of the current block. In some embodiments, the CPR MV of the current block is derived from candidates selected from a candidate list for the current block, and the candidate list includes one or more HMVP candidates obtained from a CPR HMVP table. The CPR prediction module 911 obtains reconstructed samples stored in a CPR reference buffer to generate the CPR predictor, and according to an embodiment of the present invention, the CPR reference buffer is updated one VPDU after another. If the current block is to be processed by inter-frame prediction, the inter-frame prediction module 912 performs motion estimation (ME) and motion compensation (MC) based on one or two video moments (MVs) of the current block and video data from one or more other frames to provide inter-frame predictors. In some embodiments of the invention, the one or two MVs are derived from candidates selected from a candidate list for the current block, and this candidate list includes one or more HMVP candidates obtained from a general HMVP table. The CPR HMVP table and the general HMVP table are maintained and updated separately. One of the intra-frame prediction module 910, the CPR prediction module 911, and the inter-frame prediction module 912 provides the selected predictors to the addition module 916 to form the prediction error, also known as the prediction residual.

[0075] The prediction residual of the current block is further processed by the conversion module (T) 918, followed by the quantization module (Q) 920. The converted and quantized residual signals are then encoded by the entropy encoder 934 to form an encoded video bitstream. The encoded video bitstream is then packaged with side information. The converted and quantized residual signals of the current block are processed by the inverse quantization module (IQ) 922 and the inverse conversion module (IT) 924 to recover the prediction residual. Figure 9As shown, the prediction residual is restored by adding back the selected predictor at the reconstruction module (REC) 926 to produce reconstructed samples. The reconstructed samples can be stored in the reference picture buffer (Ref.Pict.Buffer) 932 and used to predict other pictures. The reconstructed samples from REC 926 may be subject to various degradations due to encoding processing; therefore, before being stored in the reference picture buffer 932, the in-loop processing deblocking filter (DF) 928 and the sample adaptive offset (SAO) 930 can be applied to the reconstructed samples to further improve picture quality. The syntax associated with the information used for the in-loop processing deblocking filter DF 928 and the sample adaptive offset SAO 930 is provided to the entropy encoder 934 for incorporation into the encoded video bitstream.

[0076] Used for decoding Figure 9 The corresponding video decoder 1000 is as follows: (The video bitstream generated by the video encoder 900 is shown in the image.) Figure 10As shown. The encoded video bitstream is the input to the video decoder 1000 and is decoded by the entropy decoder 1010 to analyze and restore the transformed and quantized residual signals and other system information. Except that the decoder 1000 only requires motion compensation prediction in the inter-frame prediction module 1014, the decoding process of the decoder 1000 is similar to the reconstruction loop at the encoder 900. In some embodiments of the invention, two HMVP tables are maintained separately to decode CUs encoded in general inter-frame prediction mode and CUs encoded in CPR mode. For example, the current block is decoded by the intra-frame prediction module 1012, the CPR prediction module 1013, or the inter-frame prediction module 1014, and if the current block is encoded in general inter-frame prediction mode, a general HMVP table is used to construct a candidate list for the current block. If the current block is encoded in CPR mode, a CPR HMVP table is used to construct a candidate list for the current block. Based on the decoded mode information, the switch module 1016 selects an intra-predictor from the intra-prediction module 1012, a CPR predictor from the CPR predictor module 1013, or an inter-prediction module 1014. The converted and quantized residual signals associated with each block are reconstructed using the inverse quantization (IQ) module 1020 and the inverse transformation (IT) module 1022. The reconstructed converted and quantized residual signals are reconstructed by adding the predictors back in the reconstruction module 1018 to generate reconstructed samples. The reconstructed samples are further processed by the deblocking filter 1024 and the sample adaptive offset 1026 to produce the final decoded video. If the currently decoded frame is a reference frame, the reconstructed samples of the currently decoded frame are also stored in the reference frame buffer 1028 for use in subsequent frames in the decoding sequence.

[0077] Figure 9 and Figure 10The various components of the video encoder 900 and video decoder 1000 can be implemented by hardware components, one or more processors configured to execute program instructions stored in memory, or a combination of hardware and processors. For example, the processor executes program instructions to control the reception of input data associated with the current block in the current frame. The processor is equipped with one or more processing cores. In some examples, the processor executes program instructions to perform functions in some components of the encoder 900 and decoder 1000, and memory electrically coupled to the processor is used to store program instructions, information of the reconstructed image corresponding to the block, and / or intermediate data during the encoding or decoding process. In some embodiments, the memory includes non-transitory computer-readable media, such as semiconductor or solid-state memory, random access memory (RAM), read-only memory (ROM), hard disk, optical disk, or other suitable storage media. The memory can also be a combination of two or more non-transitory computer-readable media listed above. Figure 7 and Figure 8 As shown, the encoder 900 and decoder 1000 can be implemented in the same electronic device; therefore, if implemented in the same electronic device, various functional components of the encoder 900 and decoder 1000 can be shared or reused. For example, Figure 9 One or more of the following components—reconstruction module 926, inverse conversion module 924, inverse quantization module 922, deblocking filter 928, sample adaptive offset 930, and reference frame buffer 932—can also be used as... Figure 10 The system includes a reconstruction module 1018, an inverse conversion module 1022, an inverse quantization module 1020, a deblocking filter 1024, a sample adaptive offset 1026, and a reference frame buffer 1028.

[0078] Embodiments of the processing method in a video encoding / decoding system can be implemented in circuitry integrated into a video compression chip, or in program code integrated into video compression software to execute the processing described above. For example, a candidate list for encoding or decoding blocks in a general inter-frame prediction mode or CPR mode can be implemented in program code that executes on a computer processor, digital signal processor (DSP), microprocessor, or field-programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the invention by executing machine-readable software code or firmware code that defines the specific methods embodied in the invention.

[0079] The invention may be implemented in other specific forms without departing from the spirit or essential characteristics thereof. The examples described are to be considered illustrative in all respects and not restrictive. Therefore, the scope of the invention is indicated by the appended claims, not by the foregoing description. All variations within the meaning and scope of equivalents that fall within the scope of the claims will be included within its scope.

Claims

1. A method of processing video data in a video decoding system, comprising: receiving input data associated with a current picture containing a plurality of virtual pipe data units, wherein each virtual pipe data unit is of a fixed size; performing a decoding process to decode a coded unit in a current virtual pipe data unit and to generate decoded samples of the current virtual pipe data unit, wherein each coded unit coded in a current picture reference mode is decoded by retrieving decoded samples of a reference block stored in a current picture reference reference buffer; and after the decoding process of the current virtual pipe data unit is completed, updating the current picture reference reference buffer by storing the decoded samples of the current virtual pipe data unit, wherein the current picture reference reference buffer is updated in a virtual pipe data unit by a virtual pipe data unit basis.

2. The method of processing video data in a video decoding system according to claim 1, wherein, The size of each virtual pipe data unit is fixed at 64x64 luma samples or 32x32 chroma samples for a 4:2:0 color sample format.

3. The method of processing video data in a video decoding system according to claim 1, wherein, The size of each virtual pipe data unit is set to be equal to the size of a largest transform block.

4. The method of processing video data in a video decoding system according to claim 1, wherein, The decoding process for each coded unit includes decoding mode information, performing intra prediction, inter prediction, or current picture reference in accordance with the mode information to generate a predictor for the coded unit, and dequantizing and de-transforming a residual signal, and reconstructing the coded unit in accordance with the dequantized and de-transformed residual signal and the predictor to generate the decoded samples.

5. The method of processing video data in a video decoding system according to claim 1, wherein, One or more decoded samples of a reference block used to decode a current coded unit coded in the current picture reference mode and stored in the current picture reference reference buffer, and the current coded unit are not in the same coding tree unit.

Images