Prediction using intra buffer samples for intra block copy in video codecs

Abstract

A method of visual media processing comprises: determining a size of a buffer to store reference samples for prediction in an intra block copy mode; and performing conversion between a current video block of visual media data and a bitstream representation of the current video block using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is intended to timely claim, under applicable patent law and / or the rules pursuant to the Paris Convention, International Patent Application No. PCT / CN2019 / 074598 filed on February 2, 2019, International Patent Application No. PCT / CN2019 / 076695 filed on March 1, 2019, International Patent Application No. PCT / CN2019 / 076848 filed on March 4, 2019, International Patent Application No. PCT / CN2019 / 077725 filed on March 11, 2019, International Patent Application No. PCT / CN2019 / 079151 filed on March 21, 2019, International Patent Application No. PCT / CN2019 / 085862 filed on May 7, 2019, International Patent Application No. PCT / CN2019 / 088129 filed on May 23, 2019, and International Patent Application No. PCT / CN2019 / 088139 filed on June 18, 2019. PCT / CN2019 / 091691 filed on July 24, 2019, International Patent Application No. PCT / CN2019 / 093552 filed on June 28, 2019, International Patent Application No. PCT / CN2019 / 094957 filed on July 6, 2019, International Patent Application No. PCT / CN2019 / 095297 filed on July 9, 2019, International Patent Application No. PCT / CN2019 / 095504 filed on July 10, 2019, International Patent Application No. PCT / CN2019 / 095656 filed on July 11, 2019, International Patent Application No. PCT / CN2019 / 095913 filed on July 13, 2019, and International Patent Application No. PCT / CN2019 / 096048 filed on July 15, 2019. The entire disclosure of the foregoing application is incorporated by reference as a part of the disclosure of the present application for all purposes under law. Technical Field

[0003] This patent document relates to video encoding and decoding technologies, devices, and systems. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video usage is expected to continue to grow. Summary of the Invention

[0005] This document describes various embodiments and techniques for buffer management and block vector codecs for intra block copy mode for decoding or encoding video or images.

[0006] In one example aspect, a method for video or image (visual data) processing is disclosed, the method comprising: determining a size of a buffer to store reference samples for prediction in an intra block copy mode; and performing conversion between a current video block of visual media data and a bitstream representation of the current video block using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0007] In another example aspect, another method of visual data processing is disclosed, the method comprising: for a conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reconstructed samples for prediction in an intra block copy mode, wherein the buffer is configured to store the reconstructed samples before a loop filtering step; and performing the conversion using the reconstructed samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0008] In yet another example aspect, another method of visual data processing is disclosed, the method comprising: for a conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer for storing reconstructed samples for prediction in an intra block copy mode, wherein the buffer is configured to store the reconstructed samples after a loop filtering step; and performing the conversion using the reconstructed samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0009] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reconstructed samples for prediction in an intra block copy mode, wherein the buffer is configured to store both the reconstructed samples before and after a loop filtering step; and performing the conversion using the reconstructed samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0010] In another example aspect, another method of video processing is disclosed, the method comprising: using a buffer to store reference samples for prediction in an intra block copy mode, wherein a first bit depth of the buffer is different from a second bit depth used to represent visual media data in a bitstream representation; and using the reference samples stored in the buffer to perform conversion between a current video block of the visual media data and a bitstream representation of the current video block, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0011] In yet another example aspect, another method of video processing is disclosed, the method comprising: initializing a buffer to store reference samples for prediction in an intra block copy mode, wherein the buffer is initialized with a first value; and performing a conversion between a current video block of visual media data and a bitstream representation of the current video block using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0012] In yet another example aspect, another method of video processing is disclosed, the method comprising: initializing a buffer to store reference samples for prediction in an intra block copy mode, wherein the buffer is initialized with pixel values ​​of the one or more video blocks in the visual media data based on availability of one or more video blocks in the visual media data; and performing a conversion between a current video block that is not part of the one or more video blocks of the visual media data and a bitstream representation of the current video block using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0013] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reference samples for prediction in an intra block copy mode; performing the conversion using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; and for a pixel spatially located at a position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, calculating a corresponding reference (P mod M, Q mod N) in the buffer based on the reference position, wherein “mod” is a modulo operation and M and N are integers representing the x- and y-dimensions of the buffer, wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0).

[0014] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reference samples for prediction in an intra block copy mode; performing the conversion using the reference samples stored in the buffer, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; and for a pixel spatially located at a position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, calculating a corresponding reference in the buffer based on a reference position (P, Q), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0).

[0015] In yet another example aspect, another method of video processing is disclosed, the method comprising: for a conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode, wherein pixel locations in the buffer are addressed using an x-number and a y-number; and performing the conversion using the reference samples stored in the buffer based on the x-number and the y-number, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[0016] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reference samples for prediction in an intra block copy mode, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; for a pixel spatially located at a position (x0, y0) of the current video block and having a block vector (BVx, BVy), calculating a corresponding reference in the buffer at a reference position (P, Q), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0); and when it is determined that the reference position (P, Q) is outside the buffer, recalculating the reference position using samples in the buffer.

[0017] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reference samples for prediction in an intra block copy mode, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; for a pixel spatially located at a position (x0, y0) of the current video block relative to a top left corner of a codec tree unit including the current video block and having a block vector (BVx, BVy), calculating a corresponding reference in the buffer at a reference position (P, Q), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0); and, when the reference position (P, Q) is determined to be outside the buffer, constraining at least a portion of the reference position to be within a predefined range.

[0018] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block of visual media data and a bitstream representation of the current video block, a buffer storing reference samples for prediction in an intra block copy mode, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; for a pixel spatially located at a position (x0, y0) of the current video block relative to a top-left corner of a codec tree unit including the current video block and having a block vector (BVx, BVy), calculating a corresponding reference in the buffer at a reference position (P, Q), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0); and when the block vector (BVx, BVy) is determined to be outside the buffer, filling the block vector (BVx, BVy) with a block vector of sample values ​​within the buffer.

[0019] In yet another example aspect, another method of video processing is disclosed, the method comprising: during conversion between a video and a bitstream representation of the video, resetting a buffer storing reference samples used for prediction in an intra block copy mode at a video boundary; and performing the conversion using the reference samples stored in the buffer, wherein the conversion of a video block of the video is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the video block and does not reference a reference picture.

[0020] In yet another example aspect, another method of video processing is disclosed, the method comprising: performing a conversion between a current video block and a bitstream representation of the current video block; and updating a buffer for storing reference samples for prediction in an intra block copy mode, wherein the buffer is used for a conversion between a subsequent video block and a bitstream representation of the subsequent video block, wherein the conversion between the subsequent video block and the bitstream representation of the subsequent video block is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the subsequent video block and does not reference a reference picture.

[0021] In yet another example aspect, another method of video processing is disclosed, the method comprising: determining, for a conversion between a current video block and a bitstream representation of the current video block, a buffer for storing reconstructed samples for prediction in an intra block copy mode, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture; and, in response to determining that the reconstructed samples stored in the buffer are to be used for predicting sample values ​​during the conversion, applying a pre-processing operation to the reconstructed samples stored in the buffer.

[0022] In yet another example aspect, another method of video processing is disclosed, the method comprising: for converting between a current video block of a current virtual pipe data unit (VPDU) of a video region and a bitstream representation of the current video block, selectively determining whether to use K1 previously processed VPDUs from even lines of the video region and / or K2 previously processed VPDUs from odd lines of the video region; and performing the conversion, wherein the conversion does not include using a remaining portion of the current VPDU, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the video block and without reference to a reference picture.

[0023] In yet another example aspect, a video encoder or decoder apparatus is disclosed, comprising a processor configured to implement the above method.

[0024] In another exemplary aspect, a computer-readable program medium is disclosed, wherein the medium stores code embodying processor-executable instructions for implementing one of the disclosed methods.

[0025] These and other aspects are described in more detail throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Examples of current picture reference or intra block copy video or image coding techniques are shown.

[0027] Figure 2 An example of a dynamic reference region is shown.

[0028] Figure 3 An example of encoding and decoding of a block starting from (x, y) is shown.

[0029] Figure 4 Examples of possible alternative ways of selecting a previously coded 64x64 block are shown.

[0030] Figure 5 Examples of possible alternative ways of changing the encoding / decoding order of 64x64 blocks are shown.

[0031] Figure 6 is a flowchart of an example method of video or image processing.

[0032] Figure 7 It is a block diagram of a hardware platform for video or image encoding or decoding.

[0033] Figure 8 Another possible alternative way of selecting a previously coded 64x64 block when the decoding order of 64x64 blocks is from top to bottom, left to right is shown.

[0034] Figure 9 Another possible alternative way of selecting a previously coded 64x64 block is shown.

[0035] Figure 10 An example flow chart of a decoding process with shaping is shown.

[0036] Figure 11 Another possible alternative way of selecting a previously coded 64x64 block when the decoding order of 64x64 blocks is from left to right and from top to bottom is shown.

[0037] Figure 12 is an illustration of the IBC reference buffer state, where a block represents a 64×64 CTU.

[0038] Figure 13 One arrangement of reference areas for an IBC is shown.

[0039] Figure 14 Another arrangement of reference areas for an IBC is shown.

[0040] Figure 15 Another arrangement of the reference area for IBC is shown when the current Virtual Pipeline Data Unit (VPDU) is on the right side of the picture boundary.

[0041] Figure 16 An example of the state of the virtual buffer when VPDUs in a CTU row are decoded sequentially is shown.

[0042] Figure 17 is a block diagram of an example video processing system in which the disclosed technology may be implemented.

[0043] Figure 18 is a flow chart of an example method of visual data processing.

[0044] Figure 19 is a flow chart of an example method of visual data processing.

[0045] Figure 20 is a flow chart of an example method of visual data processing.

[0046] Figure 21 is a flow chart of an example method of visual data processing.

[0047] Figure 22 is a flow chart of an example method of visual data processing.

[0048] Figure 23 is a flow chart of an example method of visual data processing.

[0049] Figure 24 is a flow chart of an example method of visual data processing.

[0050] Figure 25 is a flow chart of an example method of visual data processing.

[0051] Figure 26 is a flow chart of an example method of visual data processing.

[0052] Figure 27 is a flow chart of an example method of visual data processing.

[0053] Figure 28 is a flow chart of an example method of visual data processing.

[0054] Figure 29 is a flow chart of an example method of visual data processing.

[0055] Figure 30 is a flow chart of an example method of visual data processing.

[0056] Figure 31 is a flow chart of an example method of visual data processing.

[0057] Figure 32 is a flow chart of an example method of visual data processing.

[0058] Figure 33 is a flow chart of an example method of visual data processing.

[0059] Figure 34 is a flow chart of an example method of visual data processing. DETAILED DESCRIPTION

[0060] Section headings are used in this document to facilitate understanding and do not limit the scope of the embodiments disclosed in each section to that section. This document describes various embodiments and techniques for buffer management and block vector codecs for intra block copy mode for decoding or encoding video or images.

[0061] 1. Summary

[0062] This patent document relates to video codec technology. Specifically, it relates to intra-frame block replication in video codecs. The patent application may be applied to standards currently under development, such as the Multi-Function Video Codec. The patent application may also be applicable to future video codec standards or video codecs.

[0063] 2. Brief Discussion

[0064] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Video and 264 / MPEG-4 Advanced Video Coding (AVC) standards, and the H.265 / HEVC standard. Since H.262, video codec standards have been based on a hybrid video codec architecture that utilizes temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software named the Joint Exploration Model (JEM). In April 2018, a Joint Video Expert Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard, targeting a 50% bitrate reduction compared to HEVC.

[0065] 2.1 Inter-frame prediction in HEVC / H.265

[0066] Each inter-prediction PU has motion parameters for one or two reference picture lists. The motion parameters include motion vectors and reference picture indices. The use of one of the two reference picture lists can also be signaled using inter_pred_idc. The motion vector can be explicitly encoded as a delta relative to the prediction value.

[0067] When a CU is encoded or decoded in skip mode, one PU is associated with the CU and there are no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified, whereby the motion parameters of the current PU are obtained from neighboring PUs including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just for skip mode. An alternative to Merge mode is explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference (MVD) compared to the motion vector prediction value), the corresponding reference picture index of each reference picture list, and the reference picture list usage are explicitly signaled per PU. Such a mode is referred to as Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0068] When signaling indicates that one of the two reference picture lists will be used, a PU is generated from a block of samples. This is called "unidirectional prediction." Unidirectional prediction applies to both P slices and B slices.

[0069] When signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bi-prediction." Bi-prediction is only applicable to B slices.

[0070] The following provides details about the inter prediction modes specified in HEVC. The description will start with the Merge mode.

[0071] 2.2 Current Image Reference

[0072] Current Picture Referencing (CPR), once named Intra Block Copy (IBC), has been adopted in HEVC Screen Content Coding extension (HEVC-SCC) and the current VVC test model. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 1As shown in the figure, when CPR is applied, the current block is predicted by the reference block in the same picture. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. Although CPR is not efficient for sequences captured by most cameras, it shows significant coding and decoding gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. CPR can effectively remove the redundancy between these repeated patterns. In HEVC-SCC, if the coding unit (CU) of inter-frame coding selects the current picture as its reference picture, it can apply CPR. In this case, MV is renamed block vector (BV), and BV always has integer pixel accuracy. For compatibility with the main profile HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that similarly, in the multi-view / 3D video coding standard, inter-view reference pictures are also marked as "long-term" reference pictures.

[0073] After the BV finds its reference block, it can generate a prediction by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signaling. Then the transform and quantization can be applied as in other codec modes.

[0074] Figure 1 This is an example illustration of the current image reference.

[0075] However, when the reference block is outside the picture, overlaps with the current block, is outside the reconstruction area, or is outside the valid area subject to certain constraints, some or all pixel values ​​are undefined. There are basically two solutions to this problem. One is to disallow this situation, for example, in bitstream conformance. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0076] 2.3 CPR in HEVC Screen Content Codec Extension

[0077] In the screen content codec extension of HEVC, when a block uses the current picture as a reference, it should ensure that the entire reference block is within the available reconstruction area, as indicated in the following specification text:

[0078] The variables offsetX and offsetY are derived as follows:

[0079] offsetX=(ChromaArrayType==0)? 0:(mvCLX[0]&0x7?2:0) (8-104)

[0080] offsetY=(ChromaArrayType==0)? 0:(mvCLX[1]&0x7?2:0) (8-105)

[0081] One requirement for bitstream conformance is that when the reference picture is the current picture, the luma motion vector mvLX shall obey the following constraints:

[0082] – When the derivation process of z-scan order block availability as specified in clause 6.4.1 is called with as input (xCurr, yCurr) set equal to (xCb, yCb) and the adjacent luma position (xNbY, yNbY) set equal to (xPb+(mvLX[0]>>2)-offsetX, yPb+(mvLX[1]>>2)-offsetY), the output shall be equal to TRUE.

[0083] – When the derivation of z-scan order block availability as specified in clause 6.4.1 is called with as input (xCurr, yCurr) set equal to (xCb, yCb) and the neighboring luma position (xNbY, yNbY) set equal to (xPb+(mvLX[0]>>2)+nPbW-1+offsetX, yPb+(mvLX[1]>>2)+nPbH-1+offsetY), the output shall be equal to TRUE.

[0084] – One or both of the following conditions should be true:

[0085] –The value of (mvLX[0]>>2)+nPbW+xB1+offsetX is less than or equal to 0.

[0086] The value of –(mvLX[1]>>2)+nPbH+yB1+offsetY is less than or equal to 0.

[0087] – The following conditions should be true:

[0088] (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCb / CtbSizeY<=yCb / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY (8-106)

[0089] Therefore, there is no situation where the reference block overlaps with the current block or is outside the picture. There is no need to pad the reference or prediction blocks.

[0090] 2.4 Example of CPR / IBC

[0091] In the VVC test model, the entire reference block should have the current Coding Tree Unit (CTU) and does not overlap with the current block. Therefore, there is no need to pad the reference or prediction block.

[0092] When dual tree is enabled, the partitioning structure can be different from luma CTU to chroma CTU. Therefore, for the 4:2:0 color format, a chroma block (e.g., CU) can correspond to a collocated luma region that has been divided into multiple luma CUs.

[0093] Chroma blocks can only be encoded or decoded in CPR mode when the following conditions should be true:

[0094] 1) Each luminance CU in a collocated luminance block should be coded and decoded in CPR mode

[0095] 2) Each of the BVs of the luma 4×4 block is first converted into the BV of the chroma block, and the BV of the chroma block is a valid BV.

[0096] If either of the two conditions is false, the chroma block shall not be encoded or decoded in CPR mode.

[0097] Note that the definition of "valid BV" has the following constraints:

[0098] 1) All samples within the reference block identified by the BV should be within the restricted search range (eg, in the current VVC design, should be within the same CTU).

[0099] 2) All samples within the reference block identified by BV have been reconstructed.

[0100] 2.5 Example of CPR / IBC

[0101] In some examples, the reference region for CPR / IBC is limited to the current CTU, which is up to 128 × 128. The reference region is dynamically changed to reuse memory to store reference samples for CPR / IBC, so that a CPR / IBC block can have more reference candidates, while the reference buffer for CPR / IBC can be maintained or reduced from one CTU.

[0102] Figure 2 A method is shown where the block is 64×64 and the CTU contains four 64×64 blocks. When encoding or decoding a 64×64 block, the previous three 64×64 blocks can be used as references. By doing so, the decoder only needs to store four 64×64 blocks to support CPR / IBC.

[0103] Assume that the position of the current luma CU relative to the top left corner of the picture is (x, y), and the block vector is (BVx, BVy). In the current design, whether the BV is valid can be determined by whether the luma position ((x+BVx)>>6<<6+(1<<7), (y+BVy)>>6<<6) has not been reconstructed and ((x+BVx)>>6<<6+(1<<7), (y+BVy)>>6<<6) is not equal to (x>>6<<6, y>>6<<6).

[0104] 2.6 In-loop Reshaping (ILR)

[0105] The basic idea of ​​in-loop reshaping (ILR) is to convert the original (in the first domain) signal (prediction / reconstruction signal) into the second domain (the shaped domain).

[0106] The loop luminance shaper is implemented as a pair of look-up tables (LUTs), but only one of the two LUTs needs to be signaled because the other LUT can be calculated from the signaled LUT. Each LUT is a one-dimensional, 10-bit, 1024-entry mapping table (1D-LUT). One LUT is the forward LUT, FwdLUT, which takes the input luminance code value Y i Mapped to the change value Y r :Y r =FwdLUT[Y i ]. The other LUT is the inverse LUT, InvLUT, which changes the code value Y r Map to ( Indicates Y i reconstructed value).

[0107] 2.6.1 PWL Model

[0108] Conceptually, piecewise linear (PWL) is implemented in the following way:

[0109] Assume x1 and x2 are two input pivot points, and y1 and y2 are their corresponding output pivot points for a piece. The output value y for any input value x between x1 and x2 can be interpolated by the following equation:

[0110] y=((y2-y1) / (x2-x1))*(x-x1)+y1

[0111] In a fixed-point implementation, the equation can be rewritten as:

[0112] y=((m*x+2 FP_PREC-1)>>FP_PREC)+c

[0113] Where m is a scalar, c is an offset, and FP_PREC is a constant value specifying the precision.

[0114] In some examples, the PWL model is used to pre-compute the 1024-entry FwdLUT and InvLUT mapping tables; however, the PWL model also allows implementations that compute the same mapping values ​​on the fly without pre-computing the LUTs.

[0115] 2.6.2.1 Brightness Shaping

[0116] An in-loop luma shaping approach provides a lower complexity pipeline that also eliminates the decoding delay of block-by-block intra prediction in inter slice reconstruction. Intra prediction is performed in the shaping domain for both inter and intra slices.

[0117] Regardless of the slice type, intra prediction is always performed in the shaped domain. In this arrangement, intra prediction can start immediately after the previous TU reconstruction is completed. Such an arrangement can also provide a unified process for intra mode, rather than relying on slices. Figure 10 A block diagram of the mode-based CE12-2 decoding process is shown.

[0118] A 16-segment piecewise linear (PWL) model was tested for luma and chroma residual scaling instead of a 32-segment PWL model.

[0119] Inter strip reconstruction with in-loop luma shaper (light green shaded blocks indicate signaling in the shaping domain: luma residual; intra luma predicted; and intra luma reconstructed).

[0120] 2.6.2.2 Luma-dependent Chroma Residual Scaling

[0121] Luma-dependent chroma residual scaling is a multiplication process implemented using fixed-point integer arithmetic. Chroma residual scaling compensates for the interaction between luma and chroma signals. Chroma residual scaling is applied at the TU level. More specifically, the following applies:

[0122] – For intra frames, the reconstructed brightness is averaged.

[0123] – For inter frames, the predicted luminance is averaged.

[0124] The average is used to identify the index in the PWL model. This index identifies the scaling factor cScaleInv. The chroma residual is multiplied by this number.

[0125] Note that the chroma scaling factors are computed based on the predicted luma values ​​from the forward map rather than the reconstructed luma values.

[0126] 2.6.2.3 Signaling Notification of ILR Side Information

[0127] Parameters are (currently) transmitted in the slice group header (similar to ALF). These are said to require 40-100 bits.

[0128] In some examples, added syntax is highlighted in italics.

[0129] Sequence parameter set RBSP syntax in 7.3.2.1

[0130]

[0131] General slice group header syntax in 7.3.3.1

[0132]

[0133]

[0134] Add new syntax slice group shaper model:

[0135]

[0136] In the general sequence parameter set RBSP semantics, add the following semantics:

[0137] sps_reshaper_enabled_flag equal to 1 specifies that the reshaper is used in the Coded Video Sequence (CVS). sps_reshaper_enabled_flag equal to 0 specifies that the reshaper is not used in the CVS.

[0138] In the slice group header syntax, add the following semantics

[0139] tile_group_reshaper_model_present_flag equal to 1 specifies that tile_group_reshaper_model() is present in the slice group header. tile_group_reshaper_model_present_flag equal to 0 specifies that tile_group_reshaper_model() is not present in the slice group header. When tile_group_reshaper_model_present_flag is not present, it is inferred to be equal to 0.

[0140] tile_group_reshaper_enabled_flag equal to 1 specifies that the reshaper is enabled for the current slice group. tile_group_reshaper_enabled_flag equal to 0 specifies that the reshaper is not enabled for the current slice group. When tile_group_reshaper_enable_flag is not present, it is inferred to be equal to 0.

[0141] tile_group_reshaper_chroma_residual_scale_flag equal to specifies that chroma residual scaling is enabled for the current slice group. tile_group_reshaper_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current slice group. When tile_group_reshaper_chroma_residual_scale_flag is not present, it is inferred to be equal to 0. Added tile_group_reshaper_model() syntax

[0142] reshape_model_min_bin_idx specifies the minimum bin (or segment) index to be used during reshape construction. The value of reshape_model_min_bin_idx should be in the range of 0 to MaxBinIdx, inclusive. The value of MaxBinIdx should be equal to 15.

[0143] reshape_model_delta_max_bin_idx specifies the maximum allowed bin (or segment) index MaxBinIdx minus the maximum bin index to be used during reshaper construction. The value of reshape_model_max_bin_idx is set equal to MaxBinIdx - reshape_model_delta_max_bin_idx.

[0144] reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1 specifies the number of bits used to represent the syntax reshape_model_bin_delta_abs_CW[i].

[0145] reshape_model_bin_delta_abs_CW[i] specifies the absolute delta codeword value for the i-th bin. reshaper_model_bin_delta_sign_CW_flag[i] specifies the sign of reshape_model_bin_delta_abs_CW[i], as shown below:

[0146] –If reshape_model_bin_delta_sign_CW_flag[i] is equal to 0, the corresponding variable RspDeltaCW[i] is positive.

[0147] –Otherwise (reshape_model_bin_delta_sign_CW_flag[i] is not equal to 0), the corresponding variable RspDeltaCW[i] is negative.

[0148] When reshape_model_bin_delta_sign_CW_flag[i] is not present, it is inferred to be equal to 0. Variable RspDeltaCW[i] = (1 2*reshape_model_bin_delta_sign_CW[i])*reshape_model_bin_delta_abs_CW[i];

[0149] The variable RspCW[i] is derived as follows:

[0150] The variable OrgCW is set equal to (1 < <BitDepth Y ) / (MaxBinIdx+1).

[0151] – If reshaper_model_min_bin_idx<=i<=reshaper_model_max_bin_idxRspCW[i]=OrgCW+RspDeltaCW[i].

[0152] –Otherwise, RspCW[i]=0.

[0153] If BitDepth Y If the value of is equal to 10, the value of RspCW[i] should be in the range of 32 to 2*OrgCW-1.

[0154] The variable InputPivot[i] (i is in the range of 0 to MaxBinIdx+1, inclusive) is derived as follows

[0155] InputPivot[i]=i*OrgCW

[0156] The variables ReshapePivot[i] (i is in the range of 0 to MaxBinIdx+1, inclusive), ScaleCoef[i] and InvScaleCoeff[i] (i is in the range of 0 to MaxBinIdx, inclusive) are derived as follows:

[0157]

[0158] The variable ChromaScaleCoef[i] (i is in the range of 0 to MaxBinIdx, inclusive) is derived as follows:

[0159] ChromaResidualScaleLut

[64] ={16384,16384,16384,16384,16384,16384,16384,8192,8192,8192,8192 ,5461,5461,5461,5461,4096,4096,4096,4096,3277,3277,3277,3277,2731,2731,2731,2731,2341,234 1,2341,2048,2048,2048,1820,1820,1820,1638,1638,1638,1489,1489,1489,1365,1365,1365,1260,1260,1260,1260,1170,1170,1170,1092,1092,1092,1024,1024,1024};

[0160] shiftC=11

[0161] –if(RspCW[i]==0)

[0162] ChromaScaleCoef[i]=(1< <shiftC)

[0163] – Otherwise (RspCW[i]!=0), ChromaScaleCoef[i]=ChromaResidualScaleLut[RspCW[i]>>1]

[0164] 2.6.2.4 Use of ILR

[0165] On the encoder side, each picture (or slice group) is first converted to the shaped domain. All encoding and decoding processes are performed in the shaped domain. For intra prediction, neighboring blocks are in the shaped domain; for inter prediction, reference blocks (generated from the original domain from the decoded picture buffer) are first converted to the shaped domain. The residual is then generated and encoded and decoded into the bitstream.

[0166] After the entire picture (or slice group) is encoded / decoded, the samples in the shaped domain are converted to the original domain and then the deblocking filter and other filters are applied.

[0167] Disable forward shaping of the prediction signal for the following cases:

[0168] The current block is intra-coded

[0169] The current block is coded as CPR (Current Picture Reference, also known as Intra Block Copy, IBC)

[0170] The current block is coded in Combined Inter-Intra Mode (CIIP) and forward shaping is disabled for intra-predicted blocks.

[0171] 3. Examples of Problems Solved by Various Embodiments

[0172] There are several problems in the current design of CPR / IBC.

[0173] 1) The reference region changes dynamically, which complicates encoder / decoder processing.

[0174] 2) Invalid block vectors are easy to generate and difficult to check, which makes both the encoder and decoder complex.

[0175] 3) Irregular reference regions lead to inefficient encoding and decoding of block vectors.

[0176] 4) It is unclear how to handle CTU sizes smaller than 128×128.

[0177] 5) In the process of determining whether BV is valid or invalid, for chroma blocks, the decision is based on the availability of luma samples, which may lead to erroneous decisions due to the dual-tree partitioning structure.

[0178] 4. Example Embodiments

[0179] In some embodiments, a regular buffer may be used for the CPR / IBC block to obtain a reference.

[0180] The function isRec(x, y) is defined to indicate whether the pixel (x, y) has been reconstructed and referenced by the IBC mode. isRec(x, y) returns false when (x, y) is outside the picture, outside a different slice / slice / brick; isRec(x, y) returns false when (x, y) has not been reconstructed. In another example, when the sample (x, y) has been reconstructed but meets some other conditions, it can also be marked as unusable, such as being outside the reference region / in a different VPDU, and isRec(x, y) returns false.

[0181] The function isRec(c, x, y) is defined to indicate whether sample (x, y) of component c is available. For example, if sample (x, y) has not been reconstructed, it is marked as unavailable. In another example, when sample (x, y) has been reconstructed but meets some other conditions, it can also be marked as unavailable, such as being outside the picture / in a different slice / slice / block / in a different VPDU, or outside the allowed reference area. When sample (x, y) is unavailable, isRec(c, x, y) returns false, otherwise it returns true.

[0182] In the following discussion, the reference samples may be reconstructed samples. Note that a "pixel buffer" may correspond to a "buffer for one color component" or a "buffer for multiple color components."

[0183] Reference buffer for CPR / IBC

[0184] 1. Propose to use an M×N pixel buffer to store luminance reference samples for CPR / IBC.

[0185] a. In one example, the buffer size is 64×64.

[0186] b. In one example, the buffer size is 128×128.

[0187] c. In one example, the buffer size is 64×128.

[0188] d. In one example, the buffer size is 128×64.

[0189] e. In one example, N is equal to the height of the CTU.

[0190] f. In one example, N=nH, where H is the height of the CTU and n is a positive integer.

[0191] g. In one example, M is equal to the width of the CTU.

[0192] h. In one example, M=mW, where W is the width of the CTU and m is a positive integer.

[0193] i. In one example, the buffer size is not equal to the CTU size, such as 96×128 or 128×96.

[0194] j. In one example, the buffer size is equal to the CTU size.

[0195] k. In one example, M=mW and N=H, where W and H are the width and height of the CTU, and m is a positive integer.

[0196] 1. In one example, M=W and N=nH, where W and H are the width and height of the CTU, and n is a positive integer.

[0197] m. In one example, M=mW and N=nH, where W and H are the width and height of the CTU, and m and n are positive integers.

[0198] n. In the above examples, m and n may depend on the CTU size.

[0199] i. In one example, when the CTU size is 128×128, m=1 and n=1.

[0200] ii. In one example, when the CTU size is 64×64, m=4 and n=1.

[0201] iii. In one example, when the CTU size is 32×32, m=16 and n=1.

[0202] iv. In one example, when the CTU size is 16×16, m=64 and n=1.

[0203] o. Alternatively, the buffer size corresponds to the CTU size.

[0204] p. Alternatively, the buffer size corresponds to the Virtual Pipe Data Unit (VPDU) size.

[0205] q. M and / or N may be signaled from the encoder to the decoder, such as in the VPS / SPS / PPS / picture header / slice header / slice group header.

[0206] 2. In different profiles / levels / tiers defined in the standard, M and / or N may be different. It is proposed to use another Mc×Nc pixel buffer to store chroma reference samples for CPR / IBC.

[0207] a. In one example, for 4:2:0 video, Mc=M / 2 and Nc=N / 2

[0208] b. In one example, for 4:4:4 video, Mc=M and Nc=N

[0209] c. In one example, for 4:2:2 video, Mc=M and Nc=N / 2

[0210] Alternatively, Mc and Nc can be independent of M and N.

[0211] e. In one example, the chroma buffer includes two channels, corresponding to Cb and Cr.

[0212] f. In one example, Mc=M and Nc=N.

[0213] 3. Propose to use M×N sample buffer to store RGB reference samples for CPR / IBC.

[0214] a. In one example, the buffer size is 64×64.

[0215] b. In one example, the buffer size is 128×128.

[0216] c. In one example, the buffer size is 64×128.

[0217] d. In one example, the buffer size is 128×64.

[0218] e. Alternatively, the buffer size corresponds to the CTU size.

[0219] f. Alternatively, the buffer size corresponds to the Virtual Pipe Data Unit (VPDU) size.

[0220] 4. It is proposed that a buffer can store reconstructed pixels before loop filtering. Loop filtering can refer to a deblocking filter, an adaptive loop filter (ALF), a sample adaptive offset (SAO), a cross-component ALF, or any other filter.

[0221] a. In one example, the buffer can store samples in the current CTU.

[0222] b. In one example, the buffer can store samples outside the current CTU.

[0223] c. In one example, the buffer can store samples from any part of the current picture.

[0224] d. In one example, the buffer can store samples from other pictures.

[0225] 5. It is proposed that a buffer can store reconstructed pixels after loop filtering. Loop filtering can refer to a deblocking filter, an adaptive loop filter (ALF), a sample adaptive offset (SAO), a cross-component ALF, or any other filter.

[0226] a. In one example, the buffer can store samples in the current CTU.

[0227] b. In one example, the buffer can store samples outside the current CTU.

[0228] c. In one example, the buffer can store samples from any part of the current picture.

[0229] d. In one example, the buffer can store samples from other pictures.

[0230] 6. It is proposed that the buffer can store both reconstructed samples before loop filtering and reconstructed samples after loop filtering. Loop filtering can refer to a deblocking filter, an adaptive loop filter (ALF), sample adaptive offset (SAO), cross-component ALF or any other filter.

[0231] a. In one example, the buffer can store both samples from the current picture and samples from other pictures, depending on the availability of these samples.

[0232] b. In one example, the reference samples from other pictures are reconstructed samples after loop filtering.

[0233] c. In one example, the reference samples from other pictures are from the reconstructed samples before loop filtering.

[0234] 7. It is proposed that the buffer stores samples with a given bit-depth, where the given bit-depth may be different from the bit-depth of the coded video data.

[0235] a. In one example, the bit depth of the reconstructed buffer / encoded video data is greater than the bit depth of the IBC reference samples stored in the buffer.

[0236] b. In one example, even when the internal bit depth is different from the input bit depth of the video sequence, such as (10 bits vs 8 bits), the IBC reference samples are stored aligned with the input bit depth.

[0237] c. In one example, the bit depth is the same as the bit depth of the reconstruction buffer.

[0238] d. In one example, the bit depth is the same as the bit depth of the input image / video.

[0239] e. In one example, the bit depth is the same as a predefined number.

[0240] f. In one example, the bit depth depends on the profile of the standard.

[0241] g. In one example, the bit depth or the bit depth difference compared to the output bit depth / input bit depth / internal bit depth may be signaled in the SPS / PPS / sequence header / picture header / slice header / slice group header / slice header or other kinds of video data units.

[0242] h. The proposed methods can be applied together with the proposed buffer zone definitions mentioned in other project specifications, alternatively, they can also be adapted to existing designs of IBCs.

[0243] i. The bit depth of each color component of the buffer can be different.

[0244] Buffer initialization

[0245] 8. Propose to initialize the buffer with a given value

[0246] a. In one example, a buffer is initialized with a given value.

[0247] i. In one example, a given value may depend on the input bit depth and / or the internal bit depth.

[0248] ii. In one example, the buffer is initialized with a mid-grey value (eg, 128 for an 8-bit signal, or 512 for a 10-bit signal).

[0249] iii. In one example, when ILR is used, the buffer is initialized with forwardLUT(m), for example, m=1<<(Bitdepth-1).

[0250] b. Alternatively, the buffer is initialized with the value signaled in the SPS / VPS / APS / PPS / Sequence Header / Slice Group Header / Picture Header / Slice / CTU / Codec Unit / VPDU / area.

[0251] c. In one example, the given value may be derived from samples of a previously decoded picture or slice or CTU row or CTU or CU.

[0252] d. For different color components, the given value can be different.

[0253] 9. Alternatively, it is proposed to initialize the buffer with decoded pixels from a previously coded block.

[0254] a. In one example, the decoded pixels are those before loop filtering.

[0255] b. In one example, when the buffer size is CTU, the buffer is initialized with decoded pixels of a previously decoded CTU, if available.

[0256] c. In one example, when the buffer size is 64x64, its buffer size is initialized with the decoded pixels of a previously decoded 64x64 block, if available.

[0257] d. Furthermore, alternatively, if no previously coded blocks are available, the method in bullet point 8 may be applied.

[0258] A reference to a buffer

[0259] 10. For a block that uses a pixel in the buffer as a reference, it can use the position (x, y) in the buffer to indicate where to get the reference, where x = 0, 1, 2, ..., M-1; y = 0, 1, 2, ..., N-1.

[0260] 11. Alternatively, the reference position may be expressed as l=y*M+x, l=0, 1, ..., M*N-1.

[0261] 12. Denoting the upper left corner position of the block associated with the current CTU as (x0, y0), the block vector (BVx, BVy) = (x-x0, y-y0) can be transmitted to the decoder to indicate where to get the reference in the buffer.

[0262] 13. Alternatively, the block vector (BVx, BVy) can be defined as (x-x0+Tx, y-y0+Ty), where Tx and Ty are predefined offsets.

[0263] 14. For any pixel (x0, y0) and (BVx, BVy), its reference in the buffer can be found at (x0+BVx, y0+BVy).

[0264] a. In one example, when (x0+BVx, y0+BVy) is outside the buffer, it will be clipped to the boundary.

[0265] b. Alternatively, when (x0+BVx, y0+BVy) is outside the buffer zone, its reference value is predefined as a given value, such as middle gray.

[0266] c. Alternatively, the reference position is defined as ((x0+BVx) mod M, (y0+BVy) mod N) so that it is always within the buffer.

[0267] 15. For any pixel (x0, y0) and (BVx, BVy), when (x0+BVx, y0+BVy)

[0268] When outside the buffer, its reference value can be deduced from the value in the buffer.

[0269] a. In one example, this value is derived from the samples in the buffer ((x0+BVx) mod M, (y0+BVy) mod N).

[0270] b. In one example, this value is derived from the sample point in the buffer ((x0+BVx) mod M, clip(y0+BVy, 0, N-1)).

[0271] c. In one example, this value is derived from the sample in the buffer (clip(x0+BVx, 0, M-1), (y0+BVy) mod N).

[0272] d. In one example, this value is derived from the samples in the buffer (clip(x0+BVx, 0, M-1), clip(y0+BVy, 0, N-1)).

[0273] 16. It may not allow specific coordinates outside the buffer range.

[0274] a. In one example, for any pixel (x0, y0) and block vector (BVx, BVy) relative to the top left corner of a CTU, the bitstream constraint is that y0+BVy should be in the range [0, ..., N-1].

[0275] b. In one example, for any pixel (x0, y0) and block vector (BVx, BVy) relative to the top left corner of the CTU, the bitstream constraint is that x0+BVx should be in the range [0, ..., M-1].

[0276] c. In one example, for any pixel (x0, y0) and block vector (BVx, BVy) relative to the top left corner of the CTU, the bitstream constraint is that y0+BVy should be in the range [0, ..., N-1] and x0+BVx should be in the range [0, ..., M-1].

[0277] 17. When the signaled or derived block vector for a block points somewhere outside the buffer, padding may be applied based on the buffer.

[0278] a. In one example, the value of any sample point outside the buffer is defined with a predefined value.

[0279] i. In one example, the value may be 1<<(Bitdepth-1), e.g., 128 for an 8-bit signal and 512 for a 10-bit signal.

[0280] ii. In one example, when ILR is used, the value can be forwardLUT

[0281] (m). For example, m = 1 << (Bitdepth - 1).

[0282] iii. Alternatively, the indication of the predefined value may be signaled or indicated at the SPS / PPS / sequence header / picture header / slice header / slice group / slice / CTU / CU level.

[0283] b. In one example, any sample outside the buffer is defined as the value of the nearest sample in the buffer.

[0284] 18. The method of handling out-of-buffer references can be different in the horizontal and vertical directions, or can be different depending on the location of the current block (e.g., whether it is closer to the picture boundary).

[0285] a. In one example, when y0+BVy is outside [0, N-1], the sample value of (x0+BVx, y0+BVy) is designated as a predefined value.

[0286] b. In one example, when x0+BVx is outside [0, M-1], the sample value of (x0+BVx, y0+BVy) is designated as a predefined value.

[0287] c. Alternatively, the sample value of (x0+BVx, y0+BVy) is assigned to the sample value of ((x0+BVx)mod M, y0+BVy). If ((x0+BVx)mod M, y0+BVy) is still outside the buffer, it can call other methods to further derive the value.

[0288] d. Alternatively, the sample value of (x0+BVx, y0+BVy) is assigned to the sample value of (x0+BVx, (y0+BVy) mod N). If (x0+BVx, (y0+BVy) mod N) is still outside the buffer, it can call other methods to further derive the value.

[0289] Block vector representation

[0290] 19. Each component or one of the components of the block vector (BVx, BVy) may be normalized to a specific range.

[0291] a. In one example, BVx can be replaced by (BVx mod M).

[0292] b. Alternatively, BVx can be replaced by ((BVx+X) mod M)-X, where X is a predefined value.

[0293] i. In one example, X is 64.

[0294] ii. In one example, X is M / 2;

[0295] iii. In one example, X is the horizontal coordinate of the block relative to the current CTU.

[0296] c. In one example, BVy can be replaced by (BVy mod N).

[0297] d. Alternatively, BVy can be replaced by ((BVy+Y) mod N)-Y, where Y is a predefined value.

[0298] i. In one example, Y is 64.

[0299] ii. In one example, Y is N / 2;

[0300] iii. In one example, Y is the vertical coordinate of the block relative to the current CTU.

[0301] 20. BVx and BVy can have different normalized ranges.

[0302] 21. Block vector differences (BVDx, BVDy) can be normalized to a specific range.

[0303] a. In one example, BVDx can be replaced by (BVDx mod M), where the function mod returns the remainder.

[0304] b. Alternatively, BVDx can be replaced by ((BVDx+X) mod M)-X, where X is a predefined value.

[0305] i. In one example, X is 64.

[0306] ii. In one example, X is M / 2;

[0307] c. In one example, BVy can be replaced by (BVDy mod N).

[0308] d. Alternatively, BVy can be replaced by ((BVDy+Y) mod N)-Y, where Y is a predefined value.

[0309] i. In one example, Y is 64.

[0310] ii. In one example, Y is N / 2;

[0311] 22. BVDx and BVDy can have different normalized ranges.

[0312] Block vector validity check

[0313] Denote the width and height of the IBC buffer as W buf and H buf For a W×H block (which can be a luminance block, chrominance block, CU, TU, 4×4, 2×2 or other sub-block) starting from (X, Y) relative to the upper left corner of the picture, the following can be applied to determine whether the block vector (BVx, BVy) is valid. Assume W pic and H pic is the width and height of the image; and W ctu and H ctu is the width and height of the CTU. Function floor(x) returns the largest integer not greater than x. Function isRec(x, y) returns whether the sample (x, y) has been reconstructed.

[0314] 23. The block vector (BVx, BVy) can be set to valid even if any reference location is outside the picture boundary.

[0315] a. In one example, the block vector may be set to valid even if X+BVx<0.

[0316] b. In one example, even if X+W+BVx>W pic , block vectors can also be set to valid.

[0317] c. In one example, the block vector may be set to valid even if Y+BVy<0.

[0318] d. In one example, even if Y+H+BVy>H pic , block vectors can also be set to valid.

[0319] 24. The block vector (BVx, BVy) may be set to valid even if any reference location is outside the current CTU row.

[0320] a. In one example, even if Y+BVy <floor(Y / H ctu )*H ctu , block vectors can also be set to valid.

[0321] b. In one example, even if Y+H+BVy>=floor(Y / H ctu )*H ctu +H ctu , block vectors can also be set to valid.

[0322] 25. The block vector (BVx, BVy) can be set to valid even if any reference position is outside the current CTU and the (n-1) CTUs to the left, where n is the number of CTUs (including or excluding the current CTU) that can be used as the reference area for IBC.

[0323] a. In one example, even if X+BVx <floor(X / W ctu )*W ctu -(n-1)*W ctu , block vectors can also be set to valid.

[0324] b. In one example, even if X+W+BVx>floor(X / W ctu )*W ctu +W ctu , block vectors can also be set to valid.

[0325] 26. A block vector (BVx, BVy) may be set to valid even if a particular sample has not been reconstructed.

[0326] a. In one example, the block vector may be set to valid even if isRec(X+BVx, Y+BVy) is false.

[0327] b. In one example, the block vector may be set to valid even if isRec(X+BVx+W-1, Y+BVy) is false.

[0328] c. In one example, the block vector may be set to valid even if isRec(X+BVx, Y+BVy+H-1) is false.

[0329] d. In one example, the block vector may be set to valid even if isRec(X+BVx+W-1, Y+BVy+H-1) is false.

[0330] 27. When the block is not the first CTU in a CTU row, the block vector (BVx, BVy) may always be set to valid.

[0331] a. Alternatively, the block vector may be always set to valid.

[0332] 28. A block vector (BVx, BVy) can always be set to valid when all of the following 3 conditions are met:

[0333] ·X+BVx>=0

[0334] Y+BVy>=floor(Y / H ctu )

[0335] ·isRec(X+BVx+W-1, Y+BVy+H-1)==true

[0336] a. Alternatively, the block vector may always be set to valid when all three conditions are met for the block of the first CTU in a CTU row.

[0337] 29. When the block vector (BVx, BVy) is valid, the sample replication of the block can be based on the block vector.

[0338] a. In one example, the prediction of a sample point (X, Y) can be calculated based on ((X+BVx)%W buf , (Y+BVy)%H buf ).

[0339] Buffer Update

[0340] 30. The buffer can be reset when encoding or decoding a new picture or slice.

[0341] a. The term "reset" may refer to a buffer being initialized.

[0342] b. The term "reset" may refer to all samples / pixels in the buffer being set to a given value (eg, 0 or -1).

[0343] 31. When the encoding and decoding of the VPDU is completed, the buffer can be updated with the reconstructed value of the VPDU.

[0344] 32. When the encoding and decoding of the CTU is completed, the buffer can be updated with the reconstructed value of the CTU.

[0345] a. In one example, when the buffer is not full, the buffer can be updated sequentially CTU by CTU.

[0346] b. In one example, when the buffer is full, the buffer area corresponding to the oldest CTU will be updated.

[0347] c. In one example, when M=mW and N=H (W and H are CTU sizes; M and N are buffer sizes) and the previously updated region starts from (kW, 0), the next starting position to be updated will be ((k+1)W mod M, 0).

[0348] 33. The buffer can be reset at the beginning of each CTU row.

[0349] a. Alternatively, the buffer can be reset at the start of decoding each CTU.

[0350] b. Alternatively, the buffer can be reset when starting to decode a slice.

[0351] c. Alternatively, the buffer can be reset when starting to decode a slice group / picture.

[0352] 34. When the encoding and decoding of the block starting from (x, y) is completed, the corresponding area of ​​the buffer starting from (x, y) will be updated with the reconstruction according to the block.

[0353] a. In one example, (x, y) is the position relative to the top left corner of the CTU.

[0354] 35. When the encoding and decoding of a block relative to a picture is completed, the corresponding area of ​​the buffer will be updated with the reconstruction according to the block.

[0355] a. In one example, the value at position (x mod M, y mod N) in the buffer may be updated with the reconstructed pixel value at position (x, y) relative to the top left corner of the picture.

[0356] b. In one example, the value at position (x mod M, y mod N) in the buffer may be updated with the reconstructed pixel value at position (x, y) relative to the upper left corner of the current slice.

[0357] c. In one example, the value at position (x mod M, y mod N) in the buffer may be updated with the reconstructed pixel value at position (x, y) relative to the upper left corner of the current CTU row.

[0358] d. In one example, the value in the buffer may be updated with the reconstructed pixel value after bit depth alignment.

[0359] 36. When encoding and decoding of a block starting from (x, y) is completed, the corresponding area of ​​the buffer starting from (xb, yb) will be updated with the reconstruction based on the block, where (xb, yb) and (x, y) are two different coordinates.

[0360] a. In one example, (x, y) is a position relative to the top left corner of the CTU, and (xb, yb) is (x + update_x, y + update_y), where update_x and update_y point to updateable positions in the buffer.

[0361] 37. For the above example, the reconstructed value of the block may indicate the reconstructed value before applying a filter (eg, a deblocking filter).

[0362] a. Alternatively, the reconstructed value of the block may indicate the reconstructed value after applying a filter (eg, a deblocking filter).

[0363] 38. When the buffer is updated based on the reconstructed samples, the reconstructed samples may first be modified before being stored, such as the sample bit depth may be changed.

[0364] a. In one example, the buffer is updated with the reconstructed sample value after bit-depth alignment with the bit-depth of the buffer.

[0365] b. In one example, the buffer value is updated according to the value {p + [1 << (b - 1)]} >> b, where p is the reconstructed sample value and b is a predefined bit-shift value.

[0366] c. In one example, the buffer value is updated according to the value clip({p + [1 << (b - 1)]} >> b, 0, (1 << bitdepth) - 1), where p is the reconstructed sample value, b is a predefined bit-shift value, and bitdepth is the buffer bit-depth.

[0367] d. In one example, the buffer value is updated according to the value {p + [1 << (b - 1) - 1]} >> b, where p is the reconstructed sample value and b is a predefined bit-shift value.

[0368] e. In one example, the buffer value is updated according to the value clip({p + [1 << (b - 1) - 1]} >> b, 0, (1 << bitdepth) - 1), where p is the reconstructed sample value, b is a predefined bit-shift value, and bitdepth is the buffer bit-depth.

[0369] f. In one example, the buffer value is updated according to the value p >> b.

[0370] g. In one example, the buffer value is updated according to the value clip(p >> b, 0, (1 << bitdepth) - 1), where bitdepth is the buffer bit-depth.

[0371] h. In the above examples, b can be the reconstructed bit-depth minus the input sample bit-depth. 39. When using buffer samples to form a prediction, preprocessing can be applied.

[0372] a. In one example, the predicted value is p << b, where p is the sample value in the buffer and b is a predefined value.

[0373] b. In one example, the predicted value is clip(p << b, 0, 1 << bitdepth), where bitdepth is the bit-depth of the reconstructed sample.

[0374] c. In one example, the predicted value is (p << b) + (1 << (bitdepth - 1)), where p is the sample value in the buffer and b is a predefined value, and bitdepth is the bit-depth of the reconstructed sample.

[0375] d. In the above example, b can be the reconstruction bit depth minus the input sample bit depth.

[0376] 40. Buffers can be updated in a given order.

[0377] a. In one example, the buffers may be updated sequentially.

[0378] b. In one example, the buffer may be updated according to the order of the reconstructed blocks.

[0379] 41. When the buffer is full, the samples in the buffer can be replaced with the latest reconstructed samples.

[0380] a. In one example, samples may be updated in a first-in, first-out manner.

[0381] b. In a sample, the oldest sample point will be replaced.

[0382] c. In one example, samples can be assigned priorities and replaced based on the priorities.

[0383] d. In one example, a sample point can be marked as "long term" so that other samples will be replaced first.

[0384] e. In one example, a flag may be transmitted with the block to indicate high priority.

[0385] f. In one example, a number may be transmitted along with the block to indicate priority.

[0386] g. In one example, samples from a reconstructed block with certain characteristics will be assigned higher priority so that other samples will be replaced first.

[0387] i. In one example, when the percentage of samples encoded and decoded in IBC mode is greater than a threshold, all samples of the block may be assigned a high priority.

[0388] ii. In one example, when the percentage of samples encoded and decoded in Palette mode is greater than a threshold, all samples of the block may be assigned a high priority.

[0389] iii. In one example, when the percentage of samples encoded in IBC or palette mode is greater than a threshold, all samples of the block may be assigned a high priority.

[0390] iv. In one example, when the percentage of samples coded in transform skip mode is greater than a threshold, all samples of the block may be assigned a high priority.

[0391] v. The threshold value can be different according to the block size, color component, and CTU size.

[0392] vi. The threshold can be signaled in SPS / PPS / sequence header / slice header / slice group / slice level / region.

[0393] h. In one example, a buffer full may mean that the number of available samples in the buffer is equal to or greater than a given threshold.

[0394] i. In one example, the buffer may be determined to be full when the number of available samples in the buffer is equal to or greater than 64×64×3 luma samples.

[0395] Optional buffer combinations

[0396] 42. Instead of always using the three previously coded 64×64 blocks as the reference area, it is proposed to adaptively change it based on the position of the current block (or VPDU).

[0397] a. In one example, when encoding / decoding a 64×64 block, the previous three 64×64 blocks can be used as references. Figure 2 Compared to the previous 64×64 blocks, a wider variety of combinations can be applied. Figure 2 Examples of different combinations of the previous 64x64 blocks are shown.

[0398] 43. Instead of using a z-scan order, a vertical scan order can be utilized.

[0399] a. In one example, when a block is divided into 4 VPDUs with indices 0...3 in z-scan order, the encoding / decoding order is 0, 2, 1, 3.

[0400] b. In one example, when encoding / decoding a 64×64 block, the previous three 64×64 blocks can be used as references. Figure 2 Compared to 64×64 blocks, a wider variety of encoding / decoding orders can be applied. Figure 4 Examples of different encoding / decoding orders for 64x64 blocks are shown.

[0401] c. Alternatively, the above method may be applied only to screen content encoding and decoding.

[0402] d. Alternatively, the above method may be applied only when CPR is enabled for one slice / slice group / picture.

[0403] e. Alternatively, the above method may be applied only when CPR is enabled for one CTU or one CTU row.

[0404] Virtual IBC buffer

[0405] Below, the width and height of the VPDU are represented in luma samples as W and VPDU (e.g., 64) and H VPDU (e.g., 64). Alternatively, W VPDU and / or H VPDU The width and / or height of other video units (eg, CTU) may be represented.

[0406] 44. A virtual buffer may be maintained to keep track of IBC reference region status.

[0407] a. In one example, the size of the virtual buffer is mWVPDU×nHVPDU.

[0408] i. In one example, m is equal to 3 and n is equal to 2.

[0409] ii. In one example, m and / or n may depend on the picture resolution and CTU size.

[0410] iii. In one example, m and / or n may be signaled or predefined.

[0411] b. In one example, the methods described in the above bullets and sub-bullets can be applied to a virtual buffer.

[0412] c. In one example, the sample point (x, y) relative to the top left corner of the picture / slice / slice / block can be mapped to (x% (mW VPDU ), y%(nH VPDU )).

[0413] 45. An array may be used to track the availability of each sample associated with a virtual buffer.

[0414] a. In one example, a flag may be associated with a sample in a virtual buffer to specify whether the sample in the buffer may be used as an IBC reference.

[0415] b. In one example, each 4x4 block containing luma and chroma samples may share a flag to indicate whether any samples associated with the block may be used as an IBC reference.

[0416] 46. ​​After decoding of a VPDU or video unit is completed, specific samples associated with the virtual buffer may be marked as unavailable for IBC reference.

[0417] a. In one example, which samples can be marked as unavailable depends on the position of the most recently decoded VPDU.

[0418] b. When a sample point is marked as unavailable, prediction based on this sample point is not allowed.

[0419] i. Alternatively, other approaches (eg, using default values) may be further applied to derive predicted values ​​to replace unavailable samples.

[0420] 47. The location of the most recently decoded VPDU may be recorded to help identify which samples associated with the virtual buffer may be marked as unavailable.

[0421] a. In one example, when starting to decode a VPDU, certain samples associated with the virtual buffer may be marked as unavailable based on the location of the most recently decoded VPDU.

[0422] i. In one example, (xPrevVPDU, yPrevVPDU) is expressed as the top left corner position of the picture / slice / slice / block / other video processing unit relative to the top left corner of the most recently decoded VPDU. If yPrevVPDU%(nHVPDU) is equal to 0, then the specific position (x, y) can be marked as unavailable.

[0423] 1. In one example, x can be in a range, such as [xPrevVPDU-2WVPDU+2mWVPDU)%mWVPDU, ((xPrevVPDU-2WVPDU+2mWVPDU)%mWVPDU)-1+WVPDU];

[0424] 2. In one example, y can be in a range such as [yPrevVPDU%(nHVPDU),(yPrevVPDU%(nHVPDU))-1+HVPDU];

[0425] 3. In one example, x may be in a range such as [xPrevVPDU-2WVPDU+2mWVPDU)%mWVPDU, ((xPrevVPDU-2WVPDU+2mWVPDU)%mWVPDU)-1+WVPDU], and y may be in a range such as [yPrevVPDU%(nHVPDU), (yPrevVPDU%(nHVPDU))-1+HVPDU].

[0426] ii. In one example, (xPrevVPDU, yPrevVPDU) is expressed as the top left corner position of the picture / slice / slice / block / other video processing unit relative to the top left corner of the most recently decoded VPDU. If yPrevVPDU%(nHVPDU) is not equal to 0, the specific position (x, y) can be marked as unavailable.

[0427] 1. In one example, x can be in a range, such as [xPrevVPDU-WVPDU+2mWVPDU)%mWVPDU, ((xPrevVPDU-WVPDU+2mWVPDU)%mWVPDU)-1+WVPDU];

[0428] 2. In one example, y can be in a range such as [yPrevVPDU%(nHVPDU),(yPrevVPDU%(nHVPDU))-1+HVPDU];

[0429] 3. In one example, x may be in a range such as [xPrevVPDU-WVPDU+2mWVPDU)%mWVPDU, ((xPrevVPDU-WVPDU+2mWVPDU)%mWVPDU)-1+WVPDU], and y may be in a range such as [yPrevVPDU%(nHVPDU), (yPrevVPDU%(nHVPDU))-1+HVPDU].

[0430] 48. When a CU contains multiple VPDUs, instead of applying the IBC reference availability marking procedure per VPDU, the IBC reference availability marking procedure can be per CU.

[0431] a. In one example, when starting to decode a CU containing multiple VPDUs, the IBC reference availability marking process may be applied to each VPDU before the VPDUs within the CU are decoded.

[0432] b. In this case, 128×64 and 64×128 IBC blocks may not be allowed.

[0433] i. In one example, pred_mode_ibc_flag for 128x64 and 64x128 CUs may not be transmitted and may be inferred to be equal to 0.

[0434] 49. For a reference block or sub-block, it may not be necessary to check the reference availability status in the upper right corner to determine whether the block vector associated with the reference block is valid.

[0435] a. In one example, only the top left, bottom left, and bottom right corners of the block / sub-block will be checked to determine whether the block vector is valid.

[0436] 50. The IBC buffer size may depend on the VPDU size (where width / height is represented by vSize) and / or the CTB / CTU size (where width / height is represented by ctbSize).

[0437] a. In one example, the height of the buffer can be equal to ctbSize.

[0438] b. In one example, the width of the buffer may depend on min(ctbSize, 64).

[0439] i. In one example, the width of the buffer can be (128*128 / vSize, min(ctbSize, 64)).

[0440] 51. The IBC buffer may contain values ​​outside the pixel range, which indicates that the location may not be used for IBC reference, for example, not for predicting other samples.

[0441] a. The sample value may be set to a value indicating that the sample is not available.

[0442] b. In one example, the value may be -1.

[0443] c. In one example, this value can be any value outside [0, 1<<(internal_bit_depth)-1], where internal_bit_depth is a positive integer value. For example, internal_bit_depth is the internal bit depth used to encode / decode samples of a color component.

[0444] d. In one example, this value can be any value outside [0, 1<<(input_bit_depth)-1], where input_bit_depth is a positive integer value. For example, input_bit_depth is the input bit depth used to encode / decode samples of a color component.

[0445] 52. The availability flag for samples in the IBC buffer may depend on the position of the current block, the size of the current block, the CTU / CTB size, and the VPDU size. In one example, assume that (xCb, yCb) represents the position of the block relative to the top left corner of the picture; ctbSize is the size of the CTU / CTB (i.e., width and / or height); vSize = min(ctbSize, 64); wIbcBuf and hIbcBuf are the IBC buffer width and height.

[0446] a. In one example, if (xCb % vSize) equals 0 and (yCb % vSize) equals 0, then a particular set of locations in the IBC buffer may be marked as unavailable.

[0447] b. In one example, when the current block size is smaller than the VPDU size, ie, min(ctbSize, 64), the area marked as unavailable may be based on the VPDU size.

[0448] c. In one example, when the current block size is greater than the VPDU size, i.e., min(ctbSize, 64), the area marked as unavailable can be based on the CU size.

[0449] 53. When starting to decode a video unit (e.g., VPDU(xV, yV)) relative to the top - left corner position of a picture, the corresponding position in the IBC buffer can be set to a value outside the pixel range.

[0450] a. In one example, the buffer samples at the position (x % wIbcBuf, y % hIbcBuf) in the buffer will be set to the value - 1, where x = xV,..., xV + ctbSize - 1 and y = yV,..., yV + ctbSize - 1. Here, wIbcBuf and hIbcBuf are the width and height of the IBC buffer, and ctbSize is the width of the CTU / CTB.

[0451] i. In one example, hIbcBuf can be equal to ctbSize.

[0452] 54. Bit - stream consistency constraints can be based on the values of the samples in the IBC buffer.

[0453] a. In one example, if the reference block associated with the block vector in the IBC buffer contains values outside the pixel range, the bit - stream may be illegal.

[0454] 55. Bit - stream consistency constraints can be set according to the availability indication in the IBC buffer.

[0455] a. In one example, if any reference sample mapped in the IBC buffer is marked as unavailable for encoding / decoding a block, the bit - stream may be illegal.

[0456] b. In one example, when using a singletree, if any luma reference sample mapped in the IBC buffer for encoding / decoding a block is marked as unavailable, the bit - stream may be illegal.

[0457] c. The consistent bit - stream can satisfy that for an IBC - encoded / decoded block, the associated block vector can point to a reference block mapped in the IBC buffer, and each luma reference sample in the IBC buffer used for encoding / decoding the block should be marked as available (e.g., the value of the sample is within the range [K0, K1], where for example, K0 is set to 0, and K1 is set to (1 << BitDepth - 1), where BitDepth is the internal bit depth or the input bit depth).

[0458] 56. Bitstream consistency constraints can depend on the split tree type and the coding treeType of the current CU.

[0459] a. In one example, if dual trees are allowed at a high level (e.g., stripe / picture / block / slice) and the current video block (e.g., CU / PU / CB / PB) is coded with a single tree, the bitstream constraints may need to check whether the positions of all components mapped in the IBC buffer are marked as unavailable.

[0460] b. In one example, if dual trees are allowed at a high level (e.g., stripe / picture / block / slice) and the current luma video block (e.g., CU / PU / CB / PB) is coded with a dual tree, the bitstream constraints can ignore whether the positions of the chroma components mapped in the IBC buffer are marked as unavailable.

[0461] i. Alternatively, in this case, the bitstream constraints can still check whether the positions of all components mapped in the IBC buffer are marked as unavailable.

[0462] c. In one example, if a single tree is used, the bitstream constraints can ignore whether the positions of the chroma components mapped in the IBC buffer are marked as unavailable.

[0463] Improvements to the current VTM design

[0464] 57. Predictions for IBC may have lower accuracy than reconstruction.

[0465] a. In one example, the predicted value is based on the value clip{{p + [1 << (b - 1)]} >> b, 0, (1 << bitdepth) - 1} << b, where p is the reconstructed sample value, b is a predefined bit shift value, and bitdepth is the bit depth of the predicted sample.

[0466] b. In one example, the predicted value is based on the value clip{{p + [1 << (b - 1) - 1]} >> b, 0, (1 << bitdepth) - 1} << b, where p is the reconstructed sample value and b is a predefined bit shift value.

[0467] c. In one example, the predicted value is based on the value ((p >> b) + (1 << (bitdepth - 1))) << b, where bitdepth is the bit depth of the predicted sample.

[0468] d. In one example, the predicted value is based on the value (clip((p >> b), 0, (1 << (bitdepth - b))) + (1 << (bitdepth - 1))) << b, where bitdepth is the bit depth of the predicted sample.

[0469] e. In one example, depending on whether ILR is applied, the predicted value is clipped in a different way.

[0470] f. In the above example, b can be the reconstruction bit depth minus the input sample bit depth.

[0471] g. In one example, the bit depth or the bit depth difference compared to the output bit depth / input bit depth / internal bit depth can be signaled in the SPS / PPS / sequence header / picture header / slice header / tile group header / tile header or other types of video data units.

[0472] 58. Part of the prediction of IBC may have lower precision, and another part has the same precision as the reconstruction.

[0473] a. In one example, the allowed reference region can contain samples with different precisions (e.g., bit depth).

[0474] b. In one example, references from other 64×64 blocks other than the current 64×64 block being decoded have low precision, and references from the current 64×64 block have the same precision as the reconstruction.

[0475] c. In one example, references from other CTUs other than the current CTU being decoded have low precision, and references from the current CTU have the same precision as the reconstruction.

[0476] d. In one example, references from a specific set of color components have low precision, and references from other color components have the same precision as the reconstruction.

[0477] 59. When the CTU size is M×M and the reference region size is nM×nM, the reference region is the nearest available n×n CTU in the CTU row.

[0478] a. In one example, when the reference region size is 128×128 and the CTU size is 64×64, the nearest 4 available CTUs in the CTU row can be used for IBC reference.

[0479] b. In one example, when the reference region size is 128×128 and the CTU size is 32×32, the nearest 16 available CTUs in the CTU row can be used for IBC reference.

[0480] 60. When the CTU size is M and the reference area size is nM, the reference area is the CTU row /

[0481] The nearest available n-1 CTUs in the slice.

[0482] a. In one example, when the reference region size is 128×128 or 256×64 and the CTU size is 64×64, the nearest available 3 CTUs in a CTU row can be used for IBC reference.

[0483] b. In one example, when the reference region size is 128×128 or 512×32 and the CTU size is 32×32, the nearest available 15 CTUs in a CTU row can be used for IBC reference.

[0484] 61. When the CTU size is M, the VPDU size is kM, and the reference region size is nM, the reference region is the nearest available nk CTUs in the CTU row / slice.

[0485] a. In an example, the CTU size is 64×64, the VPDU size is also 64×64, the reference area size is 128×128, and the three most recent CTUs in the CTU row can be used for the IBC reference value.

[0486] b. In one example, the CTU size is 32×32, the VPDU size is also 64×64, the reference area size is 128×128, and the nearest (16-4)=12 CTUs in the CTU row can be used for IBC reference.

[0487] 62. For a w×h block with the upper left corner at (x, y) using IBC, there is a constraint to keep the reference block away from a specific area for memory reuse, where w and h are the width and height of the current block.

[0488] a. In one example, when the CTU size is 128×128 and (x, y)=(m×64, n×64), the reference block cannot overlap with the 64×64 region starting from ((m-2)×64, n×64).

[0489] b. In one example, when the CTU size is 128×128, the reference block cannot overlap with the w×h block whose upper left corner is (x-128, y).

[0490] c. In one example, when the CTU size is 128×128, (x+BVx, y+BVy) cannot be within a w*h block whose upper left corner is (x-128, y), where BVx and BVy represent the block vector of the current block.

[0491] d. In one example, when the CTU size is M×M and the IBC buffer size is k×M×M, the reference block cannot overlap with the w×h block whose upper left corner is (xk×M, y), where BVx and BVy represent the block vectors of the current block.

[0492] e. In one example, when the CTU size is M×M and the IBC buffer size is k×M×M, (x+BVx, y+BVy) cannot be within a w×h block whose upper left corner is (xk×M, y), where BVx and BVy represent the block vector of the current block.

[0493] 63. When the CTU size is not M×M and the reference region size is nM×nM, the reference region is the nearest available n×n-1 CTU in the CTU row.

[0494] a. In one example, when the reference region size is 128×128 and the CTU size is 64×64, the nearest available 3 CTUs in a CTU row can be used for IBC reference.

[0495] b. In one example, when the reference region size is 128×128 and the CTU size is 32×32, the nearest available 15 CTUs in a CTU row can be used for IBC reference.

[0496] 64. For the CU within the 64×64 block starting from (2m*64, 2n*64) (i.e., the upper left 64×64 block in the 128×128 CTU), its IBC prediction can be based on the 64×64 block starting from ((2m-2)*64, 2n*64), the 64×64 block starting from ((2m-1)*64, 2n*64), the 64×64 block starting from ((2m-1)*64, (2n+1)*64), and the reconstructed samples in the current 64×64 block.

[0497] 65. For the CU within the 64×64 block starting from ((2m+1)*64, (2n+1)*64) (i.e., the lower right 64×64 block in the 128×128 CTU), its IBC prediction can be based on the current 128×128 CTU.

[0498] 66. For the CU within the 64×64 block starting from ((2m+1)*64, 2n*64) (i.e., the upper right 64×64 block in the 128×128 CTU), its IBC prediction can be based on the 64×64 block starting from ((2m-1)*64, 2n*64), the 64×64 block starting from ((2m-1)*64, (2n+1)*64), the 64×64 block starting from (2m*64, 2n*64), and the reconstructed samples in the current 64×64 block.

[0499] a. Alternatively, if the 64×64 block starting from (2m*64, (2n+1)*64) has been reconstructed, IBC prediction can be based on the 64×64 block starting from ((2m-1)*64, 2n*64), the 64×64 block starting from (2m*64, 2n*64), the 64×64 block starting from (2m*64, (2n+1)*64), and the reconstructed samples in the current 64×64 block.

[0500] 67. For the CU within the 64×64 block starting from (2m*64, (2n+1)*64) (i.e., the lower left 64×64 block in the 128×128 CTU), its IBC prediction can be based on the 64×64 block starting from ((2m-1)*64, (2n+1)*64), the 64×64 block starting from (2m*64, 2n*64), the 64×64 block starting from ((2m+1)*64, 2n*64) and the reconstructed samples in the current 64×64 block.

[0501] a. Alternatively, if the 64×64 block starting from ((2m+1)*64, 2n*64) has not been reconstructed, IBC prediction can be based on the 64×64 block starting from ((2m-1)*64, 2n*64), the 64×64 block starting from ((2m-1)*64, (2n+1)*64), the 64×64 block starting from (2m*64, 2n*64), and the reconstructed samples in the current 64×64 block.

[0502] 68. Propose to adjust the reference area based on which 64×64 blocks the current CU belongs to.

[0503] a. In one example, for a CU starting from (x, y), when (y>>6)&1==0, two or up to two previous 64×64 blocks starting from ((x>>6<<6)-128, y>>6<<6) and ((x>>6<<6)-64, y>>6<<6) can be referenced by IBC mode.

[0504] b. In one example, for a CU starting from (x, y), when (y>>6)&1==1, a previous 64×64 block starting from ((x>>6<<6)-64, y>>6<<6) can be referenced by the IBC mode.

[0505] 69. For a block starting at (x, y) and having a block vector (BVx, BVy), if isRec(((x+BVx)>>6<<6)+128-(((y+BVy)>>6)&1)*64+(x%64),((y+BVy)>>6<<6)+(y%64)) is true, the block vector is invalid.

[0506] a. In one example, the block is a luminance block.

[0507] b. In one example, the block is a chroma block in 4:4:4 format.

[0508] c. In one example, the block contains both luma and chroma components.

[0509] 70. For a chroma block in 4:2:0 format starting at (x, y) and with a block vector (BVx, BVy), if isRec(((x+BVx)>>5<<5)+64-(((y+BVy)>>5)&1)*32+(x%32),((y+BVy)>>5<<5)+(y%32)) is true, the block vector is invalid.

[0510] 71. BV's determination of whether a block of component c is invalid may rely on the availability of samples of component x, rather than just checking luma samples.

[0511] a. For a block whose component c starts at (x, y) and has a block vector (BVx, BVy), if isRec(c, ((x+BVx)>>6<<6)+128-(((y+BVy)>>6)&1)*64+(x%64), ((y+BVy)>>6<<6)+(y%64)) is true, the block vector can be considered invalid.

[0512] i. In one example, the block is a luma block (eg, c is the luma component, or G component for RGB codecs).

[0513] ii. In one example, the block is a chroma block in 4:4:4 format (eg, c is the cb or cr component, or the B / R component for RGB codecs).

[0514] iii. In one example, the availability of samples of both the luma component and the chroma component may be checked, eg, the block contains both the luma component and the chroma component.

[0515] b. For a chroma block in 4:2:0 format starting from (x, y) of component c and having a block vector (BVx, BVy), if isRec(c, ((x+BVx)>>5<<5)+64-(((y+BVy)>>5)&1)*32+(x%32), ((y+BVy)>>5<<5)+(y%32)) is true, the block vector can be considered invalid.

[0516] c. For a chroma block or subblock starting at (x, y) of component c and having a block vector (BVx, BVy), the block vector may be considered invalid if isRec(c,x+BVx+Chroma_CTU_size,y) is true for the chroma component, where Chroma_CTU_size is the CTU size of the chroma component.

[0517] i. In one example, for 4:2:0 format, Chroma_CTU_size can be 64.

[0518] ii. In one example, the chroma sub-block may be a 2×2 block in a 4:2:0 format.

[0519] iii. In one example, the chroma sub-block may be a 4×4 block in 4:4:4 format.

[0520] iv. In one example, the chroma sub-block may correspond to the smallest CU size in the luma component.

[0521] 1. Alternatively, the chroma sub-block may correspond to the minimum CU size of the chroma component.

[0522] 72. For all the bullet points mentioned above, it is assumed that the reference buffer contains multiple M×M blocks (M=64). However, it can be extended to other cases, such as the reference buffer contains multiple N×M blocks (e.g., N=128, M=64).

[0523] 73. For all the bullet points mentioned above, a further restriction may apply, namely that the reference buffer should be within the same block / slice / slice group / strip as the current block.

[0524] a. In one example, if a portion of the reference buffer is outside the current block / slice / slice group / slice band, the use of IBC may be disabled and the signaling of IBC-related syntax elements may be skipped.

[0525] b. Alternatively, if part of the reference buffer is outside the current block / slice / slice group / strip, IBC may still be enabled for a block, however, the block vector associated with a block may only point to the remaining reference buffer.

[0526] 74. It is proposed to use the K1 most recently coded VPDUs (if available) in the first VPDU row of a CTU / CTB row and the K2 most recently coded VPDUs (if available) in the second VPDU row of a CTU / CTB row as the reference area for IBC, excluding the current VPDU.

[0527] a. In one example, K1 equals 2 and K2 equals 1.

[0528] b. In one example, when the CTU / CTB size is 128×128 and the VPDU size is 64×64, the above method can be applied.

[0529] c. In one example, the above method can be applied when the CTU / CTB size is 64×64 and the VPDU size is 64×64 and / or 32×32.

[0530] d. In one example, the above method can be applied when the CTU / CTB size is 32×32 and the VPDU size is 32×32 or smaller.

[0531] 75. The above methods can be applied at different stages.

[0532] a. In one example, a modular operation (eg, a mod b) of a block vector (BV) may be invoked during a BV availability check to determine whether the BV is valid.

[0533] b. In one example, a modulo operation (e.g., a mod b) of a block vector (BV) may be invoked to identify the location of a reference sample in an IBC virtual buffer or a reconstructed picture buffer (e.g., before a loop filtering process) (e.g., based on the location of the current sample and the modulo result of the BV).

[0534] 5. Examples

[0535] 5.1 Example #1

[0536] The implementation of the buffer zone for IBC is described as follows:

[0537] The buffer size is 128 × 128. The CTU size is also 128 × 128. For the codec of the first CTU in a CTU row, the buffer is initialized with 128 (for 8-bit video signaling). For the codec of the k-th CTU in a CTU row, the buffer is initialized with the reconstruction of the (k-1)-th CTU before loop filtering.

[0538] Figure 3 An example of encoding and decoding of a block starting from (x, y) is shown.

[0539] When encoding or decoding a block starting at (x, y) associated with the current CTU, a block vector (BVx, BVy) = (x-x0, y-y0) is transmitted to the decoder to indicate that the reference block is from (x0, y0) in the IBC buffer. Assume that the width and height of the block are w and h, respectively. When encoding or decoding the block is complete, the w×h region starting at (x, y) in the IBC buffer is updated with the block's pre-loop filtering reconstruction.

[0540] 5.2 Example #2

[0541] Figure 4 Examples of possible alternative ways of selecting a previously coded 64x64 block are shown.

[0542] 5.3 Example #3

[0543] Figure 5 Examples of possible alternative ways of changing the encoding / decoding order of 64x64 blocks are shown.

[0544] 5.4 Example #4

[0545] Figure 8 Another possible alternative way of selecting a previously coded 64x64 block when the decoding order of 64x64 blocks is from top to bottom, left to right is shown.

[0546] 5.5 Example #5

[0547] Figure 9 Another possible alternative way of selecting a previously coded 64x64 block is shown.

[0548] 5.6 Example #6

[0549] Figure 11 Another possible alternative way of selecting a previously coded 64x64 block when the decoding order of 64x64 blocks is from left to right and from top to bottom is shown.

[0550] 5.7 Example #7

[0551] Assuming a CTU size of W×W, an IBC buffer of size mM×W at the decoder and a bit depth of B, the implementation is as follows.

[0552] When decoding of a CTU row begins, the buffer is initialized with a value of (1<<(B-1)) and the starting point (xb, yb) to be updated is set to (0, 0).

[0553] When a CU of size w×h starting from (x, y) relative to the upper left corner of the CTU is decoded, the area starting from (xb+x, yb+y) and of size w×h will be updated with the reconstructed pixel values ​​of the CU after the bit depth is aligned with the B bit.

[0554] After the CTU is decoded, the starting point (xb, yb) to be updated will be set to ((xb+W) mod mW, 0).

[0555] When decoding an IBC CU with block vector (BVx, BVy), for any pixel (x, y) relative to the top left corner of the CTU, its prediction is fetched from the buffer at position ((x+BVx) mod mW, (y+BVy) mod W) after bit-depth alignment with the bit-depth of the prediction signal.

[0556] In one example, B is set to 7 or 8, while the output / input bit depth of the block may be equal to 10.

[0557] 5.8 Example #8

[0558] For a luma CU or a joint luma / chroma CU starting at (x, y) relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(((x+BVx)>>6<<6)+128-(((y+BVy)>>6)&1)*64+(x%64),((y+BVy)>>6<<6)+(y%64)) is true.

[0559] For a chroma CU starting at (x, y) relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(((x+BVx)>>5<<5)+64-(((y+BVy)>>5)&1)*32+(x%32),((y+BVy)>>5<<5)+(y%32)) is true.

[0560] 5.9 Example 9

[0561] For a chroma block or subblock starting at (x, y) in 4:2:0 format relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(c, (x+BVx+64, y+BVy) is true, where c is the chroma component.

[0562] For a chroma block or subblock starting at (x, y) in 4:4:4 format relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(c, (x+BVx+128, y+BVy) is true, where c is the chroma component.

[0563] 5.10 Example #10

[0564] For a luma CU or a joint luma / chroma CU starting at (x, y) relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(((x+BVx)>>6<<6)+128-(((y+BVy)>>6)&1)*64+(x%64),((y+BVy)>>6<<6)+(y%64)) is true.

[0565] For a chroma block or subblock starting at (x, y) in 4:2:0 format relative to the top left corner of the picture and a block vector (BVx, BVy), the block vector is invalid when isRec(c, ((x+BVx)>>5<<5)+64-(((y+BVy)>>5)&1)*32+(x%32), ((y+BVy)>>5<<5)+(y%32)) is true, where c is the chroma component.

[0566] 5.11 Example #11

[0567] This embodiment highlights an implementation of keeping the two most recently coded VPDUs in the first VPDU row and the one most recently coded VPDU in the second VPDU row of a CTU / CTB row, excluding the current VPDU.

[0568] When the VPDU encoding and decoding order is from top to bottom and from left to right, the reference area is as follows Figure 13 shown.

[0569] When the VPDU encoding and decoding order is from left to right and from top to bottom, and the current VPDU is not on the right side of the picture boundary, the reference area is as follows: Figure 14 shown.

[0570] When the VPDU encoding and decoding order is from left to right and from top to bottom, and the current VPDU is on the right side of the picture boundary, the reference area can be as follows: Figure 15 shown.

[0571] Given a luma block (x, y) of size w×h, whether the block vector (BVx, BVy) is valid can be determined by checking the following conditions:

[0572] isRec(((x+BVx+128)>>6<<6)–(refy&0x40)+(x%64),((y+BVy)>>6<<6)+(refy>>6==y>>6)?(y%64):0), where refy=(y&0x40)? (y+BVy):(y+BVy+w-1).

[0573] If the above function returns true, the block vector (BVx, BVy) is invalid, otherwise the block vector may be valid.

[0574] 5.12 Example #12

[0575] If the CTU size is 192×128, a virtual buffer of size 192×128 is maintained to track the reference samples for IBC.

[0576] The sample point (x, y) relative to the top left corner of the picture is associated with the position (x%192, y%128) relative to the top left corner of the buffer. The following steps show how to mark the availability of samples associated with the virtual buffer for IBC reference.

[0577] The position (xPrevVPDU, yPrevVPDU) relative to the top left corner of the picture is recorded to represent the top left corner sample of the most recently decoded VPDU.

[0578] 1) When starting to decode a VPDU line, all positions in the buffer are marked as unavailable. (xPrevVPDU, yPrevVPDU) are set to (0, 0).

[0579] 2) When starting to decode the first CU of a VPDU, the position (x, y) (where x = (xPrevVPDU - 2WVPDU + 2mWVPDU) % (mWVPDU), .., ((xPrevVPDU - 2WVPDU + 2mWVPDU) % (mWVPDU)) - 1 + WVPDU; and y = yPrevVPDU % (nHVPDU), .., (yPrevVPDU % (nHVPDU)) - 1 + HVPDU) may be marked as unavailable. Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), i.e., the upper left corner position of the CU relative to the picture.

[0580] 3) After decoding the CU, the position (x, y) (where x = xCU% (mWVPDU), ..., (xCU+CU_width-1)% (mWVPDU) and y = yCU% (nHVPDU), ..., (yCU+CU_height-1)% (nHVPDU)) is marked as available.

[0581] 4) For an IBC CU with a block vector (xBV, yBV), if any position (x, y) (where x = (xCU+xBV)% (mWVPDU), ..., (xCU+xBV+CU_width-1)% (mWVPDU) and y = (yCU+yBV)% (nHVPDU), ..., (yCU+yBV+CU_height-1)% (nHVPDU)) is marked as unavailable, the block vector is considered invalid.

[0582] Figure 16 The buffer status is shown as well as the VPDU decoding status in the picture.

[0583] 5.13 Example #13

[0584] If the CTU size is 128×128 or the CTU size is larger than the VPDU size (e.g., 64×64 in the current design) or the CTU size is larger than the VPDU size (e.g., 64×64 in the current design), a virtual buffer of size 192×128 is maintained to track the reference samples for IBC. Below, when a<0, (a%b) is defined as floor(a / b)*b, where Returns the largest integer not greater than c.

[0585] The sample point (x, y) relative to the top left corner of the picture is associated with the position (x%192, y%128) relative to the top left corner of the buffer. The following steps show how to mark the availability of samples associated with the virtual buffer for IBC reference.

[0586] The position (xPrevVPDU, yPrevVPDU) relative to the top left corner of the picture is recorded to represent the top left corner sample of the most recently decoded VPDU.

[0587] 1) When starting to decode a VPDU line, all positions in the buffer are marked as unavailable. (xPrevVPDU, yPrevVPDU) are set to (0, 0).

[0588] 2) When starting to decode the first CU of VPDU,

[0589] a. If yPrevVPDU%64 is equal to 0, the position (x, y) (where x = (xPrevVPDU-128)%192,...,((xPrevVPDU-128)%192)+63; and y = yPrevVPDU%128,...,(yPrevVPDU%128)+63) is marked as unavailable. Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), which is the top left corner position of the CU relative to the picture.

[0590] b. Otherwise, the position (x, y) (where x = (xPrevVPDU–64)%192, .., ((xPrevVPDU–64)%192)+63; and y = yPrevVPDU%128, .., (yPrevVPDU%128)+63) is marked as unavailable. Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), the top left corner position of the CU relative to the picture.

[0591] 3) After decoding the CU, the position (x, y) (where x = xCU%192, ..., (xCU+CU_width-1)%192 and y = yCU%128, ..., (yCU+CU_height-1)%128) is marked as available.

[0592] 4) For an IBC CU with a block vector (xBV, yBV), if any position (x, y) (where x = (xCU + xBV) % 192, ..., (xCU + xBV + CU_width - 1) % 192 and y = (yCU + yBV) % 128, ..., (yCU + yBV + CU_height - 1) % 128) is marked as unavailable, the block vector is considered invalid.

[0593] If the CTU size is S×S and S is not equal to 128, then Wbuf is assumed to be equal to 128*128 / S. A virtual buffer of size Wbuf×S is maintained to track the reference samples used for IBC. In this case, the VPDU size is equal to the CTU size.

[0594] The position (xPrevVPDU, yPrevVPDU) relative to the top left corner of the picture is recorded to represent the top left corner sample of the most recently decoded VPDU.

[0595] 1) When starting to decode a VPDU line, all positions in the buffer are marked as unavailable. (xPrevVPDU, yPrevVPDU) are set to (0, 0).

[0596] 2) When starting to decode the first CU of a VPDU, the position (x, y) (where x = (xPrevVPDU – W buf *S)%S,..,((xPrevVPDU–W buf *S)%S)+S-1;And y=yPrevVPDU%S,..,(yPrevVPDU%S)+S-1) is marked as unavailable.Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), that is, the upper left corner position of the CU relative to the picture.

[0597] 3) After decoding the CU, the position (x, y) (where x = xCU% (W buf ),...,(xCU+CU_width-1)%(W buf ) and y=yCU%S,…,(yCU+CU_height-1)%S) is marked as available.

[0598] 4) For an IBC CU with a block vector (xBV, yBV), if any position (x, y) (where x = (xCU + xBV) % (Wbuf), ..., (xCU + xBV + CU_width - 1) % (Wbuf) and y = (yCU + yBV) % S, ..., (yCU + yBV + CU_height - 1) % S) is marked as unavailable, the block vector is considered invalid.

[0599] 5.14 Example #14

[0600] If the CTU size is 128×128 or the CTU size is larger than the VPDU size (e.g., 64×64 in the current design) or the CTU size is larger than the VPDU size (e.g., 64×64 in the current design), a virtual buffer of size 256×128 is maintained to track the reference samples for IBC. Below, when a<0, (a%b) is defined as floor(a / b)*b, where Returns the largest integer not greater than c.

[0601] The sample point (x, y) relative to the top left corner of the picture is associated with the position (x%256, y%128) relative to the top left corner of the buffer. The following steps show how to mark the availability of samples associated with the virtual buffer for IBC reference.

[0602] The position (xPrevVPDU, yPrevVPDU) relative to the top left corner of the picture is recorded to represent the top left corner sample of the most recently decoded VPDU.

[0603] 1) When starting to decode a VPDU line, all positions in the buffer are marked as unavailable. (xPrevVPDU, yPrevVPDU) are set to (0, 0).

[0604] 2) When starting to decode the first CU of VPDU,

[0605] a. If yPrevVPDU%64 is equal to 0, the position (x, y) (where x = (xPrevVPDU-128)%256,...,((xPrevVPDU-128)%256)+63; and y = yPrevVPDU%128,...,(yPrevVPDU%128)+63) is marked as unavailable. Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), which is the top left corner position of the CU relative to the picture.

[0606] b. Otherwise, the position (x, y) (where x = (xPrevVPDU - 64) % 256, .., ((xPrevVPDU - 64) % 256) + 63; and y = yPrevVPDU % 128, .., (yPrevVPDU % 128) + 63) is marked as unavailable. Then (xPrevVPDU, yPrevVPDU) is set to (xCU, yCU), the top left corner position of the CU relative to the picture.

[0607] 3) After decoding the CU, the position (x, y) (where x = xCU%256, ..., (xCU+CU_width-1)%256 and y = yCU%128, ..., (yCU+CU_height-1)%128) is marked as available.

[0608] 4) For an IBC CU with a block vector (xBV, yBV), if any position (x, y) (where x = (xCU + xBV) % 256, ..., (xCU + xBV + CU_width - 1) % 256 and y = (yCU + yBV) % 128, ..., (yCU + yBV + CU_height - 1) % 128) is marked as unavailable, the block vector is considered invalid.

[0609] When the CTU size is not 128×128 or is smaller than 64×64 or smaller than 64×64, the same process as in the previous embodiment (ie, embodiment #14) applies.

[0610] 5.15 Example #15

[0611] The IBC Reference Availability Marking process is described below. Throughout this document, changes are indicated by bold, underlined, and italic text.

[0612] 7.3.7.1 General Strip Data Syntax

[0613]

[0614]

[0615] 7.3.7.5 Codec unit syntax

[0616]

[0617] 8.6.2 Derivation of Motion Vector Components for IBC Blocks

[0618] 8.6.2.1 Overall

[0619] …

[0620]

[0621] …

[0622] 8.6.3 IBC Block Decoding Process

[0623] 8.6.3.1 Overall

[0624] This process is called when decoding a codec that was coded in ibc prediction mode.

[0625] The inputs to this process are:

[0626] – Luma position (xCb, yCb), specifies the upper left corner sample of the current codec block relative to the upper left corner luminance sample of the current picture,

[0627] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0628] – The variable cbHeight specifies the height of the current codec block in luminance samples.

[0629] – Variables numSbX and numSbY specify the number of luminance codec sub-blocks in the horizontal and vertical directions,

[0630] – motion vector mv[xSbIdx][ySbIdx], where xSbIdx=0..numSbX–1 and ySbIdx=0..numSbY–1,

[0631] –Variable cIdx specifies the color component index of the current block.

[0632] –

[0633] …

[0634] For each codec sub-block at sub-block index (xSbIdx, ySbIdx) (where xSbIdx = 0..numSbX - 1 and ySbIdx = 0..numSbY - 1), the following applies:

[0635] – The luma position (xSb, ySb) of the upper left corner luma sample of the specified current codec sub-block relative to the upper left corner luma sample of the current picture is derived as follows:

[0636] (xSb,ySb)=(xCb+xSbIdx*sbWidth,yCb+ySbIdx*sbHeight)(8-913)

[0637]

[0638] 8.6.3.2 Block Vector Validity Check Process

[0639]

[0640] 8.7.5 Image Reconstruction Process

[0641] 8.7.5.1 Overall

[0642] The inputs to this process are:

[0643] – Position (xCurr, yCurr), specifies the upper left corner sample point of the current block relative to the upper left corner sample point of the current picture component,

[0644] – Variables nCurrSw and nCurrSh specify the width and height of the current block, respectively.

[0645] – Variable cIdx, specifies the color component of the current block,

[0646] – (nCurrSw)x(nCurrSh) array predSamples, specifies the prediction samples of the current block,

[0647] –(nCurrSw)x(nCurrSh) array resSamples, specifying the residual samples of the current block.

[0648] The output of this process is

[0649] –Reconstructed image sample array recSamples.

[0650]

[0651] 5.16 Example #16

[0652] Same as the previous embodiment except for the following changes.

[0653]

[0654]

[0655]

[0656] 5.17 Example #17

[0657] Throughout this document, changes in some examples are indicated by bold, underlined text.

[0658] 7.3.7 Strip Data Syntax

[0659] 7.3.7.1 General Strip Data Syntax

[0660]

[0661]

[0662] 7.4.8.5 Codec unit semantics

[0663] The history-based motion vector predictor list for the shared Merge candidate list area is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i] (for i = 0..NumHmvpIbcCand - 1) when all of the following conditions are true:

[0664] –IsInSmr[x0][y0] is equal to TRUE.

[0665] –SmrX[x0][y0] is equal to x0.

[0666] –SmrY[x0][y0] is equal to y0.

[0667] For x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight-1, specify the following:

[0668] CbPosX[x][y]=x0 (7-135)

[0669] CbPosY[x][y]=y0 (7-136)

[0670] CbWidth[x][y]=cbWidth (7-137)

[0671] CbHeight[x][y]=cbHeight (7-138)

[0672]

[0673] 8.6.2 Derivation of Motion Vector Components for IBC Blocks

[0674] 8.6.2.1 Overall

[0675] The inputs to this process are:

[0676] – The luminance position (xCb, yCb) of the upper left corner sample of the current luminance codec block relative to the upper left corner luminance sample of the current picture,

[0677] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0678] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0679] The output of this process is:

[0680] – Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0681] The luminance motion vector mvL is derived as follows:

[0682] – The derivation process of the IBC luma motion vector prediction as specified in clause 8.6.2.2 is called with the luma position (xCb, yCb), the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL.

[0683] – When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0684] 1. The variable mvd is derived as follows:

[0685] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0686] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0687] 2. Call the motion vector rounding process as specified in clause 8.5.2.14 with mvX set equal to mvL, rightShift set equal to MvShift+2, and leftShift set equal to MvShift+2 as inputs and the rounded mvL as output.

[0688] 3. The luminance motion vector mvL is modified as follows:

[0689] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[0690] mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886)

[0691] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0692] mvL[1]=(u[1]>=2 17 )?(u[1]-218 ):u[1] (8-888)

[0693] NOTE 1 - The resulting values ​​of mvL[0] and mvL[1] as specified above are always in the range -2 17 to 2 17 -1 (including -2 17 and 2 17 -1) in.

[0694] The history-based motion vector prediction list update process as specified in clause 8.6.2.6 is called with the luma motion vector mvL.

[0695]

[0696] 8.7.5 Image Reconstruction Process

[0697] 8.7.5.1 Overall

[0698] The inputs to this process are:

[0699] – Position (xCurr, yCurr), specifies the upper left corner sample point of the current block relative to the upper left corner sample point of the current picture component,

[0700] – Variables nCurrSw and nCurrSh specify the width and height of the current block, respectively.

[0701] – Variable cIdx, specifies the color component of the current block,

[0702] – (nCurrSw)x(nCurrSh) array predSamples, specifies the prediction samples of the current block,

[0703] –(nCurrSw)x(nCurrSh) array resSamples, specifying the residual samples of the current block.

[0704] The output of this process Reconstruct the image sample array resSamples

[0705] Depending on the value of the color component cIdx, the following is specified:

[0706] – If cIdx is equal to 0, recSamples corresponds to the reconstructed picture sample array S L , and the function clipCidx1 corresponds to Clip1 Y .

[0707] – Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr], and recSamples corresponds to the reconstructed chroma sample array S Cb , and the function clipCidx1 corresponds to Clip1 C .

[0708] – Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr], recSamples corresponds to the reconstructed chroma sample array S Cr , and the function clipCidx1 corresponds to Clip1 C .

[0709] Depending on the value of slice_lmcs_enabled_flag, the following applies:

[0710] If slice_lmcs_enabled_flag is equal to 0, then for i = 0..nCurrSw1, j = 0..nCurrSh-1, the (nCurrSw) x (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows: recSamples[xCurr+i][yCurr+j] = clipCidx1(predSamples[i][j] + resSamples[i][j]) (8-992)

[0711] – Otherwise (slice_lmcs_enabled_flag is equal to 1), the following applies:

[0712] – If cIdx is equal to 0, the following applies:

[0713] – The picture reconstruction and mapping process of luma samples as specified in clause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples.

[0714] – Otherwise (cIdx is greater than 0), the picture reconstruction of chroma samples and luma-dependent chroma residual scaling process as specified in clause 8.7.5.3 is invoked with the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the codec block flag tuCbfChroma of the current chroma transform block, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples as input, and the output is the reconstructed chroma sample array recSamples.

[0715]

[0716] 5.18 Example #18

[0717] Throughout this document, changes in some examples are indicated by bold, underlined, and italic text.

[0718] 7.3.7 Strip Data Syntax

[0719] 7.3.7.1 General Strip Data Syntax

[0720]

[0721] 7.4.8.5 Codec unit semantics

[0722] The history-based motion vector predictor list for the shared Merge candidate list area is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i] (for i=0..NumHmvpIbcCand-1) when all of the following conditions are true:

[0723] –IsInSmr[x0][y0] is equal to TRUE.

[0724] –SmrX[x0][y0] is equal to x0.

[0725] –SmrY[x0][y0] is equal to y0.

[0726] For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, specify the following:

[0727] CbPosX[x][y]=x0 (7-135)

[0728] CbPosY[x][y]=y0 (7-136)

[0729] CbWidth[x][y]=cbWidth (7-137)

[0730] CbHeight[x][y]=cbHeight (7-138)

[0731]

[0732] 8.6.2 Derivation of Motion Vector Components for IBC Blocks

[0733] 8.6.2.1 Overall

[0734] The inputs to this process are:

[0735] – The luminance position (xCb, yCb) of the upper left corner sample of the current luminance codec block relative to the upper left corner luminance sample of the current picture,

[0736] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0737] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0738] The output of this process is:

[0739] – Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0740] The luminance motion vector mvL is derived as follows:

[0741] – The derivation process of the IBC luma motion vector prediction as specified in clause 8.6.2.2 is called with the luma position (xCb, yCb), the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL.

[0742] – When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0743] 4. The variable mvd is derived as follows:

[0744] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0745] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0746] 5. Call the motion vector rounding process as specified in clause 8.5.2.14 with mvX set equal to mvL, rightShift set equal to MvShift+2, and leftShift set equal to MvShift+2 as inputs and the rounded mvL as output.

[0747] 6. The luminance motion vector mvL is modified as follows:

[0748] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[0749] mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886)

[0750] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0751] mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888)

[0752] NOTE 1 - The resulting values ​​of mvL[0] and mvL[1] as specified above are always in the range -2 17 to 2 17 -1 (including -2 17 and 2 17 -1) in.

[0753] The history-based motion vector prediction list update process as specified in clause 8.6.2.6 is called with the luma motion vector mvL.

[0754]

[0755]

[0756] 8.6.3 IBC Block Decoding Process

[0757] 8.6.3.1 Overall

[0758] This process is called when decoding a codec that was coded in ibc prediction mode.

[0759] The inputs to this process are:

[0760] – Luma position (xCb, yCb), specifies the upper left corner sample of the current codec block relative to the upper left corner luminance sample of the current picture,

[0761] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0762] – The variable cbHeight specifies the height of the current codec block in luminance samples.

[0763] – Color component index of the current block.

[0764]

[0765] The output of this process is:

[0766] – Array of prediction samples predSamples.

[0767]

[0768] 8.7.5 Image Reconstruction Process

[0769] 8.7.5.1 Overall

[0770] The inputs to this process are:

[0771] – Position (xCurr, yCurr), specifies the upper left corner sample point of the current block relative to the upper left corner sample point of the current picture component,

[0772] – Variables nCurrSw and nCurrSh specify the width and height of the current block, respectively.

[0773] – Variable cIdx, specifies the color component of the current block,

[0774] – (nCurrSw)x(nCurrSh) array predSamples, specifies the prediction samples of the current block,

[0775] –(nCurrSw)x(nCurrSh) array resSamples, specifying the residual samples of the current block.

[0776] The output of this process is the reconstructed image sample array recSamples

[0777] Depending on the value of the color component cIdx, the following is specified:

[0778] – If cIdx is equal to 0, recSamples corresponds to the reconstructed picture sample array S L , and the function clipCidx1 corresponds to Clip1Y .

[0779] – Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr], and recSamples corresponds to the reconstructed chroma sample array S Cb , and the function clipCidx1 corresponds to Clip1 C .

[0780] – Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr], recSamples corresponds to the reconstructed chroma sample array S Cr , and the function clipCidx1 corresponds to Clip1 C .

[0781] Depending on the value of slice_lmcs_enabled_flag, the following applies:

[0782] If slice_lmcs_enabled_flag is equal to 0, then for i = 0..nCurrSw-1, j = 0..nCurrSh-1, the (nCurrSw) x (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows:

[0783] recSamples[xCurr+i][yCurr+j]=clipCidx1(predSamples[i][j]+resSamples[i][j]) (8-992)

[0784] – Otherwise (slice_lmcs_enabled_flag is equal to 1), the following applies:

[0785] – If cIdx is equal to 0, the following applies:

[0786] – The picture reconstruction and mapping process of luma samples as specified in clause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples.

[0787] – Otherwise (cIdx is greater than 0), the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the codec block flag of the current chroma transform block

[0788] The picture reconstruction of chroma samples and luma-dependent chroma residual scaling process as specified in clause 8.7.5.3 is invoked with tuCbfChroma, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples as input, and the output is the reconstructed chroma sample array recSamples.

[0789]

[0790] 5.19 Example #19

[0791] Throughout this document, changes in some examples are indicated by bold, underlined text.

[0792] 7.3.7 Strip Data Syntax

[0793] 7.3.7.1 General Strip Data Syntax

[0794]

[0795]

[0796] 7.4.8.5 Codec unit semantics

[0797] The history-based motion vector predictor list for the shared Merge candidate list area is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i] (for i = 0..NumHmvpIbcCand - 1) when all of the following conditions are true:

[0798] –IsInSmr[x0][y0] is equal to TRUE.

[0799] –SmrX[x0][y0] is equal to x0.

[0800] –SmrY[x0][y0] is equal to y0.

[0801] For x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight-1, specify the following:

[0802] CbPosX[x][y]=x0 (7-135)

[0803] CbPosY[x][y]=y0 (7-136)

[0804] CbWidth[x][y]=cbWidth (7-137)

[0805] CbHeight[x][y]=cbHeight (7-138)

[0806]

[0807]

[0808] 8.6.2 Derivation of Motion Vector Components for IBC Blocks

[0809] 8.6.2.1 Overall

[0810] The inputs to this process are:

[0811] – The luminance position (xCb, yCb) of the upper left corner sample of the current luminance codec block relative to the upper left corner luminance sample of the current picture,

[0812] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0813] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0814] The output of this process is:

[0815] – Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0816] The luminance motion vector mvL is derived as follows:

[0817] – The derivation process of the IBC luma motion vector prediction as specified in clause 8.6.2.2 is called with the luma position (xCb, yCb), the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL.

[0818] – When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0819] 7. The variable mvd is derived as follows:

[0820] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0821] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0822] 8. Call the motion vector rounding process as specified in clause 8.5.2.14 with mvX set equal to mvL, rightShift set equal to MvShift+2, and leftShift set equal to MvShift+2 as inputs and the rounded mvL as output.

[0823] 9. The luminance motion vector mvL is modified as follows:

[0824] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[0825] mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886)

[0826] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0827] mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888)

[0828] NOTE 1 - The resulting values ​​of mvL[0] and mvL[1] as specified above are always in the range -2 17 to 2 17 -1 (including -2 17 and 2 17 -1) in.

[0829] The history-based motion vector prediction list update process as specified in clause 8.6.2.6 is called with the luma motion vector mvL.

[0830]

[0831] 8.6.3 IBC Block Decoding Process

[0832] 8.6.3.1 Overall

[0833] This process is called when decoding a codec that was coded in ibc prediction mode.

[0834] The inputs to this process are:

[0835] – Luma position (xCb, yCb), specifies the upper left corner sample of the current codec block relative to the upper left corner luminance sample of the current picture,

[0836] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0837] – The variable cbHeight specifies the height of the current codec block in luminance samples.

[0838]

[0839] 8.7.5 Image Reconstruction Process

[0840] 8.7.5.1 Overall

[0841] The inputs to this process are:

[0842] – Position (xCurr, yCurr), specifies the upper left corner sample point of the current block relative to the upper left corner sample point of the current picture component,

[0843] – Variables nCurrSw and nCurrSh specify the width and height of the current block, respectively.

[0844] – Variable cIdx, specifies the color component of the current block,

[0845] – (nCurrSw)x(nCurrSh) array predSamples, specifies the prediction samples of the current block,

[0846] –(nCurrSw)x(nCurrSh) array resSamples, specifies the residual samples of the current block. The output of this process is the reconstructed picture sample array recSamples and the IBC buffer array ibcBuf L ibcBuf Cb ibcBuf Cr .

[0847] Depending on the value of the color component cIdx, the following is specified:

[0848] – If cIdx is equal to 0, recSamples corresponds to the reconstructed picture sample array S L , and the function clipCidx1 corresponds to Clip1 Y .

[0849] – Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr], and recSamples corresponds to the reconstructed chroma sample array S Cb , and the function clipCidx1 corresponds to Clip1 C .

[0850] – Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr], recSamples corresponds to the reconstructed chroma sample array S Cr , and the function clipCidx1 corresponds to Clip1 C .

[0851] Depending on the value of slice_lmcs_enabled_flag, the following applies:

[0852] If slice_lmcs_enabled_flag is equal to 0, then for i = 0..nCurrSw–1, j = 0..nCurrSh–1, the (nCurrSw) x (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows:

[0853] recSamples[xCurr+i][yCurr+j]=clipCidx1(predSamples[i][j]+resSamples[i][j]) (8-992)

[0854] – Otherwise (slice_lmcs_enabled_flag is equal to 1), the following applies:

[0855] – If cIdx is equal to 0, the following applies:

[0856] – The picture reconstruction and mapping process of luma samples as specified in clause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples.

[0857] – Otherwise (cIdx is greater than 0), the picture reconstruction of chroma samples and luma-dependent chroma residual scaling process as specified in clause 8.7.5.3 is invoked with the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the codec block flag tuCbfChroma of the current chroma transform block, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples as input, and the output is the reconstructed chroma sample array recSamples.

[0858]

[0859]

[0860] 5.20 Example #20

[0861] Throughout this document, changes in some examples are indicated by bold, underlined, and italic text.

[0862] 7.3.7 Strip Data Syntax

[0863] 7.3.7.1 General Strip Data Syntax

[0864]

[0865]

[0866] 7.4.8.5 Codec unit semantics

[0867] The history-based motion vector predictor list for the shared Merge candidate list area is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i] (for i = 0..NumHmvpIbcCand - 1) when all of the following conditions are true:

[0868] –IsInSmr[x0][y0] is equal to TRUE.

[0869] –SmrX[x0][y0] is equal to x0.

[0870] –SmrY[x0][y0] is equal to y0.

[0871] For x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight-1, specify the following:

[0872] CbPosX[x][y]=x0 (7-135)

[0873] CbPosY[x][y]=y0 (7-136)

[0874] CbWidth[x][y]=cbWidth (7-137)

[0875] CbHeight[x][y]=cbHeight (7-138)

[0876]

[0877]

[0878] 8.6.2 Derivation of Motion Vector Components for IBC Blocks

[0879] 8.6.2.1 Overall

[0880] The inputs to this process are:

[0881] – The luminance position (xCb, yCb) of the upper left corner sample of the current luminance codec block relative to the upper left corner luminance sample of the current picture,

[0882] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0883] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0884] The output of this process is:

[0885] – Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0886] The luminance motion vector mvL is derived as follows:

[0887] – The derivation process of the IBC luma motion vector prediction as specified in clause 8.6.2.2 is called with the luma position (xCb, yCb), the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL.

[0888] – When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0889] 10. The variable mvd is derived as follows:

[0890] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0891] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0892] 11. Call the motion vector rounding process as specified in clause 8.5.2.14 with mvX set equal to mvL, rightShift set equal to MvShift+2, and leftShift set equal to MvShift+2 as inputs and the rounded mvL as output.

[0893] 12. The luminance motion vector mvL is modified as follows:

[0894] u[0]=(mvL[0]+mvd[0]+2 18 )%218 (8-885)

[0895] mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886)

[0896] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0897] mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888)

[0898] NOTE 1 - The resulting values ​​of mvL[0] and mvL[1] as specified above are always in the range -2 17 to 2 17 -1 (including -2 17 and 2 17 -1) in.

[0899] The history-based motion vector prediction list update process as specified in clause 8.6.2.6 is called with the luma motion vector mvL.

[0900]

[0901] 8.6.3 IBC Block Decoding Process

[0902] 8.6.3.1 Overall

[0903] This process is called when decoding a codec that was coded in ibc prediction mode.

[0904] The inputs to this process are:

[0905] – Luma position (xCb, yCb), specifies the upper left corner sample of the current codec block relative to the upper left corner luminance sample of the current picture,

[0906] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0907] – The variable cbHeight specifies the height of the current codec block in luminance samples.

[0908] –Variable cIdx specifies the color component index of the current block.

[0909]

[0910]

[0911] 8.7.5 Image Reconstruction Process

[0912] 8.7.5.1 Overall

[0913] The inputs to this process are:

[0914] – Position (xCurr, yCurr), specifies the upper left corner sample point of the current block relative to the upper left corner sample point of the current picture component,

[0915] – Variables nCurrSw and nCurrSh specify the width and height of the current block, respectively.

[0916] – Variable cIdx, specifies the color component of the current block,

[0917] – (nCurrSw)x(nCurrSh) array predSamples, specifies the prediction samples of the current block,

[0918] –(nCurrSw)x(nCurrSh) array resSamples, specifying the residual samples of the current block.

[0919] The output of this process yes Reconstructed image sample array recSamples and the IBC buffer array ibcBuf L 、 ibcBuf Cb 、ibcBuf Cr .

[0920] Depending on the value of the color component cIdx, the following is specified:

[0921] – If cIdx is equal to 0, recSamples corresponds to the reconstructed picture sample array S L , and the function clipCidx1 corresponds to Clip1 Y .

[0922] – Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr], and recSamples corresponds to the reconstructed chroma sample array S Cb , and the function clipCidx1 corresponds to Clip1 C .

[0923] – Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr], recSamples corresponds to the reconstructed chroma sample array S Cr , and the function clipCidx1 corresponds to Clip1 CDepending on the value of slice_lmcs_enabled_flag, the following applies:

[0924] If slice_lmcs_enabled_flag is equal to 0, then for i = 0..nCurrSw–1, j = 0..nCurrSh–1, the (nCurrSw) x (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows:

[0925] recSamples[xCurr+i][yCurr+j]=clipCidx1(predSamples[i][j]+resSamples[i][j]) (8-992)

[0926] – Otherwise (slice_lmcs_enabled_flag is equal to 1), the following applies:

[0927] – If cIdx is equal to 0, the following applies:

[0928] – The picture reconstruction and mapping process of luma samples as specified in clause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples.

[0929] – Otherwise (cIdx is greater than 0), the picture reconstruction of chroma samples and luma-dependent chroma residual scaling process as specified in clause 8.7.5.3 is invoked with the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the codec block flag tuCbfChroma of the current chroma transform block, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples as input, and the output is the reconstructed chroma sample array recSamples.

[0930]

[0931]

[0932] Figure 66 is a flow chart of an example method 600 for visual media (video or image) processing. The method 600 includes: determining (602) a size of a buffer to store reference samples of the current video block using an intra block copy codec mode for conversion between a current video block and a bitstream representation of the current video block; and performing (604) the conversion using the reference samples stored in the buffer.

[0933] The following clauses describe some example preferred features implemented by embodiments of method 600 and other methods.Additional examples are provided in Section 4 of this document.

[0934] 1. A method of video processing, comprising: determining, for conversion between a current video block and a bitstream representation of the current video block, a size of a buffer to store reference samples of the current video block using an intra block copy codec mode; and performing the conversion using the reference samples stored in the buffer.

[0935] 2. The method of clause 1, wherein the size of the buffer is a predetermined constant.

[0936] 3. The method of any of clauses 1-2, wherein the size is M×N, where M and N are integers.

[0937] 4. The method of clause 3, wherein M×N is equal to 64×64 or 128×128 or 64×128.

[0938] 5. The method of clause 1, wherein the size of the buffer is equal to the size of the codec tree unit of the current video block.

[0939] 6. The method of clause 1, wherein the size of the buffer is equal to the size of the virtual pipe data unit used during the conversion.

[0940] 7. The method of clause 1, wherein the size of the buffer corresponds to a field in the bitstream representation.

[0941] 8. A method according to clause 7, wherein the field is included in the bitstream representation at the video parameter set or sequence parameter set or picture parameter set or picture header or slice header or slice group header level.

[0942] 9. A method according to any of clauses 1-8, wherein the buffer has different sizes for the reference samples of the luma component and the reference samples of the chroma components.

[0943] 10. A method according to any of clauses 1-8, wherein the size of the buffer depends on the chroma subsampling format of the current video block.

[0944] 11. A method according to any of clauses 1-8, wherein the reference samples are stored in RGB format.

[0945] 12. The method according to any of clauses 1 to 11, wherein the buffer is used to store reconstructed samples before loop filtering and reconstructed samples after loop filtering.

[0946] 13. The method of clause 12, wherein the loop filtering comprises deblocking filtering or adaptive loop filtering (ALF) or sample adaptive offset (SAO) filtering.

[0947] 14. A method of video processing, comprising: for conversion between a current video block and a bitstream representation of the current video block, initializing a buffer using initial values ​​of reference samples to store reference samples of the current video block using an intra block copy codec mode; and performing the conversion using the reference samples stored in the buffer.

[0948] 15. The method of clause 14, wherein the initial value corresponds to a constant.

[0949] 16. A method according to any of clauses 14-15, wherein the initial value is a function of the bit depth of the current video block.

[0950] 17. The method of clause 15, wherein the constant corresponds to an intermediate grayscale value.

[0951] 18. The method of clause 14, wherein the initial value corresponds to a pixel value of a previously decoded video block.

[0952] 19. The method of clause 18, wherein the previously decoded video block corresponds to a decoded block prior to loop filtering.

[0953] 20. The method according to any of clauses 14-19, wherein the size of the buffer is as described in one of clauses 1-13.

[0954] 21. A method according to any of clauses 1 to 20, wherein pixel positions within the buffer are addressed using an x-number and a y-number.

[0955] 22. A method according to any of clauses 1 to 20, wherein pixel positions within the buffer are addressed using a single number extending from 0 to M*N-1, where M and N are the pixel width and pixel height of the buffer.

[0956] 23. A method according to any of clauses 1-20, wherein the current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0, y-y0), where (x0, y0) corresponds to the upper left corner position of the codec tree unit of the current video block.

[0957] 24. A method according to any of clauses 1-20, wherein the current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0+Tx, y-y0+Ty), where (x0, y0) corresponds to the top left corner position of the codec tree unit of the current video block, and where Tx and Ty are offset values.

[0958] 25. The method of clause 24, wherein Tx and Ty are predefined offset values.

[0959] 26. A method according to any of clauses 1-20, wherein during the conversion, for a pixel at position (x0, y0) and having a block vector (BVx, BVy), a corresponding reference in the buffer is found at a reference position (x0+BVx, y0+BVy).

[0960] 27. A method according to clause 26, wherein, in the event that the reference position is outside the buffer zone, the reference in the buffer zone is determined by clipping at the border of the buffer zone.

[0961] 28. The method of clause 26, wherein, in the event that the reference position is outside the buffer zone, the reference in the buffer zone is determined to have a predetermined value.

[0962] 29. A method according to any of clauses 1-20, wherein during the conversion, for a pixel at position (x0, y0) and having a block vector (BVx, BVy), the corresponding reference in the buffer is found at the reference position ((x0+BVx)mod M, (y0+BVy)mod N), where "mod" is a modulo operation and M and N are integers representing the x-dimension and y-dimension of the buffer.

[0963] 30. A method of video processing, comprising: resetting a buffer storing reference samples for intra block copy coding at video boundaries during conversion between a video and a bitstream representation of a current video block; and performing the conversion using the reference samples stored in the buffer.

[0964] 31. The method of clause 30, wherein the video boundary corresponds to a new picture or a new slice.

[0965] 32. The method of clause 30, wherein the converting is performed by updating the buffer with reconstructed values ​​of virtual pipe data units (VPDUs) after a reset.

[0966] 33. The method of clause 30, wherein the converting is performed by updating the buffer with the reconstructed value of the codec tree unit after a reset.

[0967] 34. The method of clause 30, wherein resetting is performed at the beginning of each codec tree unit row.

[0968] 35. The method of clause 1, wherein the size of the buffer corresponds to L 64x64 previously decoded blocks, where L is an integer.

[0969] 36. A method according to any of clauses 1 to 35, wherein a vertical scan order is used for reading samples from a buffer or storing samples in a buffer during the conversion.

[0970] 37. A method of video processing, comprising: for conversion between a current video block and a bitstream representation of the current video block, using a buffer to store reference samples of the current video block using an intra block copy codec mode, wherein a first bit depth of the buffer is different from a second bit depth of the codec data; and performing the conversion using the reference samples stored in the buffer.

[0971] 38. The method of clause 37, wherein the first bit depth is greater than the second bit depth.

[0972] 39. A method according to any of clauses 37-38, wherein the first bit depth is the same as the bit depth of the reconstruction buffer used during the conversion.

[0973] 40. A method according to any of clauses 37-39, wherein the first bit depth is signaled in the bitstream representation as a value or a difference.

[0974] 41. A method according to any of clauses 37-40, wherein the conversion uses different bit depths for chroma components and luma components.

[0975] Additional embodiments and examples of clauses 37 to 41 are described in Section 4, Item 7.

[0976] 42. A method of video processing, comprising: performing a conversion between a current video block and a bitstream representation of the current video block using an intra block copy mode, wherein in the intra block copy mode, a first precision used for prediction calculations during the conversion is lower than a second precision used for reconstruction calculations during the conversion.

[0977] 43. The method according to clause 43, wherein the prediction calculation includes determining a predicted sample value from a reconstructed sample value using clip{{p + [1 << (b - 1)]} >> b, 0, (1 << bitdepth) - 1} << b, where p is the reconstructed sample value, b is a predefined bit shift value, and bitdepth is the prediction sample precision.

[0978] Additional embodiments and examples of clauses 42 to 43 are described in items 28 to 31 and item 34 of Chapter 4.

[0979] 44. A method for video processing, comprising: performing a conversion between a current video block using an intra block copy mode and a bitstream representation of the current video block, wherein in the intra block copy mode, a reference region of size nM × nM is used for a coding tree unit of size M × M, where n and M are integers, and wherein the current video block is located in the coding tree unit, and wherein the reference region is the nearest available n × n coding tree unit in the row of the coding tree unit corresponding to the current video block.

[0980] Additional embodiments and examples of clause 4 are described in item 35 of Chapter 4.

[0981] 45. A method for video processing, comprising: performing a conversion between a current video block using an intra block copy mode and a bitstream representation of the current video block, wherein in the intra block copy mode, a reference region of size nM × nM is used for a coding tree unit other than M × M, where n and M are integers, and wherein the current video block is located in the coding tree unit, and wherein the reference region is the nearest available n × n - 1 coding tree unit in the row of the coding tree unit corresponding to the current video block.

[0982] Additional embodiments and examples of clause 4 are described in item 36 of Chapter 4. Figure 8 and Figure 9 Additional example embodiments are shown.

[0983] 46. The method according to clause 3, wherein M = mW and N = H, where W and H are the width and height of a coding tree unit (CTU) of the current video block, and m is a positive integer.

[0984] 47. The method according to clause 3, wherein M = W and N = nH, where W and H are the width and height of a coding tree unit (CTU), and n is a positive integer.

[0985] 48. The method according to clause 3, wherein M = mW and N = nH, where W and H are the width and height of a coding tree unit (CTU), and m and n are positive integers.

[0986] 49. A method according to any of clauses 46-48, wherein n and m depend on the size of the CTU.

[0987] 50. A method of video processing, comprising: for a conversion between a current video block of a video and a bitstream representation of the current video block, determining the validity of a block vector corresponding to a component c of the video using a component X of the video, wherein the component X is different from a luma component of the video; and performing the conversion using the block vector when the block vector is determined to be valid for the current video block. Here, the block vector represented as (BVx, BVy) is equal to (x-x0, y-y0), where (x0, y0) corresponds to the top left corner position of a codec tree unit of the current video block.

[0988] 51. The method of clause 50, wherein component c corresponds to a luma component of the video.

[0989] 52. The method of clause 50, wherein the current video block is a chroma block and the video is in 4:4:4 format.

[0990] 53. A method as described in clause 50, wherein the video is in 4:2:0 format, and wherein the current video block is a chroma block starting at position (x, y), and wherein the determining includes determining the block vector as invalid for the case where isRec(c, ((x+BVx)>>5<<5)+64-(((y+BVy)>>5)&1)*32+(x%32), ((y+BVy)>>5<<5)+(y%32)) is true.

[0991] 54. A method as described in clause 50, wherein the video is in 4:2:0 format, and wherein the current video block is a chroma block starting at position (x, y), and wherein the determining includes determining the block vector as invalid for the case if isRec(c,x+BVx+Chroma_CTU_size,y) is true.

[0992] 55. A method of video processing, comprising: for a conversion between a current video block of a current virtual pipe data unit (VPDU) of a video region and a bitstream representation of the current video block, selectively determining to use K1 previously processed VPDUs from a first row of the video region and K2 previously processed VPDUs from a second row of the video region; and performing the conversion, wherein the conversion does not include using a remaining portion of the current VPDU.

[0993] 56. The method of clause 55, wherein K1=1 and K2=2.

[0994] 57. A method according to any of clauses 55-56, wherein the current video block is selectively processed based on the size of the video region or the size of the current VPDU.

[0995] 58. A method of video processing, comprising: performing a validity check of a block vector for a conversion between a current video block and a bitstream representation of the current video block, wherein the block vector is used in an intra block copy mode; and using a result of the validity check to selectively use the block vector during the conversion.

[0996] 59. The method of clause 58, wherein an intra-block copy (IBC) buffer is used during the conversion, wherein the width and height of the IBC buffer are Wbuf and Hbuf, the size of the current video block is W×H, and wherein the block vector is represented as (BVx, BVy), and wherein the current video block is in a current picture having sizes Wpic and Hpic and a codec tree unit having Wctu and Hctu as width and height, and wherein the validity check uses predetermined rules.

[0997] 60. A method according to any of clauses 58-59, wherein the current video block is a luma block, a chroma block, a coding unit (CU), a transform unit (TU), a 4x4 block, a 2x2 block, or a child block of a parent block starting from pixel coordinates (X, Y).

[0998] 61. A method according to any of clauses 58-60, wherein the validity check considers block vectors that fall outside the boundaries of the current picture to be valid.

[0999] 62. A method according to any of clauses 58-60, wherein the validity check considers block vectors that fall outside the boundaries of a codec tree unit as valid.

[1000] Items 23-30 in the previous section provide additional examples and variations of the above items 58-62.

[1001] 63. A method according to any of clauses 1-62, wherein the converting comprises generating a bitstream representation from the current video block.

[1002] 64. A method according to any of clauses 1-62, wherein the converting comprises generating pixel values ​​of the current video block from a bitstream representation.

[1003] 65. A video encoder apparatus comprising a processor configured to implement the method of any one or more of clauses 1-62.

[1004] 66. A video decoder device comprising a processor configured to implement the method according to any one or more of clauses 1-62.

[1005] 67. A computer-readable medium having stored thereon code embodying processor-executable instructions for implementing the method of any one or more of clauses 1-62.

[1006] Figure 7 7 is a block diagram of a hardware platform for a video / image processing device 700. The device 700 can be used to implement one or more of the methods described herein. The device 700 can be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 700 may include one or more processors 702, one or more memories 704, and video processing hardware 706. The processor(s) 702 may be configured to implement one or more methods described in this document (including but not limited to method 600). The memory(s) 704 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 706 may be used to implement some of the techniques described in this document in hardware circuitry.

[1007] The bitstream representation corresponding to the current video block does not need to be a contiguous set of bits and may be distributed across headers, parameter sets, and Network Abstraction Layer (NAL) packets.

[1008] Section A: Another Additional Example Embodiment

[1009] In Section A, we present another example embodiment in which the current version of the VVC standard can be modified to implement some of the techniques described in this document.

[1010] This section analyzes several issues in the current IBC reference buffer design and presents different designs to address the issues. Instead of mixing with the decoder memory, a separate IBC reference buffer is proposed. Compared to the current anchor, the proposed scheme shows an AI / RA / LD-B luminance BD rate of -0.99% / -0.71% / -0.79% for class F and -2.57% / -1.81% / -1.36% for 4:2:0 TGM, with a 6.7% memory reduction; or an AI / RA / LD-B luminance BD rate of -1.31% / -1.01% / -0.81% for class F and -3.23% / -2.33% / -1.71% for 4:2:0 TGM, with a 6.7% memory increase.

[1011] A1. Introduction

[1012] The Intra Block Copy, or IBC (or Current Picture Reference, or previous CPR) codec mode is used. It has been recognized that the IBC reference samples should be stored in on-chip memory, so a limited reference area of ​​a CTU is defined. In order to limit the additional on-chip memory used for the buffer, the current design reuses 64×64 memory for decoding the current VPDU, so that only 3 additional 64×64 blocks of memory are needed to support IBC. When the CTU size is 128×128, the reference area is currently Figure 2 is shown in .

[1013] In the current draft (VVC draft 4), regions are defined as:

[1014]

[1015]

[1016] Therefore, the total reference size is CTU.

[1017] A2. Potential problems with the current design

[1018] The current design assumes that 64×64 memory is reused for decoding the current VPDU and aligns the IBC reference with the VPDU memory reuse accordingly. This design ties the VPDU decoding memory and the IBC buffer together. There may be several problems:

[1019] 1. Handling smaller CTUs may be a problem. Assuming a CTU size of 32×32, it is unclear whether the current 64×64 memory used to decode the current VPDU can effectively support 32×32 level memory reuse in a different architecture.

[1020] 2. The reference region changes significantly. As a result, excessive bitstream consistency constraints are introduced. This places an additional burden on the encoder to effectively utilize the reference region and avoid generating legal bitstreams. This also increases the possibility of having invalid BVs in different modules (for example, merge lists). To handle those invalid BVs, additional logic or additional consistency constraints may be introduced. This not only imposes a burden on the encoder or decoder, but may also cause differences between BV and MV codecs.

[1021] 3. The design does not scale well. Because VPDU decoding is intermixed with the IBC buffer, it is not easy to increase or decrease the reference area relative to the current 128×128 CTU design. This may limit the flexibility to exploit tradeoffs for better codec efficiency versus on-chip memory in future developments (e.g., lower or higher profiles).

[1022] 4. The bit depth of the IBC reference buffer is tied to the decode buffer. Even though the screen content typically has a lower bit depth than the internal decode bit depth, the buffer still requires memory to store bits that primarily represent rounding or quantization noise. This problem becomes even more severe when considering higher decode bit depth configurations.

[1023] A3. Clear IBC buffer design

[1024] To address the issues listed in the above subsections, we propose to utilize a dedicated IBC buffer that is not intermingled with the decoded memory.

[1025] For 128×128 CTU, the buffer is defined as 128×128 with 8-bit samples. When a CU (x, y) of size w×h has been decoded, its reconstruction before loop filtering is converted to 8 bits and written to a w×h block area starting from position (x%128, y%128). Here, the modulo operator % always returns a positive number, i.e. for x<0, For example, -3%128=125.

[1026] Assume that pixel (x, y) is encoded and decoded in IBC mode with BV=(BVx, BVy), its predicted sample in the IBC reference buffer is located at ((x+BVx)%128, (y+BVy)%128), and the pixel value will be converted to 10 bits before prediction.

[1027] When the buffer is considered to be (W, H), after decoding the CTU or CU starting from (x, y), the reconstructed pixels before loop filtering will be stored in the buffer starting from (x%W, y%H). Therefore, after decoding the CTU, the corresponding IBC reference buffer will be updated accordingly. Such a setting may happen when the CTU size is not 128×128. For example, for a 64×64 CTU, with the current buffer size, it can be considered as a 256×64 buffer. For a 64×64 CTU, Figure 2 Shows the buffer status.

[1028] Figure 12 is an illustration of the IBC reference buffer state, where a block represents a 64×64 CTU.

[1029] In such a design, since the IBC buffer is different from the VPDU decoding memory, all IBC reference buffers can be used as references.

[1030] When the bit depth of the IBC buffer is 8 bits, the on-chip memory increase is (8*4) / (10*3)-100%=6.7% compared to the current design which requires 3 additional 10-bit 64×64 buffers.

[1031] If we further reduce the bit depth, we can further reduce the memory requirement. For example, for a 7-bit buffer, the on-chip memory saving is 100% - (7*4) / (10*3) = 6.7%.

[1032] With this design, the only bitstream consistency constraint is that the reference block should be within the reconstructed region in the current CTU row of the current slice.

[1033] When initialization to 512 is allowed at the beginning of each CTU row, all bitstream consistency constraints can be removed.

[1034] A4. Experimental Results

[1035] In some embodiments, the disclosed methods may be implemented using VTM-4.0 software.

[1036] For the 10-bit buffer implementation and CTC, the decoder is fully compatible with the current VTM4.0 encoder, which means that the proposed decoder can correctly decode VTM-4.0 CTC bitstreams.

[1037] For the 7-bit buffer implementation, the results are shown in Table 1.

[1038] For the 8-bit buffer implementation, the results are shown in Table II.

[1039] Table I.7 Performance of the bit buffer. The anchor is VTM-4.0 with IBC turned on for all sequences.

[1040]

[1041]

[1042] Table II. Performance of 8-bit buffer. The anchor is VTM-4.0 with IBC turned on for all sequences.

[1043]

[1044]

[1045] Figure 171 is a block diagram illustrating an example video processing system 1700 in which the various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1700. System 1700 may include an input 1702 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or may be in a compressed or encoded format. Input 1702 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[1046] System 1700 may include a codec component 1704 that can implement the various codecs or encoding methods described in this document. Codec component 1704 can reduce the average bit rate of the video from input 1702 to the output of codec component 1704 to produce a coded representation of the video. Codec technology is therefore sometimes referred to as video compression or video transcoding technology. The output of codec component 1704 can be stored or sent via a communication connection such as represented by component 1706. The stored or communicated bitstream (or codec) representation of the video received at input 1702 can be used by component 1708 to generate pixel values ​​or transmit to a displayable video of display interface 1710. The process of generating user-viewable video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it will be understood that the codec tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the codec results will be performed by the decoder.

[1047] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, and IDE interfaces, etc. The technology described in this document may be embodied in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[1048] Figure 181 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 1 in Section 4 of this document. At step 1802, the process determines the size of a buffer to store reference samples for prediction in intra block copy mode. At step 1804, the process uses the reference samples stored in the buffer to perform a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the conversion is performed in intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1049] Figure 19 1 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 4 in Section 4 of this document. At step 1902, the process determines a buffer storing reconstructed samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the buffer is used to store the reconstructed samples before a loop filtering step. At step 1904, the process uses the reconstructed samples stored in the buffer to perform the conversion, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1050] Figure 20 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 5 in Section 4 of this document. At step 2002, the process determines a buffer storing reconstructed samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the buffer is used to store the reconstructed samples after a loop filtering step. At step 2004, the process uses the reconstructed samples stored in the buffer to perform the conversion, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1051] Figure 21is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 6 in Section 4 of this document. At step 2102, the process determines a buffer storing reconstructed samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the buffer is used to store both the reconstructed samples before a loop filtering step and the reconstructed samples after the loop filtering step. At step 2104, the process uses the reconstructed samples stored in the buffer to perform the conversion, wherein the conversion is performed in an intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[1052] Figure 22 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 7 in Section 4 of this document. At step 2202, the process uses a buffer to store reference samples for prediction in an intra block copy mode, wherein a first bit depth of the buffer is different from a second bit depth used to represent the visual media data in a bitstream representation. At step 2204, the process uses the reference samples stored in the buffer to perform a conversion between a current video block of the visual media data and a bitstream representation of the current video block, wherein the conversion is performed in an intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[1053] Figure 23 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 8 in Section 4 of this document. At step 2302, the process initializes a buffer to store reference samples for prediction in intra block copy mode, wherein the buffer is initialized with a first value. At step 2304, the process uses the reference samples stored in the buffer to perform a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the conversion is performed in intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture.

[1054] Figure 24is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 9 in Section 4 of this document. At step 2402, the process initializes a buffer to store reference samples for prediction in an intra block copy mode, wherein the buffer is initialized with pixel values ​​of one or more video blocks in the visual media data based on the availability of one or more video blocks in the visual media data. At step 2404, the process uses the reference samples stored in the buffer to perform a conversion between a current video block that is not part of the one or more video blocks of the visual media data and a bitstream representation of the current video block, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1055] Figure 25 is a flow chart of an example method for visual data processing. The steps of the flow chart are discussed in conjunction with Example Embodiment 14c in Section 4 of this document. At step 2502, the process determines a buffer storing reference samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block. At step 2504, the process performs the conversion using the reference samples stored in the buffer, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture. At step 2506, the process calculates a corresponding reference (P mod M, Q mod N) in the buffer based on the reference position, where "mod" is a modulo operation and M and N are integers representing the x-dimension and y-dimension of the buffer, for the pixel spatially located at position (x0, y0) and having the block vector (BVx, BVy) included in the motion information, wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0).

[1056] Figure 26is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiments 14a-14b in Section 4 of this document. At step 2602, the process determines a buffer storing reference samples for prediction in intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block. At step 2604, the process uses the reference samples stored in the buffer to perform the conversion, wherein the conversion is performed in intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture. At step 2606, the process calculates a corresponding reference in the buffer based on a reference position (P, Q) for a pixel located at a spatial position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0).

[1057] Figure 27 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 10 in Section 4 of this document. At step 2702, the process determines a buffer storing reference samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein pixel locations in the buffer are addressed using an x-number and a y-number. At step 2704, the process performs the conversion using the reference samples stored in the buffer based on the x-number and the y-number, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1058] Figure 28is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 15 in Section 4 of this document. At step 2802, the process determines a buffer storing reference samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the conversion is performed in intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture. At step 2804, the process calculates a corresponding reference in the buffer at a reference position (P, Q) for a pixel spatially located at a position (x0, y0) of the current video block and having a block vector (BVx, BVy), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0). At step 2806, the process recalculates the reference position using samples in the buffer when it is determined that the reference position (P, Q) is outside the buffer.

[1059] Figure 29 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 16 in Section 4 of this document. At step 2902, the process determines a buffer storing reference samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the conversion is performed in intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture. At step 2904, the process calculates a corresponding reference in the buffer at a reference position (P, Q) for a pixel spatially located at a position (x0, y0) of the current video block relative to the top left corner of the codec tree unit that includes the current video block and having a block vector (BVx, BVy), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0). At step 2906 , the process constrains at least a portion of the reference position to be within a predefined range upon determining that the reference position (P, Q) is outside the buffer zone.

[1060] Figure 30is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 17 in Section 4 of this document. At step 3002, the process determines a buffer storing reference samples for prediction in an intra block copy mode for a conversion between a current video block of visual media data and a bitstream representation of the current video block, wherein the conversion is performed in intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture. At step 3004, the process calculates a corresponding reference in the buffer at a reference position (P, Q) for a pixel spatially located at a position (x0, y0) of the current video block relative to the top left corner of the codec tree unit that includes the current video block and having a block vector (BVx, BVy), wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0). At step 1706 , upon determining that the block vector (BVx, BVy) is outside the buffer, the process fills the block vector (BVx, BVy) with block vectors of sample values ​​within the buffer.

[1061] Figure 31 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 30 in Section 4 of this document. At step 3102, the process resets a buffer storing reference samples used for prediction in an intra block copy mode at a video boundary during conversion between a video and a bitstream representation of the video. At step 3104, the process performs the conversion using the reference samples stored in the buffer, wherein the conversion of a video block of the video is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the video block and without reference to a reference picture.

[1062] Figure 32 32 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 34 in Section 4 of this document. At step 3202, the process performs a conversion between a current video block and a bitstream representation of the current video block. At step 3204, the process updates a buffer for storing reference samples for prediction in an intra block copy mode, wherein the buffer is used for a conversion between a subsequent video block and a bitstream representation of the subsequent video block, wherein the conversion between the subsequent video block and the bitstream representation of the subsequent video block is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the subsequent video block and does not reference a reference picture.

[1063] Figure 33is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 39 in Section 4 of this document. At step 3302, the process determines a buffer for storing reconstructed samples for prediction in an intra block copy mode for a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion is performed in the intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not reference a reference picture. At step 3304, the process applies a pre-processing operation to the reconstructed samples stored in the buffer in response to determining that the reconstructed samples stored in the buffer are to be used for predicting sample values ​​during the conversion.

[1064] Figure 34 is a flowchart of an example method for visual data processing. The steps of the flowchart are discussed in conjunction with Example Embodiment 42 in Section 4 of this document. At step 3402, the process selectively determines whether to use K1 previously processed VPDUs from even lines of the video region and / or K2 previously processed VPDUs from odd lines of the video region for a conversion between a current video block of a current virtual pipe data unit (VPDU) of a video region and a bitstream representation of the current video block. At step 3404, the process performs the conversion, wherein the conversion does not include using a remaining portion of the current VPDU, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the video block and without reference to a reference picture.

[1065] Some embodiments of this document are now presented in a clause-based format.

[1066] 1. A method for visual media processing, comprising:

[1067] For conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode;

[1068] performing the conversion using reference samples stored in a buffer, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture; and

[1069] For a pixel spatially located at position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, a corresponding reference (P mod M, Q mod N) in the buffer is calculated based on the reference position, where "mod" is a modulo operation and M and N are integers representing the x-dimension and y-dimension of the buffer, where the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0).

[1070] 2. The method of clause 1, wherein the reference position (P, Q) is determined as P = x0 + BVx and Q = y0 + BVy.

[1071] 3. A method for visual media processing, comprising:

[1072] For conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode;

[1073] performing the conversion using reference samples stored in a buffer, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture; and

[1074] For a pixel spatially located at position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, a corresponding reference in the buffer is calculated based on a reference position (P, Q), where the reference position (P, Q) is determined using the block vector (BVx, BVy) and position (x0, y0).

[1075] 4. The method of clause 3, wherein the reference position (P, Q) is determined as P=x0+BVx and Q=y0+BVy.

[1076] 5. The method according to any one or more of clauses 3-4, further comprising:

[1077] In the event that the reference position (P, Q) is outside the buffer, the corresponding reference in the buffer is calculated by clipping the reference position (P, Q) at the border of the buffer.

[1078] 6. The method according to any one or more of clauses 3-4, further comprising:

[1079] In the event that the reference position (P, Q) is outside the buffer zone, the corresponding reference in the buffer zone is calculated to be a predetermined value.

[1080] 7. The method of clause 6, wherein the predetermined value corresponds to an intermediate gray value.

[1081] 8. A method of visual media processing, comprising:

[1082] For conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode, wherein pixel locations within the buffer are addressed using an x-number and a y-number; and

[1083] The conversion is performed using reference samples stored in a buffer based on the x-number and the y-number, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

[1084] 9. The method according to clause 8, wherein the x-number and the y-number are varied as follows:

[1085] x=0, 1, 2, ..., M-1; y=0, 1, 2, ..., N-1, where the size of the buffer is expressed as M×N, where M and N are the pixel width and pixel height of the buffer.

[1086] 10. A method according to clause 8, wherein pixel positions within the buffer are addressed using a single number denoted as l such that l=y*M+x, l=0, 1, ..., M*N-1, where M and N are the pixel width and pixel height of the buffer.

[1087] 11. A method according to any one or more of clauses 8-10, wherein the current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0, y-y0) in the buffer, wherein (x0, y0) corresponds to the upper left corner position of the codec tree unit that includes the current video block.

[1088] 12. A method according to any one or more of clauses 8-10, wherein the current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0+Tx, y-y0+Ty) in the buffer, wherein (x0, y0) corresponds to the top left corner position of the codec tree unit including the current video block, and wherein Tx and Ty are offset values.

[1089] 13. The method of clause 12, wherein Tx and Ty are predefined offset values.

[1090] 14. A method according to any of clauses 1-13, wherein the converting comprises generating a bitstream representation from the current video block.

[1091] 15. A method according to any of clauses 1-13, wherein the converting comprises generating pixel values ​​of the current video block from a bitstream representation.

[1092] 16. A video encoder apparatus comprising a processor configured to implement the method according to any one or more of clauses 1-13.

[1093] 17. A video decoder device comprising a processor configured to implement the method according to any one or more of clauses 1-13.

[1094] 18. A computer-readable medium having stored thereon code embodying processor-executable instructions for implementing the method of any one or more of clauses 1-13.

[1095] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of a current video block may correspond to bits that are collocated or dispersed throughout the bitstream, as defined by a syntax. For example, a macroblock may be encoded based on transform and codec error residual values, and may also be encoded using bits in headers and other fields in the bitstream.

[1096] It will be understood from the foregoing that specific embodiments of the presently disclosed technology have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of the invention. Accordingly, the presently disclosed technology is not to be limited except as set forth in the accompanying claims.

[1097] The embodiments of the subject matter and functional operations described in this patent document can be implemented in various systems, digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium, which are used to be executed by a data processing device or to control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect machine-readable propagation signals, or a combination of one or more of them. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device can also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[1098] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages), and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or code portions). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[1099] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and the apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[1100] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to receive data from the one or more large-capacity storage devices or to transfer data to the one or more large-capacity storage devices, or to receive data from the one or more large-capacity storage devices and to transfer data thereto. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

[1101] It is intended that the specification and drawings be considered exemplary only, where exemplary means example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly dictates otherwise.

[1102] Although this patent document contains many details, these details should not be interpreted as limitations on any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable subcombination. Furthermore, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be directed to subcombinations or variations of subcombinations.

[1103] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown, or in sequential order, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1104] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for visual media processing, comprising: For conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode; performing the conversion using the reference samples stored in the buffer, wherein the conversion is performed in an intra block copy mode, wherein the intra block copy mode is based on motion information associated with a reconstructed block located in the same video region as the current video block and does not refer to a reference picture; and for a pixel spatially located at a position (x0, y0) and having a block vector (BVx, BVy) included in the motion information, calculating a corresponding reference (P mod M, Q mod N) in the buffer based on a reference position, where "mod" is a modulo operation and M and N are integers representing the x-dimension and y-dimension of the buffer, wherein the reference position (P, Q) is determined using the block vector (BVx, BVy) and the position (x0, y0), Here, the reference position (P, Q) is determined as P=x0+BVx and Q=y0+BVy.

2. The method according to claim 1, further comprising: In the event that a reference position (P, Q) is outside the buffer zone, the corresponding reference in the buffer zone is calculated by clipping the reference position (P, Q) at the border of the buffer zone.

3. The method according to claim 2, further comprising: In the event that a reference position (P, Q) is outside the buffer zone, the corresponding reference in the buffer zone is calculated as a predetermined value.

4. The method according to claim 3, wherein: The predetermined value corresponds to an intermediate gray value.

5. The method according to claim 1, further comprising: For a conversion between a current video block of visual media data and a bitstream representation of the current video block, determining a buffer storing reference samples for prediction in an intra block copy mode, wherein pixel locations in the buffer are addressed using x-numbers and y-numbers; and The conversion is performed using the reference samples stored in the buffer based on the x-number and the y-number, wherein the conversion is performed in an intra block copy mode based on motion information associated with a reconstructed block located in the same video region as the current video block and without reference to a reference picture.

6. The method according to claim 5, wherein: The x and y numbers vary as follows: x=0, 1, 2, ..., M-1; y=0, 1, 2, ..., N-1, where the size of the buffer is expressed as M×N, where M and N are the pixel width and pixel height of the buffer.

7. The method according to claim 5, wherein: The pixel positions within the buffer are addressed using a single number denoted as l such that l=y*M+x, l=0, 1, ..., M*N-1, where M and N are the pixel width and pixel height of the buffer.

8. The method according to any one of claims 5 to 7, wherein The current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0, y-y0) in the buffer, wherein (x0, y0) corresponds to the upper left corner position of the codec tree unit including the current video block.

9. The method according to any one of claims 5 to 7, wherein The current bitstream representation includes a block vector for the conversion, wherein the block vector represented as (BVx, BVy) is equal to (x-x0+Tx, y-y0+Ty) in the buffer, wherein (x0, y0) corresponds to the upper left corner position of the codec tree unit including the current video block, and wherein Tx and Ty are offset values.

10. The method according to claim 9, wherein: Tx and Ty are predefined offset values.

11. The method according to any one of claims 1 to 7, wherein The converting includes generating the bitstream representation from the current video block.

12. The method according to any one of claims 1 to 7, wherein The converting includes generating pixel values ​​for the current video block from the bitstream representation.

13. A video encoder apparatus comprising a processor configured to implement the method according to any one of claims 1-12.

14. A video decoder device comprising a processor configured to implement the method according to any one of claims 1-12.

15. A computer-readable medium having codes stored thereon, wherein when the codes are executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images