A video processing method, apparatus and medium

Abstract

Methods, systems, and devices are described that use intra block copy (IBC) with non-adjacent neighboring blocks. One example method of video processing includes, for a conversion between a video comprising a current video block and a bitstream of the video, determining, based on a coding tree unit (CTU) row comprising a non-adjacent neighboring block and the current video block, whether to use a block vector of the non-adjacent neighboring block to predict a block vector of the current video block, and performing the conversion based on the determination.

Description

[0001] Cross-reference to related applications

[0002] This application is based on International Patent Application No. PCT / CN2021 / 098764, filed on June 8, 2021, which claims priority and interest in International Patent Application No. PCT / CN2020 / 094930, filed on June 8, 2020. All of the aforementioned patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] This application relates to image and video encoding and decoding. Background Technology

[0004] Digital video accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document discloses an intra-block copy (IBC) technique that uses non-immediately adjacent blocks, which image and video encoders and decoders can use to perform image or video encoding, decoding, or processing.

[0006] In one example aspect, a video processing method is disclosed. The method includes: for a conversion between a video including a current video block and a bitstream of the video, determining, based on a codec tree unit (CTU) row including non-nearest neighbor blocks and the current video block, whether to use the block vector of the non-nearest neighbor blocks to predict the block vector of the current video block; and performing the conversion based on the determination.

[0007] In another example, a different video processing method is disclosed. The method includes: for a conversion between a video including a current video block and a bitstream of the video, determining, based on video units including non-nearest neighbor blocks and the current video block, whether to use the block vectors of the non-nearest neighbor blocks to predict the block vector of the current video block; and performing the conversion based on the determination.

[0008] In another example, a different video processing method is disclosed. This method includes: for a conversion between a video including a current video block and a bitstream of the video, determining the range of BV components in a one-dimensional block vector (BV) search based on properties of the current video block and a starting BV; and performing the conversion based on the determination.

[0009] In yet another example, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the methods described above.

[0010] In yet another example, a video decoder apparatus is disclosed. The video decoder includes components configured to implement the methods described above.

[0011] In yet another example, a computer-readable medium is disclosed on which code is stored. The code implements one of the methods described herein in the form of processor-executable code.

[0012] This document describes these and other features. Attached Figure Description

[0013] Figure 1 An example of intra-block copy (IBC) is shown.

[0014] Figure 2 An example of the location of the spatial merge candidate is shown.

[0015] Figure 3 An example of a non-immediately adjacent block is shown.

[0016] Figure 4 Another example of a non-immediately adjacent block is shown.

[0017] Figure 5 A block diagram of an exemplary video processing system that can implement various techniques of this disclosure is shown.

[0018] Figure 6 This is a block diagram of an exemplary hardware platform for video processing.

[0019] Figure 7 This is a block diagram illustrating an exemplary video codec system that can implement some embodiments of the present disclosure.

[0020] Figure 8 This is a block diagram illustrating an exemplary encoder that can implement some embodiments of the present disclosure.

[0021] Figure 9 This is a block diagram illustrating an exemplary decoder that can implement some embodiments of the present disclosure.

[0022] Figure 10-12 A flowchart of an exemplary video processing method is shown. Detailed Implementation

[0023] In this document, chapter headings are used for ease of understanding and not to limit the applicability of the techniques and embodiments disclosed in each chapter to that chapter. Furthermore, the use of H.266 terminology in some descriptions is merely for ease of understanding and not to limit the scope of the disclosed techniques. Therefore, the techniques described herein are also applicable to other video codec protocols and designs.

[0024] 1. Introduction

[0025] The techniques described in this document can be used to encode and decode visual media data, such as images or videos, which are generally referred to as video in this document. Specifically, they relate to intra-block copying in video encoding and decoding. They can be applied to existing video codec standards, such as HEVC or upcoming standards (Multi-Functional Video Codec, Audio Video Standard 3). They can also be used in future video codec standards or video codecs.

[0026] 2. Preliminary Discussion

[0027] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Visual. These two organizations jointly developed the H.264 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Coding (AVC) standards, as well as the H.265 / HEVC standard. Since H.262, video codec standards have been based on a hybrid video codec architecture, where temporal prediction plus transform coding has been used. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and applied them to reference software called the Joint Exploration Model (JEM). In April 2018, VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) established the Joint Video Expert Team (JVET) to work on the VVC standard, which reduces the bit rate by 50% compared to HEVC.

[0028] The latest version of the VVC draft, namely Multi-Functional Video Codec (Draft 9), can be found at the following URL:

[0029] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 18_Alpbach / wg11 / JVET-R2001-v10.zip

[0030] The latest reference software for VVC, called VTM, can be found at the following website:

[0031] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-9.0

[0032] 2.1 Derivation of merge candidates based on history

[0033] History-based MVP (HMVP) merge candidates are added to the merge table following the spatial MVP and TMVP. In this method, motion information from previous codec blocks is stored in the table and used as the MVP of the current CU. The table with multiple HMVP candidates is maintained during encoding / decoding. When a new CTU row is encountered, the table is reset (cleared). Whenever a non-sub-block intra-codec CU exists, the associated motion information is added as the last entry in the table as a new HMVP candidate.

[0034] The size S of the HMVP table is set to 6, indicating that a maximum of 6 history-based MVP (HMVP) candidates can be added to the table. When a new candidate is inserted into the table, a first-in-first-out (FIFO) rule is used, where a redundancy check is first applied to find if a duplicate HMVP exists in the table. If a duplicate HMVP is found, it is removed from the table, and all HMVP candidates are then moved forward.

[0035] HMVP candidates can be used in the merge candidate table construction process. The latest few HMVP candidates in the table are checked sequentially and inserted into the candidate table after the TMVP candidates. Redundancy checks are applied to HMVP candidates for spatial or temporal merge candidates.

[0036] To reduce the number of redundant check operations, the following simplifications are introduced:

[0037] 1) Is the number of HMPV candidates used to generate the merge table set to (N<=4)? M: (8-N), where N indicates the number of existing candidates in the merge table and M indicates the number of available HMVP candidates in the table.

[0038] 2) Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the process of building the merge candidate table from HMVP is terminated.

[0039] The HMVP concept has also been extended to block vector prediction in intra-block copy (mode).

[0040] 2.2 Intra-frame block copying

[0041] Intra-block copy (IBC), also known as current picture reference, has been adopted in HEVC Screen Content Coding extensions (HEVC-SCC) and the current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 1 As shown, when IBC is applied, the current block is predicted from a reference block in the same image. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. While IBC is inefficient for most camera-captured sequences, it exhibits significant encoding / decoding gains for screen content. This is because screen content images contain many repeating patterns, such as icons and text characters. IBC effectively eliminates redundancy between these repeating patterns. In HEVC-SCC, if the current image is selected as the reference image, the inter-frame coding unit (CU) can apply IBC. In this case, the MV is renamed to a block vector (BV), which always has integer pixel precision. For compatibility with the primary HEVC standard, the current image is marked as the "long-term" reference image in the Decoded Picture Buffer (DPB). It should be noted that, similarly, in multi-view / 3D video codec standards, inter-view reference images are also marked as "long-term" reference images.

[0042] After BV finds its reference block, predictions can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixel from the original signal. Then, transforms and quantization can be applied as in other codec modes.

[0043] However, when the reference block is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region of certain constraints, some or all pixel values ​​are undefined. Basically, there are two solutions to handle this problem. One is to prevent this situation, for example, in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following sections detail the solutions.

[0044] 2.3 IBC in HEVC Screen Content Codec Extension

[0045] In HEVC's screen content encoding and decoding extensions, when a block uses the current image as a reference, it should be ensured that the entire reference block is within the available reconstruction area, as shown in the following specification text:

[0046] The derivation of variables offsetX and offsetY is as follows:

[0047] offsetX=(ChromaArrayType==0)? 0:(mvCLX[0]&0x7?2:0) (8-104)

[0048] offsetY=(ChromaArrayType==0)? 0:(mvCLX[1]&0x7?2:0) (8-105)

[0049] The requirement for bitstream consistency is that, when the reference image is the current image, the luminance motion vector mvLX should adhere to the following constraints:

[0050] – When the derivation of the availability of the z-scan sequence block as specified in Clause 6.4.1 is invoked, where (xCurr, yCurr) is set to be equal to (xCb, yCb) and the adjacent brightness position (xNbY, yNbY) is set to be equal to (xPb+(mvLX[0]>>2)-offsetX, yPb+(mvLX[1]>>2)-offsetY) as input, the output should be equal to TRUE.

[0051] – When the derivation of the availability of the z-scan sequence block as specified in Clause 6.4.1 is invoked, where (xCurr, yCurr) is set to be equal to (xCb, yCb) and the adjacent brightness position (xNbY, yNbY) is set to be equal to (xPb+(mvLX[0]>>2)+nPbW-1+offsetX, yPb+(mvLX[1]>>2)+nPbH-1+offsetY) as input, the output should be equal to TRUE.

[0052] – One or both of the following conditions should be true:

[0053] The value of –(mvLX[0]>>2)+nPbW+xB1+offsetX is less than or equal to 0.

[0054] The value of –(mvLX[1]>>2)+nPbH+yB1+offsetY is less than or equal to 0.

[0055] –The following conditions should be true:

[0056] (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCurr / CtbSizeY<=

[0057] yCurr / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY (8-106)

[0058] Therefore, there will be no situation where the reference block overlaps with the current block or the reference block is outside the image. There is no need to fill in the reference block or the prediction block.

[0059] 2.4 IBC in the VVC test model

[0060] In the current VVC test model (i.e., the VTM-4.0 design), the entire reference block should be consistent with the current coding tree unit (CTU) and should not overlap with the current block. Therefore, there is no need to pad the reference block or prediction block. The IBC flag is encoded and decoded into the prediction mode of the current CU. Therefore, each CU has a total of three prediction modes: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0061] 2.4.1 IBC Merge Mode

[0062] In IBC merge mode, the indexes pointing to entries in the IBC merge candidate table are parsed from the bitstream. The construction of the IBC merge table can be summarized as follows:

[0063] Step 1: Derive the airspace candidates.

[0064] Step 2: Insert History-based Block Vector Prediction (HBVP) candidates.

[0065] Step 3: Insert pairwise average candidates.

[0066] In the derivation of spatial merge candidates, from the location located Figure 2 At most four merge candidates are selected from the candidates at the indicated positions. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, and A0 is unavailable (e.g., because B2 belongs to another stripe or slice) or if IBC mode encoding / decoding is not used. After adding the candidate at position A1, a redundancy check is performed on the insertion of the remaining candidates to ensure that candidates with the same motion information in the table are excluded, thereby improving encoding / decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only those are considered. Figure 2 The pairs connected by arrows are added to the table only if the corresponding candidates used for redundancy checks do not have the same motion information.

[0067] After inserting a spatial candidate, if the size of the IBC merge table is still smaller than the size of the maximum IBC merge table, an IBC candidate from the HBVP table can be inserted. Redundancy checks need to be performed when inserting an HBVP candidate.

[0068] Finally, the paired average candidates are inserted into the IBC merge table.

[0069] A merge candidate is called an invalid merge candidate when the reference block identified by the merge candidate is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region restricted by certain constraints.

[0070] It should be noted that invalid merge candidates can be inserted into the IBC merge table.

[0071] 2.4.2 IBC AMVP Mode

[0072] In the Advanced Motion Vector Prediction (AMVP) mode of IBC, the AMVP index pointing to an entry in the IBC AMVP table is parsed from the bitstream. The construction of the IBC AMVP table can be summarized as follows:

[0073] Step 1: Derive the airspace candidates.

[0074] Check A0 and A1 until a usable candidate is found.

[0075] o Check B0, B1, B2 until a usable candidate is found.

[0076] Step 2: Insert HBVP candidate.

[0077] Step 3: Insert zero candidates.

[0078] After inserting a spatial candidate, if the size of the IBC AMVP table is still smaller than the size of the largest IBC AMVP table, then an IBC candidate from the HBVP table can be inserted.

[0079] Finally, the zero candidate is inserted into the IBC AMVP table.

[0080] 2.5 IBC AMVP Mode in AVS3

[0081] In Audio Video Coding Standard 3 (AVS3), an HBVP table is maintained to store the BVs of previously encoded blocks. For each entry in the HBVP table, in addition to the BV, information about the block associated with the BV is stored, including the block's width and height, and the coordinates of the block's top-left sample (relative to the top-left sample of the image). A counter indicating how many times the BV has been encountered is also stored in the entry. In the following text, the coordinates of the block's top-left sample are also used as the block's coordinates.

[0082] In IBC AMVP mode, when building the IBC AMVP (Advanced Motion Vector Prediction) table for the current block, the BVs in the HBVP table are first checked sequentially and divided into 7 classes. Each class can contain at most one BV, and if multiple BVs are classified into the same class, the most recently checked BV is used for that class.

[0083] • For BV, if the size (e.g., width * height) of the block associated with the BV is greater than or equal to 64, it is classified as class 0.

[0084] • For BV, if its counter is greater than or equal to 3, it is classified into the first category.

[0085] • For BV, it is also classified in the following order:

[0086] If its horizontal coordinate is less than the horizontal coordinate of the current block, and its vertical coordinate is less than the vertical coordinate of the current block, then it is classified into the fourth category, such as the top left category.

[0087] Otherwise, if its horizontal coordinate is greater than or equal to the horizontal coordinate of the current block plus the width of the current block, it is classified into the fifth category, such as the top right category.

[0088] Otherwise, if its vertical coordinate is greater than or equal to the vertical coordinate of the current block plus the height of the current block, it is classified into the sixth category, such as the bottom left category.

[0089] Otherwise, if its vertical coordinate is less than the vertical coordinate of the current block, it is classified into the third category, such as the above category.

[0090] Otherwise, if its horizontal coordinate is less than the horizontal coordinate of the current block, it is classified into the second category, such as the left category.

[0091] Second, classes 0-6 (BVs) are inserted into the AMVP table in sequence. If a class is not empty, the corresponding BV can be added to the AMVP table after pruning with the already inserted AMVP candidates.

[0092] In the BV estimation process, an initial BV is first determined. Then, a one-dimensional vertical BV search, a one-dimensional horizontal BV search, and a two-dimensional BV search are successfully performed to find the optimal BV. Each BV search stage starts with the same initial BV. In the one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y–H. Similarly, in the one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x–W.

[0093] 3. The technical problem solved by the disclosed technical solution

[0094] 1. It is inefficient not to use the block vector (BV) of non-immediately adjacent blocks when constructing IBC merge tables or AMVP tables.

[0095] 2. In AVS3, when classifying BVs in the HBVP table, the block size associated with the BV (e.g., block width * height) is compared with a fixed value (e.g., 64) to determine whether the BV should be classified as class 0, regardless of the current block size, which may be unreasonable.

[0096] 3. In AVS3, very strict constraints are applied to the vertical and horizontal BV components during the one-dimensional BV search phase, which is inefficient.

[0097] 4. Example Solutions and Implementation Examples

[0098] The following items are intended as examples to illustrate general concepts. These items should not be interpreted narrowly. Furthermore, these items can be combined in any way.

[0099] Represent the coordinates of the current block (e.g., the coordinates of the top-left sample point of the block) as (x, y), and the width and height of the current block as W and H, respectively. Represent the coordinates of the non-immediately adjacent samples as (x – deltaX, y – deltaY), where deltaX and deltaY are positive integers, negative integers, or 0. The non-immediately adjacent block is the S1*S2 block (S1 and S2 are integers, e.g., S1 = S2 = 4) covering the samples. Assume that the current CTU row containing the current block starts at coordinates (0, ctuRowY).

[0100] 1. Propose that when predicting the BV of the current block, the BV of its non-immediately adjacent blocks can be used.

[0101] a) It is proposed that the BVs of non-immediately adjacent blocks can be inserted into the IBC merge table and / or the IBC AMVP table (e.g., the block vector prediction table).

[0102] b. In one example, the position of a non-immediately adjacent block can depend on the width and / or height of the current block.

[0103] i. For example, when constructing the IBC merge table and / or the IBC AMVP table, it is possible to examine non-nearest neighbor blocks covering positions (x–M, y–M), (x–M, y+H / 2), (x–M, y+H), (x+W / 2, y–M), and (x+W, y–M), where M is an integer, such as Figure 3 As shown. For example, M = 8.

[0104] 1. Alternatively, when constructing the IBC merge table and / or the IBC AMVP table, non-immediately adjacent blocks covering positions (x–M,y–M), (x–M,y+H–1), (x–M,y+H), (x+W–1,y–M), and (x+W,y–M) can be checked.

[0105] 2. Alternatively, when constructing the IBC merge table and / or the IBC AMVP table, non-immediately adjacent blocks covering positions (x–M,y), (x,y–M), (x–M,y+3*H / 2), (x–M,y+2*H), (x+3*W / 2,y–M), and (x+2*W,y–M) can be checked.

[0106] ii. For example, when constructing the IBC merge table and / or the IBC AMVP table, it is possible to examine non-nearest neighbor blocks covering positions (x–M–1, y–M–1), (x–M–1, y–M–1+(H+M) / 2), (x–M–1, y+H), (x–M–1+(W+M) / 2, y–M–1), and (x+W, y–M–1), where M is an integer, such as Figure 4 As shown. For example, M = 8.

[0107] 1. Alternatively, when constructing the IBC merge table and / or the IBC AMVP table, you can check for non-immediately adjacent blocks covering positions (x–M–1,y–M–1), (x–M–1,y+H–1), (x–M–1,y+H), (x+W–1,y–M–1), and (x+W,y–M–1).

[0108] 2. Alternatively, when constructing the IBC merge table and / or the IBC AMVP table, non-immediately adjacent blocks covering positions (x–M–1,y), (x,y–M–1), (x–M–1,y–M–1+3*(H+M) / 2), (x–M–1,y+2*H+M), (x–M–1+3*(W+M) / 2,y–M–1), and (x+2*W+M,y–M–1) can be checked.

[0109] c. In one example, how many non-immediately adjacent blocks to check can depend on the shape or size of the current block.

[0110] d. In one example, how many non-immediately adjacent blocks can be checked depends on the coordinates of the current block.

[0111] e. In one example, when the non-immediately adjacent block and the current block are in two different CTU lines, the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

[0112] f. In one example, when the non-immediately adjacent block and the current block are in two different CTU rows, and the difference between the vertical coordinate of the current block and the vertical coordinate of the current CTU row (e.g., ctuRowY) is less than or equal to Th, the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

[0113] i. For example, Th equals 0. In this case, the current block is located at the top of the current CTU line.

[0114] ii. For example, Th equals 4.

[0115] iii. For example, Th equals M–4, where M is defined above to indicate the location of non-immediately adjacent blocks.

[0116] g. In one example, when the non-immediately adjacent block and the current block are in two different CTU lines, the position of the non-immediately adjacent block can be clipped to be in the same CTU line as the current block, and the BV at the clipped position can be used to predict the BV of the current block.

[0117] h. In one example, when the non-immediately adjacent block and the current block are in two different CTU rows, the vertical position of the non-immediately adjacent block can be clipped to within the distance of the vertical coordinate of the current CTU row, and the BV at the clipped position can be used to predict the BV of the current block.

[0118] i. For example, the vertical position of a non-immediately adjacent block can be clipped to ctuRowY–Th2, where Th2 is an integer. For example, Th2 equals 0, 4, or 8.

[0119] i. In one example, when the non-immediately adjacent block and the current block are in two different slices / strips / sub-pictures, the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

[0120] i. Alternatively, the location of a non-immediately adjacent block can be cropped to be within the same slice / strip / sub-image as the current block, and the BV at the cropped location can be used to predict the BV of the current block.

[0121] 2. It was proposed that the inspection order of non-immediately adjacent blocks can depend on the relative position of the adjacent blocks with respect to the current block.

[0122] a. In one example, the order of checking non-immediately adjacent blocks can be as follows: the top left adjacent block, the top right adjacent block, the bottom left adjacent block, the top adjacent block, and the left adjacent block of the current block.

[0123] i. For example, check the non-immediately adjacent blocks covering the positions (x–M,y–M), (x–M,y–M), (x–M,y–H), (x–W / 2,y–M), (x–M,y–H / 2) in the order of (x–M,y–M), (x–M,y–H / 2), (x–M,y–H), (x–W / 2,y–M), (x+W,y–M).

[0124] ii. For example, check the non-immediately adjacent blocks of the locations (x–M–1,y–M–1), (x+W,y–M–1), (x–M–1,y+H), (x–M+(W+M) / 2,y–M–1), (x–M–1,y–M+(H+M) / 2) in the order of (x–M–1,y–M–1), (x–M–1,y–M+(H+M) / 2), (x–M–1,y+H), (x–M+(W+M) / 2,y–M–1), and (x+W,y–M–1).

[0125] b. In one example, the order of checking non-immediately adjacent blocks can be as follows: the left neighbor of the current block, the top neighbor, the top-left neighbor, the top-right neighbor, and the bottom-left neighbor.

[0126] i. For example, check the non-immediately adjacent blocks covering the positions (x–M,y–M), (x–M,y+H / 2), (x–M,y+H), (x–W / 2,y–M), (x–M,y–M), (x–W,y–M), (x–M,y+H) in the order of (x–M,y–M), (x–M,y+H / 2), (x–M,y+H), (x–W / 2,y–M), (x+W,y–M).

[0127] ii. For example, check the non-immediately adjacent blocks of the locations (x–M–1,y–M+(H+M) / 2), (x–M+(W+M) / 2,y–M–1), (x–M–1,y–M–1), (x+W,y–M–1), (x–M–1,y+H) in the order of (x–M–1,y–M–1), (x–M–1,y–M+(H+M) / 2), (x–M–1,y+H), (x–M+(W+M) / 2,y–M–1), (x+W,y–M–1).

[0128] c. In one example, the order of checking non-immediately adjacent blocks can be as follows: the left neighbor of the current block, the top neighbor, the top right neighbor, the bottom left neighbor, and the top left neighbor.

[0129] d. In one example, the order of checking non-immediately adjacent blocks can be as follows: the current block's bottom left neighbor, left neighbor, top right neighbor, top neighbor, and top left neighbor.

[0130] e. In one example, the order of checking non-immediately adjacent blocks can be as follows: the top-left adjacent block of the current block, the left adjacent block, the top adjacent block, the top-right adjacent block, and the bottom-left adjacent block.

[0131] f. In one example, the order of checking non-immediately adjacent blocks can be as follows: the top-left adjacent block of the current block, the top adjacent block, the left adjacent block, the top-right adjacent block, and the bottom-left adjacent block.

[0132] g. In one example, the order of checking non-immediately adjacent blocks can be as follows: the current block's upper neighbor, left neighbor, upper-left neighbor, upper-right neighbor, and lower-left neighbor.

[0133] h. In one example, non-immediately adjacent blocks can be divided into multiple groups, and candidates in each group are checked in a predefined order. Up to N (N is an integer, e.g., N=1) candidates from a group can be inserted into the IBC merge table and / or the IBC AMVP table.

[0134] i. For example, non-immediately adjacent blocks can be divided into two groups: {bottom left, left} - adjacent blocks, {top right, top, top left} - adjacent blocks.

[0135] ii. For example, non-immediately adjacent blocks can be divided into two groups: {bottom left, left, top left} - adjacent blocks, and {top right, top} - adjacent blocks.

[0136] i. In one example, the order in which non-immediately adjacent blocks are checked can depend on the distance from the adjacent block to the current block.

[0137] i. For example, the distance can be defined as the distance from the top-left sample point of the neighboring block to the top-left sample point of the current block.

[0138] 1. This distance can be defined as the sum of the horizontal and vertical distances from the top-left sample point of the neighboring block to the top-left sample point of the current block.

[0139] 2. This distance can be defined as the sum of the squared horizontal and vertical distances from the top-left sample point of the neighboring block to the top-left sample point of the current block.

[0140] ii. For example, non-immediately adjacent blocks can be checked in ascending order of distance.

[0141] iii. For example, non-immediately adjacent blocks can be checked in descending order of distance.

[0142] j. In one example, the order in which non-immediately adjacent blocks are checked can depend on the size or shape of the current block.

[0143] i. For example, for a block with W > M1*H (e.g., M1 = 2), the top neighbor block, top right neighbor block, and top left neighbor block can be given higher priority than the bottom left neighbor block and left neighbor block.

[0144] ii. For example, for a block with W > M1*H (e.g., M1 = 2), the top neighbor block, top right neighbor block, and top left neighbor block can be given a lower priority than the bottom left neighbor block and left neighbor block.

[0145] iii. For example, for a block with H > M1*W (e.g., M1 = 2), the top neighbor block, top right neighbor block, and top left neighbor block can be given higher priority than the bottom left neighbor block and left neighbor block.

[0146] iv. For example, for a block with H > M1*W (e.g., M1 = 2), the top neighbor block, top right neighbor block, and top left neighbor block can be given a lower priority than the bottom left neighbor block and left neighbor block.

[0147] k. In one example, the order in which non-immediately adjacent blocks are checked can depend on the size of the adjacent blocks.

[0148] i. For example, non-immediately adjacent blocks can be checked in ascending order of size (width * height).

[0149] ii. For example, non-immediately adjacent blocks can be checked in descending order of size (width * height).

[0150] 3. The suggestion to insert BVs of non-immediately adjacent blocks into the IBC merge table and / or IBC AMVP table may depend on the availability of BVs from the HBVP table and / or the availability of BVs of immediately adjacent blocks.

[0151] a. In one example, insert the BV of a non-immediately adjacent block after the BV from the HBVP table.

[0152] i. Alternatively, insert the BV of the non-immediately adjacent block before the BV from the HBVP table.

[0153] ii. Alternatively, the BVs of non-immediately adjacent blocks are alternated with the BVs from the HBVP table.

[0154] b. In one example, insert a BV from a non-immediately adjacent block after the BV of the immediately adjacent block.

[0155] i. Alternatively, insert a BV that is not immediately adjacent to the adjacent block before the BV of the immediately adjacent block.

[0156] ii. Alternatively, the BVs of non-immediately adjacent blocks are alternated with the BVs of immediately adjacent blocks.

[0157] c. In one example, after inserting a BV from the HBVP table and / or a BV from an immediately adjacent block, a BV from a non-immediately adjacent block is not inserted if there are no empty entries in the IBCmerge / AMVP table.

[0158] d. In one example, the BVs of non-immediately adjacent blocks can be divided into multiple classes in a manner similar to the BVs from the HBVP table.

[0159] i. For example, non-immediately adjacent blocks can be classified into five classes based on their relative positions to the current block: top-left, top-right, bottom-left, top, and left. One or more non-immediately adjacent blocks can be classified into one class.

[0160] ii. In one example, when the HBVP table does not contain any available BVs in the first category, a BV belonging to a non-immediately adjacent block that is in the first category (if available) can be used instead.

[0161] 1. In one example, the BVs of one or more non-immediately adjacent blocks belonging to the first category can be checked in a predefined order until a usable BV is found or all BVs have been checked.

[0162] 2. BVs belonging to one or more non-immediately adjacent blocks in the first category can be checked in a predefined order until the BVs in the first category are inserted into the IBC merge / AMVP table or all BVs have been checked.

[0163] iii. In one example, when there is a available BV that is both from the HBVP table and belongs to the first category of non-immediately adjacent blocks, which BV to use may depend on the distance from the block associated with the BV to the current block (similar to what is defined in item 2.e).

[0164] 1. For example, you can check BV in descending order of distance until you find a usable BV or check all BVs.

[0165] 2. For example, BV can be checked in descending order of distance until the BV is inserted into the IBC merge / AMVP table or all BVs have been checked.

[0166] 3. For example, you can check BV in ascending order of distance until you find a usable BV or check all BVs.

[0167] 4. For example, BV can be checked in ascending order of distance until the BV is inserted into the IBC merge / AMVP table or all BVs have been checked.

[0168] e. In one example, when the HBVP table does not contain any available BVs in the first category (e.g., the first category could be one of categories 0, 1, 2, 3, 4, 5, or 6), the BVs of non-immediately adjacent blocks can be used for the first category.

[0169] i. In one example, when the HBVP table does not contain any available BVs in the first category, the BVs of the first group of non-immediately adjacent blocks can be checked sequentially until an available BV is found, or all BVs can be checked.

[0170] ii. In one example, when the HBVP table does not contain any available BVs in the second category, the BVs of the second group of non-immediately adjacent blocks can be checked sequentially until an available BV is found. The first group of non-immediately adjacent blocks can be different from the second group of non-immediately adjacent blocks when the first category differs from the second category.

[0171] 1. Alternatively, the first group of non-immediately adjacent blocks can be the same as the second group of non-immediately adjacent blocks.

[0172] iii. In one example, if a non-immediately adjacent block belongs to the first non-immediately adjacent block group, it may not belong to the second non-immediately adjacent block group, which is different from the first non-immediately adjacent block group.

[0173] iv. In one example, when the BV of the first non-immediately adjacent block is used for the first category, the BV may not need to be checked again for the second category.

[0174] 1. Alternatively, when checking the BV of the first non-immediately adjacent block for the first category, the BV may not need to be checked again for the second category.

[0175] f. In one example, before inserting a BV from a non-immediately adjacent block, the BV can be compared with one or more BVs already inserted in the IBCmerge / AMVP table.

[0176] i. In one example, if a BV from a non-immediately adjacent block is the same as one of one or more BVs already inserted into the IBC merge / AMVP table, then it is not inserted into the IBC merge / AMVP.

[0177] ii. In one example, if a BV from a non-immediately adjacent block is similar to one of one or more BVs already inserted into the IBC merge / AMVP table, it is not inserted into the IBC merge / AMVP.

[0178] iii. In one example, this comparison can be made between one or more BVs from non-immediately adjacent blocks.

[0179] iv. Or, without comparison.

[0180] 4. It is proposed that the classification of BVs from the HBVP table into the Nth class (N is a non-negative integer, e.g., N=0) can be determined based on the block size associated with the BV (denoted as BvBlkSize) and the size of the current block (denoted as CurBlkSize).

[0181] a. In one example, BV can be classified into class N when BvBlkSize is greater than or equal to the factor * CurBlkSize, where the factor is a positive number. For example, the factor is equal to 1.

[0182] b. In one example, when BvBlkSize is greater than the factor * CurBlkSize, BV can be classified into class N, where the factor is a positive number. For example, the factor is equal to 1.

[0183] c. In one example, BV can be classified into class N when BvBlkSize is less than or equal to factor * CurBlkSize, where the factor is a positive number. For example, the factor is equal to 1.

[0184] d. In one example, when BvBlkSize is less than the factor * CurBlkSize, BV can be classified into class N, where the factor is a positive number. For example, the factor is equal to 1.

[0185] e. In one example, when BvBlkSize equals factor * CurBlkSize, BV can be classified into class N, where the factor is a positive number. For example, the factor equals 1.

[0186] f. Alternatively, it can be determined whether a BV from the HBVP table should be classified as Class N based on the block size associated with the BV and the size of the current block.

[0187] 5. It is proposed that the range of BV components in a one-dimensional BV search can depend only on the coordinates of the current block.

[0188] a. Alternatively, in a one-dimensional BV search, the range of the BV components may not depend on the size of the current block.

[0189] b. In one example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y – N1 (N1 is an integer, for example, N1 = 0, 8 or -8).

[0190] c. In one example, in a one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x – N2 (N2 is an integer, such as N2 = 0, 8 or -8).

[0191] d. Alternatively, in a one-dimensional BV search, the range of the BV components can depend on both the size and coordinates of the current block.

[0192] i. For example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y+H–N1 (N1 is an integer, for example, N1 = 0, 8 or -8).

[0193] ii. For example, in a one-dimensional horizontal BV search, the horizontal BV components are constrained to be less than or equal to x + W – N2 (N2 is an integer, such as N2 = 0, 8 or -8).

[0194] iii. Alternatively, the range of the BV components may further depend on the starting BV (startBvX, startBvY).

[0195] 1. For example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y + startBvY + K1*H – N1. N1 is an integer, such as N1 = 0, 8, or -8, and K1 is an integer, such as K1 = 2, 3, or 4.

[0196] 2. For example, in a one-dimensional horizontal BV search, the horizontal BV components are constrained to be less than or equal to x + startBvX + K2*W – N2. N2 is an integer, such as N2 = 0, 8, or -8, and K2 is an integer, such as K2 = 2, 3, or 4.

[0197] 5. Examples

[0198] The following are some example embodiments of the inventions summarized above in this section, to which the VVC specification can be applied. Most of the relevant parts that have been added or modified are indicated by bold, underline, and italics, such as "Using A...". Some deleted parts are indicated by italics with strikethrough, such as "based on". "B". There are also other changes that are themselves editable, so they are not highlighted.

[0199] 5.1 Example #1

[0200] Section 9.5.6.3.2 Block Vector Prediction

[0201] First, construct the class blockMotionClassY (Y = 0~6) according to the following steps:

[0202] a) Initialize the number of candidate block copy intra-predictive historical motion information NumAllowedHbvpCand to Min(CntHbvp,NumOfHbvpCand).

[0203] b) If NumAllowedHbvpCand equals 0, and both MvPredXBv and MvPredYBv equal 0, then the export process ends; otherwise, continue with the following steps.

[0204] c) Otherwise, for Y ranging from 0 to 6, initialize the number of intra-frame prediction motion information candidates cntClassY for block replication in each class blockMotionClassY to 0. For X = 0 to NumAllowedHbvpCand-1, HbvpCandateList[X] is categorized as follows:

[0205] 1) If widthCandX * heightCandX is greater than or equal to Then add HbvpCandateList[X] to blockMotionClass0 and increment the value of cntClassY by 1;

[0206] 2) If cntCandX is greater than or equal to 3, add HbvpCandidateList[X] to blockMotionClass1 and increment the value of cntClassY by 1;

[0207] 3) If xCandX is less than xCur and yCandX is less than yCur, then add HbvpCandidateList[X] to blockMotionClass4 and increment the value of cntClassY by 1;

[0208] 4) Otherwise, if xCandX is greater than or equal to xCur+widthCur, add HbvpCandidateList[X] to blockMotionClass5 and increment the value of cntClassY by 1;

[0209] 5) Otherwise, if yCandX is greater than or equal to yCur+heightCur, add HbvpCandidateList[X] to blockMotionClass6 and increment the value of cntClassY by 1;

[0210] 6) Otherwise, if yCandX is less than yCur, add HbvpCandidateList[X] to blockMotionClass3 and increment the value of cntClassY by 1;

[0211] 7) Otherwise, add HbvpCandidateList[X] to blockMotionClass2 and increment the value of cntClassY by 1.

[0212] Then, the block vectors containing intra-frame copy motion information in blockMotionClassY are labeled as bvClassY, and the candidate class table CbvpCanddateList is derived according to the following method:

[0213] a) Initialize cntCbvp, cntCheck, and Y to 0;

[0214] b) If the value of cntClassY is greater than 0, perform the following steps:

[0215] 1) Initialize candIdx to 0;

[0216] 2) If cntCbvp equals 0, then the block vector of CbvpCanddateList[cntCbvp] equals bvClassY, increment cntCbwp by 1, and execute step c);

[0217] 3) Otherwise, if the block vector of CbvpCandidateList[candIdx] is the same as bvClassY, then proceed to step c);

[0218] 4) Otherwise, increment candIdx by 1. If candIdx is less than cntCheck, proceed to step 3).

[0219] 5) Otherwise, the block vector of CbvpCanddateList[cntCbvp] is equal to bvClassY, and cntCbvp is incremented by 1.

[0220] c) Increment the value of Y by 1, and then perform the following steps:

[0221] 1) If the value of Y is less than or equal to 2, set the value of cntCheck to cntCbvp and then continue to step b);

[0222] 2) Otherwise, if Y is less than 7, then continue with step b);

[0223] 3) Otherwise, end. The process of deriving CbvpCanddateList.

[0224]

[0225]

[0226] If cntCbvp equals 0, then both MvPredXBv and MvPredYBv equal 0. Otherwise, MvPredXBv and MvPredYBv equal the x and y coordinates of CbvpCandidateList[CbvpIndex], respectively.

[0227] Figure 5 A block diagram of an example video processing system 5000 that can implement various technologies of this disclosure is shown. Various implementations may include some or all of the components of system 5000. System 5000 may include an input 5002 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 5002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0228] System 5000 may include an encoding / decoding component 5004, which may implement various encoding / decoding or coding methods described in this disclosure. Encoding / decoding component 5004 may reduce the average bit rate of the video from input 5002 to the output of encoding / decoding component 5004 to produce an encoded / decoded representation of the video. Therefore, encoding / decoding techniques are sometimes referred to as video compression or video transcoding techniques. The output of encoding / decoding component 5004 may be stored or transmitted via a communication connection as represented by component 5006. The stored or communicated bitstream (or encoded / decoded) representation of the video received at input 5002 may be used by component 5008, which generates pixel values ​​or displayable video to be sent to display interface 5010. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “encoding / decoding” operations or tools, it should be understood that encoding / decoding tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the encoded result will be performed by the decoder.

[0229] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The techniques described in this disclosure can be implemented in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0230] Figure 6 This is a block diagram of a video processing apparatus 6000. Apparatus 6000 can be used to implement one or more methods described in this disclosure. Apparatus 6000 can be located in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 6000 may include one or more processors 6002, one or more memories 6004, and video processing hardware 6006. The processors (multiple) 6002 can be configured to implement one or more methods described in this disclosure (e.g., in...). Figure 10-11 (In the middle). Multiple (multiple) memories 6004 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 6006 may be used in hardware circuitry to implement some of the techniques described herein. In some embodiments, hardware 6006 may be partially or wholly located within one or more processors 6002, such as a graphics processor.

[0231] Figure 7 This is a block diagram illustrating an example video codec system 100 that can utilize the technology disclosed herein.

[0232] like Figure 7 As shown, the video encoding / decoding system 100 may include a source device 110 and a target device 120. The source device 110 generates encoded video data and may be referred to as a video encoding device. The target device 120 decodes the encoded video data generated by the source device 110 and may be referred to as a video decoding device. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0233] Video source 112 may include, for example, a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. Video data may include one or more images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include bit sequences forming a codec representation of the video data. The bitstream may include codec images and associated data. A codec image is a codec representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data may be transmitted directly to target device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by target device 120.

[0234] The target device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0235] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with target device 120, or may be external to target device 120, which is configured to interface with an external display device.

[0236] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Coding (HEVC) standard, the Multi-Function Video Coding (VVC) standard, and other current and / or other standards.

[0237] Figure 8 This is a block diagram illustrating an example of a video encoder 200. The video encoder 200 can be... Figure 7 The video encoder 114 in the system 100 described herein.

[0238] The video encoder 200 can be configured to perform any or all of the technologies disclosed herein. Figure 8 In the example shown, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0239] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203), a motion estimation unit 204, a motion compensation unit 205, an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding / decoding unit 214.

[0240] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture containing the current video block.

[0241] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, can be highly integrated, but for descriptive purposes... Figure 8 The examples are shown separately.

[0242] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0243] The mode selection unit 203 can select one of the encoding / decoding modes (intra-frame or inter-frame) based, for example, on the error result, and provide the resulting intra-frame or inter-frame codec block to the residual generation unit 207 to generate residual block data, and the reconstruction unit 212 reconstructs the codec block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on the inter-frame prediction signal and the intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 can also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision).

[0244] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on motion information and decoded samples from images other than those associated with the current video block from buffer 213.

[0245] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-band, P-band, or B-band.

[0246] In some examples, motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search for reference images in table 0 or 1 of the reference video blocks for the current video block. Motion estimation unit 204 can then generate a reference index indicating that the reference images of the reference video blocks contained in table 0 or 1, and a motion vector indicating the spatial displacement between the current video block and the reference video blocks. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current block based on the reference video blocks indicated by the motion information of the current video block.

[0247] In other examples, motion estimation unit 204 can perform bidirectional prediction on the current video block. Motion estimation unit 204 can search for reference images in table 0 of the reference video blocks of the current video block, and can also search for reference images in table 1 of the reference video blocks of the current video block. Motion estimation unit 204 can then generate reference indices indicating the reference images in tables 0 and 1, which contain reference video blocks and motion vectors indicating the spatial displacement between the reference video blocks and the current video block. Motion estimation unit 204 can output the reference index and motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[0248] In some examples, the motion estimation unit 204 can output a complete set of motion information for the decoder to use in the decoding process.

[0249] In some examples, motion estimation unit 204 may not output the complete set of motion information for the current video. Instead, motion estimation unit 204 may signal the motion information of the current video block to another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0250] In one example, the motion estimation unit 204 may instruct the video decoder 300 in the syntax structure associated with the current video block to indicate that the current video block has the same motion information as another video block.

[0251] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) within the syntactic structure associated with the current video block. The motion vector difference represents the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0252] As described above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge pattern signaling notification.

[0253] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0254] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a negative sign) multiple predicted video blocks from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0255] In other examples, for the current video block, such as in skip mode, residual data for the current video block may not exist, and residual generation unit 207 may not perform subtraction operations.

[0256] The transform processing unit 208 can generate one or more transform coefficient video blocks of the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0257] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0258] Inverse quantization unit 210 and inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct the residual video block from the transform coefficient video block. Reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples from one or more predicted video blocks generated by prediction unit 202 to produce a reconstructed video block associated with the current block stored in buffer 213.

[0259] After the video block is reconstructed by the reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0260] The entropy encoding / decoding unit 214 can receive data from other functional components of the video encoder 200. When the entropy encoding / decoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0261] Figure 9 This is a block diagram illustrating an example of a video decoder 300, which can be... Figure 7 The video decoder 114 in the system 100 shown.

[0262] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 9In the example shown, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0263] In such Figure 9 The example shown includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform functions typically associated with the video encoder 200. Figure 8 The encoding channel is the opposite of the decoding channel described.

[0264] The entropy decoding unit 301 can obtain the encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy-coded video data, and based on the entropy-decoded video data, the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference picture table indexes, and other motion information. For example, the motion compensation unit 302 can determine such information by executing AMVP and merge modes.

[0265] The motion compensation unit 302 can generate motion compensation blocks and can perform interpolation based on an interpolation filter. Identifiers for the interpolation filter with sub-pixel precision can be included in the syntax elements.

[0266] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during the encoding of the video block to calculate the interpolation of sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information and use the interpolation filter to generate the prediction block.

[0267] The motion compensation unit 302 can use some syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, mode indicating how each segment is encoded, one or more reference frames (and a reference frame table) for each inter-frame codec block, and other information for decoding the encoded video sequence.

[0268] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 303 performs inverse quantization, i.e., dequantization, on the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0269] The reconstruction unit 306 can add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If necessary, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates the decoded video for presentation on the display device.

[0270] Figure 10-12 It shows that it can be done in, for example Figure 5-9 The embodiments shown are example methods for implementing the above technical solutions.

[0271] Figure 10 A flowchart of an example method 1000 for video processing is shown. Method 1000 includes, in operation 1010, for a conversion between a video and a bitstream of the video including the current video block, determining whether to use the block vector of the non-nearest neighbor block to predict the block vector of the current video block based on the codec tree unit (CTU) row including the non-nearest neighbor block and the current video block.

[0272] Method 1000 includes, in operation 1020, performing a transformation based on a determination.

[0273] Figure 11 A flowchart of an example method 1100 for video processing is shown. Method 1100 includes, in operation 1110, for a conversion between a video and a video bitstream including the current video block, determining whether to use the block vector of the non-nearest neighbor block to predict the block vector of the current video block based on the video units including the non-nearest neighbor blocks and the current video block.

[0274] Method 1100 includes, in operation 1120, performing a transformation based on a determination.

[0275] Figure 12 A flowchart of an example method 1200 for video processing is shown. Method 1200 includes, in operation 1210, determining the range of BV components in a one-dimensional block vector (BV) search for a conversion between a video including the current video block and a video bitstream, based on the properties of the current video block and the starting BV.

[0276] Method 1200 includes operation 1220, which performs a transformation based on determination.

[0277] The following solutions illustrate example implementations of the techniques discussed in the previous section (e.g., Items 1-5).

[0278] The following is a list of preferred solutions for some embodiments.

[0279] 1. A video processing method, comprising: for a conversion between a video including a current video block and a bitstream of the video, determining, based on a codec tree unit (CTU) row including a non-nearest neighbor block and the current video block, whether to use the block vector of the non-nearest neighbor block to predict the block vector of the current video block; and performing the conversion based on the determination.

[0280] 2. The method of Solution 1 further includes: in response to the current video block and the non-adjacent neighbor block being in different CTU rows, avoiding using the block vector of the non-adjacent neighbor block to predict the block vector of the current video block.

[0281] 3. The method of Solution 1 further includes: in response to (i) the current video block and the non-adjacent neighbor block being in different CTU rows, and (ii) the difference between the vertical coordinate of the current video block and the vertical coordinate of the current CTU row being less than or equal to a threshold Th, avoiding using the block vector of the non-adjacent neighbor block to predict the block vector of the current video block.

[0282] 4. The method described in Solution 3, where Th = 0.

[0283] 5. The method described in Solution 3, where Th = 4.

[0284] 6. The method described in Solution 3, wherein Th = M – 4, where M is an integer indicating the position of the non-immediately adjacent block.

[0285] 7. The method of Solution 1 further includes: in response to the current video block and the non-adjacent neighbor block being in different CTU rows, cropping the position of the non-adjacent neighbor block such that the non-adjacent neighbor block is in the same CTU row as the current video block, wherein the block vector at the cropped position is used for the prediction of the block vector of the current video block.

[0286] 8. The method of Solution 1 further includes: in response to the current video block and the non-adjacent neighbor block being in different CTU rows, cropping a first vertical position of the non-adjacent neighbor block such that the first vertical position is within a predetermined distance of a second vertical position ctuRowY of the current CTU row, wherein the block vector at the cropped first vertical position is used for the prediction of the block vector of the current video block.

[0287] 9. The method of Solution 8, wherein the first vertical position is cropped to ctuRowY–Th, where Th is a threshold.

[0288] 10. The method described in Solution 9, wherein Th = 0, Th = 4, or Th = 8.

[0289] 11. A video processing method, comprising: for a conversion between a video including a current video block and a bitstream of the video, determining, based on video units including non-nearest neighbor blocks and the current video block, whether to use the block vector of the non-nearest neighbor blocks to predict the block vector of the current video block; and performing the conversion based on the determination.

[0290] 12. The method of Solution 11 further includes: in response to the current video block and the non-adjacent neighbor block being in different video units, avoiding using the block vector of the non-adjacent neighbor block to predict the block vector of the current video block.

[0291] 13. The method of Solution 11 further includes: in response to the current video block and the non-adjacent neighbor block being in different video units, cropping the position of the non-adjacent neighbor block such that the non-adjacent neighbor block is in the same video unit as the current video block, wherein the block vector at the cropped position is used for the prediction of the block vector of the current video block.

[0292] 14. The method described in any of solutions 11-13, wherein the video unit comprises a slice, strip, or sub-picture.

[0293] 15. A video processing method, comprising: for a conversion between a video including a current video block and a bitstream of the video, determining the range of BV components in a one-dimensional block vector (BV) search based on attributes of the current video block and a starting BV; and performing the conversion based on the determination.

[0294] 16. The method of Solution 15, wherein the starting block vector is represented as (startBvX, startBvY), wherein the one-dimensional BV search includes a one-dimensional vertical BV search, wherein the vertical BV component is constrained to be less than or equal to (y+startBvY+K×H–N), where H is the height of the current video block, and N and K are integers.

[0295] 17. The method of Solution 15, wherein the starting block vector is represented as (startBvX, startBvY), wherein the one-dimensional BV search includes a one-dimensional horizontal BV search, wherein the horizontal BV component is constrained to be less than or equal to (x + startBvX + K × W – N), wherein W is the width of the current video block, and N and K are integers.

[0296] 18. The method described in solution 16 or 17, wherein N = -8, N = 0, or N = 8.

[0297] 19. The method described in any of solutions 16-18, wherein K = 2, K = 3, or K = 4.

[0298] 20. The method of any of solutions 15-19, wherein the initial BV is a BV used to initialize the one-dimensional BV search.

[0299] 21. The method of any one of solutions 1-20, wherein the conversion includes decoding the video from the bitstream.

[0300] 22. The method of any one of solutions 1-20, wherein the conversion includes encoding the video into the bitstream.

[0301] 23. A method for storing a bitstream representing a video into a computer-readable recording medium, comprising: generating the bitstream from the video according to any one or more of the methods described in solutions 1-20; and storing the bitstream in the computer-readable recording medium.

[0302] 24. A video processing apparatus, comprising a processor configured to implement the method described in any one or more of solutions 1-23.

[0303] 25. A computer-readable medium having instructions stored thereon, wherein, when executed, the instructions cause a processor to perform the method described in any one or more of solutions 1-23.

[0304] 26. A computer-readable medium storing the bit stream generated by any one or more of solutions 1-23.

[0305] 27. A video processing apparatus for storing bitstreams, wherein the video processing apparatus is configured to implement the method described in any one or more of solutions 1-23.

[0306] The following provides a list of alternative solutions preferred for some embodiments.

[0307] P1. A video processing method comprising constructing a table of motion candidates for a conversion between video blocks of a video and the codec representation of the video by adding one or more block vectors corresponding to one or more non-adjacent blocks of the current video block according to rules, and performing the conversion based on the table of motion candidates.

[0308] P2. The method of solution P1, wherein the table includes an intra-block copy merge table.

[0309] P3. The method of any one of solutions P1 to P2, wherein the table includes an advanced motion vector prediction table.

[0310] P4. The method of any one of solutions P1 to P3, wherein the rule specifies the order in which motion candidates of the one or more non-nearest neighbor blocks are examined based on the position of the one or more non-nearest neighbor blocks relative to the current video block.

[0311] P5. The method described in solution P4, wherein the order includes first checking the top-left neighboring block of the current block, then checking the top-right neighboring block, then checking the bottom-left neighboring block, and then checking the top neighboring block and the left neighboring block.

[0312] P6. The method of solution P4, wherein the order includes: the left neighboring block of the current block, the top neighboring block, the top left neighboring block, the top right neighboring block, and the bottom left neighboring block.

[0313] P7. A video processing method comprising: for a conversion between a current video block of a video and a bitstream representation of the video, determining whether a condition of one or more block vectors of one or more non-immediately adjacent blocks is satisfied, wherein the condition depends on the availability of block vectors from a historical block vector prediction table or the availability of block vectors of immediately adjacent blocks; and performing the conversion based on the determination.

[0314] P8. The method described in solution P7, wherein the condition for adding the block vector of the one or more non-immediately adjacent blocks is to insert all history-based block vectors into the table.

[0315] P9. A video processing method comprising: for a conversion between a current video block of a video and a codec representation of the video, determining whether a block vector in a historical block vector prediction (HBVP) table is classified into a Nth class according to a rule depending on a block size associated with the block vector or the size of the current video block; and performing the conversion based on the determination.

[0316] P10. The method described in Solution P9, wherein the rule specifies that the block vector is classified if the block size associated with the block vector is a factor multiple of the size of the current video block.

[0317] P11. The solution method described in P10, wherein the factor is equal to 1.

[0318] P12. A video processing method, comprising: for a conversion between a current video block of a video and a codec representation of the video, determining a one-dimensional search range based on rules to determine a block vector, the rules being based on attributes of the current video block; and performing the conversion according to the determination.

[0319] P13. The method described in Solution P12, wherein the attribute includes the coordinates of the current video block, and wherein the attribute is capable of determining the search range.

[0320] P14. The method described in Solution P12, wherein the attribute includes the size of the current video block.

[0321] P15. The method of any one of solutions P1 to P14, wherein performing the conversion includes encoding the video to generate the codec representation.

[0322] P16. The method of any one of solutions P1 to P14, wherein performing the conversion includes parsing and decoding the codec representation to generate the video.

[0323] P17. A video decoding apparatus, including a processor configured to perform one or more of the methods described in solutions P1 to P16.

[0324] P18. A video encoding device, including a processor configured to perform one or more of the methods described in solutions P1 to P16.

[0325] P19. A computer program product having computer code stored thereon, which, when executed by a processor, causes the processor to perform the method described in any one of solutions P1 to P16.

[0326] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to its corresponding bitstream representation, and vice versa. For example, as defined in the syntax, the bitstream representation (or simply bitstream) of the current video block may correspond to bits that are co-located or distributed at different positions within the bitstream. For example, macroblocks may be encoded based on the error residuals from the transform and encoding / decoding, and bits may also be used in the header and other fields of the bitstream.

[0327] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or combinations thereof. The disclosures and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for execution by a data processing apparatus or for controlling the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that influences machine-readable propagated signals, or one or more of these. The terms "data processing unit" or "data processing apparatus" include all means, devices, and machines for processing data, including, for example, programmable processors, computers, or multiprocessors or computer groups. In addition to hardware, the apparatus may also include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or combinations thereof. The propagated signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[0328] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to that program, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed and executed on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0329] The processing and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by special-purpose logic circuitry, and the devices can be implemented as special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0330] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as one or more of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store the instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or receive data from or transfer data to one or more mass storage devices via operative coupling, or both. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0331] While this patent document contains numerous details, it should not be construed as limiting the scope of any invention or claim, but rather as a description of features of specific embodiments of a particular invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented individually in multiple embodiments, or in any suitable sub-combination. Furthermore, although the foregoing features may be described as functioning in certain combinations, or even initially claimed to be so, in certain circumstances, one or more features from a combination of claims may be removed from the combination, and a combination of claims may refer to a sub-combination or a variation of a sub-combination.

[0332] Similarly, although the operations are described in a specific order in the accompanying drawings, this should not be construed as requiring the specific order or sequence shown to perform such operations, or all the described operations, in order to obtain the desired result. Furthermore, the separation of various system components in the embodiments of this patent document should not be construed as requiring such separation in all embodiments.

[0333] Only some implementations and examples are described; other implementations, enhancements, and variations can be made based on the content described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For the conversion between the video including the current video block and the bitstream of the video, based on whether the non-nearest neighbor block and the current video block are in different codec tree unit (CTU) rows, it is determined whether to insert the block vector of the non-nearest neighbor block into a motion candidate list, which is used to predict the block vector of the current video block; as well as The conversion is performed based on the determination. In response to the current video block and the non-immediately adjacent block being in different CTU rows, the block vector of the non-immediately adjacent block is not inserted into the list of motion candidates used to predict the block vector of the current video block.

2. The method according to claim 1, wherein, When constructing the list of motion candidates, examine the non-nearest neighbor blocks covering the position (x – M, y + H / 2) or the position (x + W / 2, y – M), where the coordinates of the top-left sample point of the current video block are (x, y), M is an integer, W is the width of the current video block, and H is the height of the current video block.

3. The method according to claim 2, wherein, M＝8。 4. The method according to claim 1, wherein, After the block vectors from the historical block vector prediction HBVP candidate list are inserted into the list, the block vectors of the non-immediately adjacent blocks are inserted into the list of motion candidates.

5. The method according to claim 1, wherein, After inserting block vectors from (i) the historical block vector prediction HBVP candidate list and / or from (ii) the block vectors of the immediate neighboring blocks of the current video block, in response to the fact that the list of motion candidates does not contain empty entries, the block vectors of the non-immediate neighboring blocks are not inserted into the list of motion candidates.

6. The method according to claim 1, wherein, The list includes the Intra-Block Copy (IBC) Merge list.

7. The method according to claim 1, wherein, The list includes a list of Intra-Block Copy (IBC) Advanced Motion Vector Prediction (AMVP) programs.

8. The method according to claim 1, wherein, The conversion includes decoding the video from the bitstream.

9. The method according to claim 1, wherein, The conversion includes encoding the video into the bitstream.

10. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein, When the instruction is executed by the processor, it causes the processor to: For the conversion between the video including the current video block and the bitstream of the video, based on whether the non-nearest neighbor block and the current video block are in different codec tree unit (CTU) rows, it is determined whether to insert the block vector of the non-nearest neighbor block into a motion candidate list, which is used to predict the block vector of the current video block; as well as The conversion is performed based on the determination. In response to the current video block and the non-immediately adjacent block being in different CTU rows, the block vector of the non-immediately adjacent block is not inserted into the list of motion candidates used to predict the block vector of the current video block.

11. The apparatus according to claim 10, wherein, When constructing the list of motion candidates, examine the non-nearest neighbor blocks covering the position (x – M, y + H / 2) or the position (x + W / 2, y – M), where the coordinates of the top-left sample point of the current video block are (x, y), M is an integer, W is the width of the current video block, and H is the height of the current video block.

12. The apparatus according to claim 11, wherein, M＝8。 13. The apparatus according to claim 10, wherein, After the block vectors from the historical block vector prediction HBVP candidate list are inserted into the list, the block vectors of the non-immediately adjacent blocks are inserted into the list of motion candidates.

14. The apparatus according to claim 10, wherein, After inserting a block vector from (i) the history-based block vector prediction HBVP candidate list and / or a block vector from (ii) the immediate neighbor block of the current video block, in response to the fact that the list of motion candidates does not contain empty entries, the block vector of the non-immediate neighbor block is not inserted into the list of motion candidates.

15. A non-transitory computer-readable storage medium for storing instructions, said instructions causing a processor to: For the conversion between the video including the current video block and the bitstream of the video, based on whether the non-nearest neighbor block and the current video block are in different codec tree unit (CTU) rows, it is determined whether to insert the block vector of the non-nearest neighbor block into a motion candidate list, the list being used to predict the block vector of the current video block; and The conversion is performed based on the determination. in, In response to the current video block and the non-immediately adjacent block being in different CTU rows, the block vector of the non-immediately adjacent block is not inserted into the list of motion candidates used to predict the block vector of the current video block.

16. The non-transitory computer-readable storage medium according to claim 15, wherein, When constructing the list of motion candidates, examine the non-nearest neighbor blocks covering the position (x – M, y + H / 2) or the position (x + W / 2, y – M), where the coordinates of the top-left sample point of the current video block are (x, y), M = 8, W is the width of the current video block, and H is the height of the current video block.

17. The non-transitory computer-readable storage medium according to claim 15, wherein, After block vectors from the historical block vector prediction HBVP candidate list are inserted into the list, the block vectors of the non-immediately adjacent blocks are inserted into the motion candidate list, and Wherein, after inserting block vectors from (i) the historical block vector prediction HBVP candidate list and / or from (ii) the immediate neighboring block of the current video block, in response to the motion candidate list not containing empty entries, the block vectors of the non-immediate neighboring blocks are not inserted into the motion candidate list.

18. A non-transitory computer-readable recording medium for storing bitstreams of instructions and video, wherein, The instruction causes the processor to: Based on whether the non-nearest neighbor blocks and the current video block of the video are in different codec tree unit (CTU) rows, determine whether to insert the block vector of the non-nearest neighbor blocks into a list of motion candidates, the list being used to predict the block vector of the current video block; as well as The bit stream is generated based on the determination. In response to the current video block and the non-immediately adjacent block being in different CTU rows, the block vector of the non-immediately adjacent block is not inserted into the list of motion candidates used to predict the block vector of the current video block.

19. The non-transitory computer-readable recording medium according to claim 18, wherein, When constructing the list of motion candidates, examine the non-nearest neighbor blocks covering the position (x – M, y + H / 2) or the position (x + W / 2, y – M), where the coordinates of the top-left sample point of the current video block are (x, y), M = 8, W is the width of the current video block, and H is the height of the current video block.

20. The non-transitory computer-readable recording medium according to claim 18, wherein, After block vectors from the historical block vector prediction HBVP candidate list are inserted into the list, the block vectors of the non-immediately adjacent blocks are inserted into the motion candidate list, and Wherein, after inserting block vectors from (i) the historical block vector prediction HBVP candidate list and / or from (ii) the immediate neighboring block of the current video block, in response to the motion candidate list not containing empty entries, the block vectors of the non-immediate neighboring blocks are not inserted into the motion candidate list.

21. A method for storing a bitstream of video, comprising: Based on whether the non-nearest neighbor blocks and the current video block of the video are in different codec tree unit (CTU) rows, determine whether to insert the block vector of the non-nearest neighbor blocks into a list of motion candidates, the list being used to predict the block vector of the current video block; The bit stream is generated based on the determination; as well as The bitstream is stored in a non-transitory computer-readable recording medium. In response to the current video block and the non-immediately adjacent block being in different CTU rows, the block vector of the non-immediately adjacent block is not inserted into the list of motion candidates used to predict the block vector of the current video block.

Images