Extended method for motion reference in intra block copy mode

Abstract

The present application relates to an extended method for motion reference in an intra-block copy mode. A method for video processing is disclosed. The method for video processing includes: during a conversion between a current block and a bitstream representation of the current block, determining one or more padding values; and performing the conversion based at least on the one or more padding values.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a Chinese national phase application of International Patent Application No. PCT / CN2019 / 121642 filed on November 28, 2019, and this application promptly claims priority to and benefits of International Patent Application No. PCT / CN2018 / 117831 filed on November 28, 2018, International Patent Application No. PCT / CN2018 / 122953 filed on December 22, 2018, International Patent Application No. PCT / CN2018 / 125403 filed on December 29, 2018, and International Patent Application No. PCT / CN2019 / 070149 filed on January 2, 2019. The entire disclosure of the above applications is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This application document relates to image and video encoding and decoding. Background Art

[0004] Digital video consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention

[0005] The disclosed techniques may be used by video decoder or encoder embodiments during video decoding or encoding using an intra video codec tool such as a current picture reference (CPR) coding tool.

[0006] In one example aspect, a video processing method is disclosed. The method includes: applying an intra codec or an inter codec to a current video block of a current video picture for conversion between a current video block and a bitstream representation of the current video block, the current video block being referenced by the intra codec or the inter codec from a reference video block at least partially located in the current video picture; determining one or more padding values ​​during the conversion, wherein the one or more padding values ​​are signaled in the bitstream representation; and performing the conversion using the one or more padding values ​​and the intra codec or the inter codec.

[0007] In yet another example aspect, another video processing method is disclosed. The method includes: during a conversion between a current video block and a bitstream representation of the current video block, determining an intra candidate list for an intra codec mode of the current video block; and performing the conversion between the current video block and the bitstream representation using the intra candidate list, wherein the intra candidate list for the intra codec mode is different from a candidate list for an inter codec mode of the current video block; wherein the candidate list is one of a Merge list, a history-based motion vector predictor list, or an affine Merge list.

[0008] In yet another example aspect, another video processing method is disclosed. The method includes: performing a determination that a codec mode of a current video block is a transform-quantization bypass codec mode, wherein the current video block is encoded into a bitstream representation by omitting a transform step and a quantization step; and based on the determination, performing a conversion between the current video block and the bitstream representation according to the transform-quantization bypass codec mode, wherein the conversion is performed without filtering based on an adaptive loop filter.

[0009] In yet another example aspect, another video processing method is disclosed. The method includes: determining that a block vector of a current video block that is a chrominance block cannot be derived based on a luma block in a collocated luma region of the current video block; based on the determination, selecting a default block vector as the block vector of the chrominance block; and performing a conversion between the current video block and a bitstream representation of the current video block based on the block vector.

[0010] In yet another example aspect, another video processing method is disclosed. The method includes: during conversion between a current block and a bitstream representation of the current block, determining one or more padding values; and performing the conversion based at least on the one or more padding values.

[0011] In yet another example aspect, another video processing method is disclosed. The method includes: during a first conversion between a first video block of a video and a bitstream representation of the video, determining a first candidate list construction method for a first video block having a first codec mode; performing a first conversion based at least on a first candidate list obtained according to the first candidate list construction method; wherein the conversion of the first video block is based on samples in a current picture, and the first candidate list construction method is different from a second candidate list construction method, and during a second conversion between a second video block of a video having a second codec mode and a bitstream representation of the video, applying the second candidate list construction method on the second video block.

[0012] In yet another example aspect, another video processing method is disclosed. The method includes: during a conversion between a current block and a bitstream representation of the current block, determining that the current block is encoded using a transform quantization bypass codec mode; and in response to the current block being encoded using the transform quantization bypass codec mode, performing the conversion between the current block and the bitstream representation of the current block without filtering based on an adaptive loop filter, wherein the transform quantization bypass codec mode is a codec mode in which the block is encoded without using one or more of transform, quantization, and loop filtering.

[0013] In yet another example aspect, another video processing method is disclosed. The method includes: during conversion between a current block and a bitstream representation of the current block, performing processing of one or more transform quantization bypass flags based on color components of the current block, wherein the transform quantization bypass flag is associated with a transform quantization bypass codec mode, wherein the transform quantization bypass codec mode is a codec mode in which the block is encoded without using one or more of transform, quantization, and loop filtering; performing conversion based on the processing.

[0014] In yet another example embodiment, a video encoder apparatus is disclosed. The encoder apparatus includes a processor configured to implement the above method.

[0015] In yet another aspect, a video decoder device is disclosed. The video decoder device includes a processor configured to implement the above method.

[0016] In another aspect, a computer-readable program medium is disclosed. The medium includes code. The code includes processor-executable instructions for implementing the above method.

[0017] These and other aspects are described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 An example illustration of a current picture reference (CPR) is shown.

[0019] Figure 2 An example illustration of horizontal padding is shown.

[0020] Figure 3 An example of a simplified affine motion model is shown.

[0021] Figure 4 An example of the affine MVF for each sub-block is shown.

[0022] Figure 5A An example of a 4-parameter affine model is shown.

[0023] Figure 5B An example of a 6-parameter affine model is shown.

[0024] Figure 6 An example of MVP for AF_INTER mode is shown.

[0025] Fig. 7A and 7B Examples of candidates for the AF_Merge mode are shown.

[0026] Figure 8 An example of candidate positions of the affine merge mode is shown.

[0027] Fig. 9 An example of ATMVP motion prediction for a CU is shown.

[0028] Fig.10 is an example of decoding using the proposed HMVP method.

[0029] Fig.11 is an example of updating the table in the proposed HMVP method.

[0030] Fig.12 is an example of a codec tree unit (CTU) and a dual tree.

[0031] Fig.13 is a block diagram of an example device that may implement the encoding or decoding techniques described herein.

[0032] Fig.14 is a flow chart of an example method of video processing.

[0033] Fig.15 is a flow chart of an example method of video processing.

[0034] Fig.16 is a flow chart of an example method of video processing.

[0035] Fig.17 is a flow chart of an example method of video processing.

[0036] Fig.18 is a flow chart of an example method of video processing. DETAILED DESCRIPTION

[0037] This document provides various techniques that can be used by a decoder of a video bitstream to improve the quality of a decompressed or decoded digital video or image. In addition, a video encoder can also implement these techniques in the encoding process to reconstruct decoded frames for further encoding.

[0038] For ease of understanding, section headings are used in this document, and embodiments and techniques are not limited to the corresponding sections. Therefore, embodiments from one section can be combined with embodiments from other sections.

[0039] 1. Introductory remarks

[0040] This patent document relates to video coding technology. Specifically, it relates to current picture reference coding mode, and some methods can be extended to traditional inter-frame coding mode in video coding. It can be applied to existing video coding standards (such as HEVC), or to a pending standard (multi-function video codec). It may also be applicable to future video coding standards or video codecs.

[0041] 2. Description

[0042] 2.1 Current Image Reference

[0043] In HEVC Screen Content Codec Extension (HEVC-SCC) [1] and the current VVC test model, Current Picture Reference (CPR) (formerly known as Intra Block Copy (IBC)) is used. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 1 As shown, when CPR is applied, the current block is predicted by the reference block in the same picture. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. Although CPR is not very effective for sequences captured by most cameras, it shows significant codec gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. CPR can effectively remove the redundancy between these repeated patterns. In HEVC-SCC, CPR can be applied if the inter-coded codec unit (CU) selects the current picture as its reference picture. In this case, MV is renamed as block vector (BV), and BV always has integer pixel precision. In order to be compatible with the main specification HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that, similarly, in the multi-view / 3D video codec standard, the inter-view reference picture is also marked as a "long-term" reference picture.

[0044] After the BV finds its reference block, a prediction can be generated by copying the reference block. The residual is obtained by subtracting the reference pixels from the original signal. Transforms and quantization can then be applied like in other codec modes.

[0045] Figure 1 An example illustration of a current picture reference is shown.

[0046] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to certain constraints, some or all pixel values ​​are not defined. Basically, there are two solutions to deal with such a problem. One is to prohibit this situation, for example in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0047] 2.2 CPR in HEVC Screen Content Codec Extension

[0048] In the screen content codec extension of HEVC, when a block uses the current picture as a reference, it should ensure that the entire reference block is within the available reconstruction area, as shown in the following specification text:

[0049] The variables offsetX and offsetY are derived as follows:

[0050] OffsetX=(ChromaArrayType==0)? 0: (mvCLX[0] & 0x7? 2: 0) (8-104)

[0051] OffsetY=(ChromaArrayType==0)? 0: (mvCLX[1] & 0x7? 2: 0) (8-105)

[0052] The bitstream conformance requirement is that when the reference picture is the current picture, the luma motion vector mvLX shall obey the following constraints:

[0053] -When the derivation of z-scan order block availability specified in Section 6.4.1 is called with (xCurr, yCurr) set equal to (xCb, yCb) and the adjacent luminance position (xNbY, yNbY) set equal to (xPb+(mvLX[0]>>2)-offsetX, yPb+(mvLX[1]>>2)-offsetY) as input, the output shall be true.

[0054] -When the derivation of z-scan order block availability specified in Section 6.4.1 is called with (xCurr, yCurr) set equal to (xCb, yCb) and the adjacent luminance position (xNbY, yNbY) set equal to (xPb+(mvLX[0]>>2)+nPbW-1+offsetX, yPb+(mvLX[1]>>2)+nPbH-1+offsetY) as input, the output shall be true.

[0055] -One or both of the following conditions should be true:

[0056] The value of -(mvLX[0]>>2)+nPbW+xBl+offsetX is less than or equal to 0.

[0057] The value of -(mvLX[l]>>2)+nPbH+yBl+offsetY is less than or equal to 0.

[0058] - The following conditions should be true:

[0059] (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCb / CtbSizeY<=

[0060] yCb / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY

[0061] (8-106)

[0062] Therefore, it does not happen that the reference block overlaps with the current block or is outside the picture. No padding of reference or prediction blocks is required.

[0063] 2.3 CPR in the VVC test model

[0064] In the current VVC test model, the entire reference block should be consistent with the current codec tree unit (CTU) and not overlap with the current block. Therefore, there is no need to pad the reference block or prediction block.

[0065] When dual-tree is enabled, the partitioning structure may be different from luma to chroma CTU. Thus, for a 4:2:0 color format, one chroma block (e.g., CU) may correspond to a collocated luma region that has been partitioned into multiple luma CUs.

[0066] Chroma blocks can only be encoded or decoded in CPR mode when the following conditions are true:

[0067] 1) Each luma CU in a collocated luma block should be coded and decoded using the CPR mode;

[0068] 2) The BV of each luma 4×4 block is first converted to the BV of the chroma block, and the BV of the chroma block is a valid BV.

[0069] If either of these two conditions is false, the chroma blocks shall not be encoded or decoded using CPR mode.

[0070] It is important to note that the definition of “effective BV” has the following restrictions:

[0071] 1) All samples within the reference block identified by BV should be within the restricted search range (e.g., should be within the same CTU in current VVC designs).

[0072] 2) All samples in the reference block identified by BV have been reconstructed.

[0073] 2.4Padding in CPR / IBC Codec Mode

[0074] During the development of CPR / IBC for HEVC, several methods were proposed to use padding to handle the problem of undefined pixel values.

[0075] 2.4.1 Preset of middle gray value

[0076] In some embodiments, when a pixel value in a reference block is undefined, it is inferred to be 128, a middle gray value in an 8-bit video signal representation.

[0077] 2.4.2 Horizontal Padding

[0078] In some embodiments, when a pixel is undefined due to overlap with the current block, the pixel is filled with horizontally available pixels.Some embodiments further apply this method to process pixels outside of a picture.

[0079] Figure 2 An example illustration of horizontal padding is shown.

[0080] 2.4.3 Horizontal or vertical padding depending on BVx and BVy values

[0081] When |BVy|>|BVx|, vertical padding is applied. Otherwise, horizontal padding is applied.

[0082] 2.4.4 Filling along BV

[0083] In some embodiments, when a pixel needs to be filled, it will use the BV to find the pixel in the valid area and use its value as the fill value.

[0084] 2.4.5 Fill with CU / CTU-level values ​​derived from adjacent samples via histogram or large samples

[0085] In some embodiments, two methods of filling using adjacent pixels are proposed. One is to calculate the histogram of adjacent pixels and select the most frequent value as the filling value. Some embodiments propose similar methods. Another is to calculate the longest connected pixel value in the adjacent area and use this value as the filling value. Therefore, filling requires a lot of calculations, which brings potential burden to the decoding of each CU.

[0086] 2.5 Affine Motion Compensated Prediction

[0087] In HEVC, motion compensated prediction (MCP) only applies the translation motion model. However, in the real world, there may be a variety of motions, such as zooming in / out, rotation, perspective motion, and other irregular motions. In JEM, a simplified affine transformation motion compensated prediction is applied. Figure 1 As shown, the affine motion field of the block is described by two control point motion vectors.

[0088] Figure 3 An example of a simplified affine motion model is shown.

[0089] The motion vector field (MVF) of a block is described by the following equation:

[0090]

[0091] Among them, (v 0x ,v 0y ) is the motion vector of the upper left control point, and (v 1x ,v 1y ) is the motion vector of the upper right control point.

[0092] To further simplify motion compensated prediction, sub-block based affine transform prediction is applied. The sub-block size M×N is derived as Equation 2, where MvPre is the fractional precision of the motion vector (e.g., 1 / 16 in JEM). (v 2x ,v 2y ) is the motion vector of the lower left control point, which is calculated according to Equation 1.

[0093]

[0094] After being derived from Equation 2, M and N should be adjusted downwards, if necessary, to make them divisors of w and h, respectively.

[0095] In order to derive the motion vector of each M×N sub-block, as Figure 4 As shown, the motion vector of the center sample of each sub-block can be calculated according to Equation 1 and rounded to 1 / 16 fractional accuracy.

[0096] Figure 4 An example of the affine MVF for each sub-block is shown.

[0097] After MCP, the high-precision motion vector of each sub-block is rounded and saved to the same precision as the regular motion vector.

[0098] 2.5.1AF_INTER mode

[0099] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with width and height greater than 8, AF_INTER mode can be applied. In the bitstream, the affine flag at the CU level is signaled to indicate whether AF_INTER mode is used. In this mode, neighboring blocks are used to construct a pair of motion vectors {(v0,v1)|v0={v A ,v B ,v c},v1={v D ,v E}} candidate list. Figure 6 As shown, v0 is selected from the motion vector of block A, B or C. The motion vector from the adjacent block is scaled according to the reference list and the relationship between the POC of the adjacent block reference, the POC of the current CU reference and the POC of the current CU. The method of selecting v1 from adjacent blocks D and E is similar. When the number of candidate lists is less than 2, the list is filled by copying the motion vector pairs consisting of each AMVP candidate. When the candidate list is greater than 2, the candidates can be first sorted according to the consistency of the adjacent motion vectors (the similarity of the two motion vectors in a pair of candidates), and only the first two candidates are retained. The RD cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) of the current CU. And the index indicating the position of the CPMVP in the candidate list is signaled in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied, and the control point motion vector (CPMV) is found. Then, the difference between CPMV and CPMVP is signaled in the bitstream.

[0100] Figure 5A An example of a 4-parameter affine model is shown. Figure 5B An example of a 6-parameter affine model is shown.

[0101] Figure 6 An example of MVP for AF_INTER mode is shown.

[0102] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and thus 2 / 3 MVDs need to be encoded and decoded for these control points, such as Figure 6 In some embodiments, it is proposed to derive MV as follows, that is, predict mvd1 and mvd2 from mvd0.

[0103]

[0104] in, mvd iand mv1 are the predicted motion vector, motion vector difference and motion vector of the upper left pixel (i=0), upper right pixel (i=1) or lower left pixel (i=2), respectively. Figure 5B Note that the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of the two components, i.e., newMV=mvA+mvB, and the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0105] 2.5.2AF_MERGE Mode

[0106] When a CU is applied in AF_MERGE mode, it obtains the first block encoded and decoded in affine mode from the valid adjacent reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, such as Fig. 7A If the adjacent lower left block A is encoded and decoded in affine mode, such as Figure 7B As shown, the motion vectors v2, v3 and v4 of the upper left corner, upper right corner and lower left corner of the CU containing block A are derived. And the motion vector v0 of the upper left corner of the current CU is calculated based on v2, v3 and v4. Secondly, the motion vector v1 of the upper right corner of the current CU is calculated.

[0107] After deriving the CPMV v0 and v1 of the current CU, the MVF of the current CU is generated according to the simplified affine motion model equation 1. In order to identify whether the current CU is encoded or decoded in AF_MERGE mode, an affine flag is signaled in the bitstream when at least one neighboring block is encoded or decoded in affine mode.

[0108] Fig. 7A and 7B Examples of candidates for the AF_MERGE mode are shown.

[0109] In some embodiments, the following steps are used to construct an affine merge candidate:

[0110] 1) Insert inherited affine candidates

[0111] An inherited affine candidate is one that is derived from the affine motion model of its valid neighboring affine codec blocks. On a common basis, such as Figure 8 As shown, the scanning order of candidate positions is: A1, B1, B0, A0 and B2.

[0112] After deriving the candidates, a full pruning is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.

[0113] 2) Insert the constructed affine candidate

[0114] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5 in this paper), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate means that the candidate is constructed by combining the adjacent motion information of each control point.

[0115] First from Figure 8 The prescribed spatial and temporal neighbors shown derive the motion information of the control point. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions used to predict CPk (k = 1, 2, 3); T is the temporal position used to predict CP4.

[0116] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

[0117] Figure 8 An example of candidate positions of the affine merge mode is shown.

[0118] Get the motion information for each control point in the following order of priority:

[0119] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is not available, use B3. If both B2 and B3 are not available, use A2. If none of the three candidates are available, the motion information of CP1 cannot be obtained.

[0120] For CP2, the check priority is B1->B0;

[0121] For CP3, the check priority is A1->A0;

[0122] For CP4, use T.

[0123] Second, affine merge candidates are constructed using combinations of control points.

[0124] The motion information of three control points is required to construct a 6-parameter affine candidate. These three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted to a 6-parameter motion model represented by the top left, top right and bottom left control points.

[0125] The motion information of two control points is required to construct a 4-parameter affine candidate. The two control points can be selected from one of the following six combinations ({CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}). The combination {CP1,CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4} will be converted to a 4-parameter motion model represented by the top left and top right control points.

[0126] Insert the combinations of constructed affine candidates into the candidate list in the following order:

[0127] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}.

[0128] For a combined reference list X (X is 0 or 1), the reference index with the highest usage rate in the control point is selected as the reference index of list X, and the motion vectors pointing to different reference pictures are scaled.

[0129] After deriving the candidates, a full pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.

[0130] 3) Fill zero motion vectors

[0131] 4) If the number of candidates in the affine merge candidate list is less than 5, insert zero motion vectors with zero reference index into the candidate list until the list is full.

[0132] 2.6Merge

[0133] VVC supports three different Merge list construction processes:

[0134] 1) Sub-block Merge candidate list: It includes ATMVP and affine Merge candidates. For both affine mode and ATMVP mode, a Merge list construction process is shared. Here, ATMVP and affine Merge candidates can be added in sequence. The sub-block Merge list size is signaled in the slice header, and the maximum value is 5.

[0135] 2) Unidirectional prediction TPM Merge list: For triangular prediction mode, even if the two partitions can select their own Merge candidate indexes, the two partitions share a Merge list construction process. When constructing this Merge list, the spatial neighbors of the block and the two temporal blocks are checked. In our IDF, the motion information derived from the spatial neighbors and temporal blocks is called regular motion candidates. These regular motion candidates are further used to derive multiple TPM candidates. Note that the conversion is performed at the entire block level, even if the two partitions can use different motion vectors to generate their own prediction blocks. The unidirectional prediction TPM Merge list size is fixed to 5.

[0136] 3) Regular Merge List: For the rest of the codec blocks, a Merge list construction process is shared. Here, spatial / temporal / HMVP, paired combined bidirectional prediction Merge candidates and zero motion candidates can be inserted in sequence. The regular Merge list size is signaled in the slice header, and the maximum value is 6.

[0137] 2.7ATMVP

[0138] In the optional temporal motion vector prediction (ATMVP) method, the motion vector temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. Fig. 9 As shown, the sub-CU is a square N×N block (the default N is set to 4).

[0139] ATMVP predicts the motion vectors of sub-CUs within a CU in two steps. The first step is to identify the corresponding block in the reference picture using the so-called time domain vector. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU, such as Fig. 9 shown.

[0140] Fig. 9 An example of ATMVP motion prediction for a CU is shown.

[0141] In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid repeated scanning of neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the temporal vector and the index of the motion source picture. In this way, in ATMVP, the corresponding block can be identified more accurately than in TMVP, where the corresponding block (sometimes called the collocated block) is always located at the lower right corner or center relative to the current CU.

[0142] In the second step, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture by adding the time domain vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the minimum motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding N×N block, it is converted into the motion vector and reference index of the current sub-CU, which is the same as the TMVP method of HEVC, where motion scaling and other processing are applied. For example, the decoder checks whether the low latency condition is met (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MV x (motion vector corresponding to reference picture list X) to predict the motion vector MV for each sub-CU y (X is equal to 0 or 1 and Y is equal to 1-X).

[0143] 2.8HMVP

[0144] A history-based MVP (HMVP) method is proposed, where the HMVP candidate is defined as the motion information of the previously coded block. A table with multiple HMVP candidates is maintained during the encoding / decoding process. When a new slice is encountered, the table will be cleared. Whenever there is a non-affine block for inter-frame coding, the associated motion information is added to the last entry of the table as a new HMVP candidate. The overall coding and decoding process is as follows Fig.10 shown.

[0145] Fig.10 An example of a decoding flow chart with the proposed HMVP method is shown.

[0146] Fig.11 An example of updating a table in the proposed HMVP method is shown.

[0147] In this document, the table size S is set to 6, which indicates that up to 6 HMVP candidates can be added to the table. When inserting a new motion candidate into the table, a constrained FIFO rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, and then all HMVP candidates are moved forward, i.e., the index is reduced by 1.

[0148] HMVP candidates can be used in the construction process of the Merge candidate list. The latest few HMVP candidates in the table will be checked in order and inserted into the candidate list after the TMVP candidate. Pruning is applied to the HMVP candidates, which is applied to the spatial or temporal Merge candidates that do not include sub-block motion candidates (i.e., ATMVP).

[0149] To reduce the number of pruning operations, three simplifications are introduced:

[0150] 1) The number of HMPV candidates to be checked, denoted by L, is set as follows:

[0151] L= (N <=4)? M: (8 - N) (1)

[0152] Where N indicates the number of non-sub-block Merge candidates available in the table, and M indicates the number of HMVP candidates available in the table.

[0153] 2) In addition, once the total number of available Merge candidates reaches the signaled maximum allowed Merge candidate minus 1, the process of constructing the Merge candidate list from the HMVP list is terminated.

[0154] 3) In addition, the number of pairs used to combine bi-predictive Merge candidate derivations is reduced from 12 to 6.

[0155] Similarly, HMVP candidates can also be used in the AMVP candidate list construction process. The motion vectors of the last K HMVP candidates in the table are inserted after the TMVP candidates. Only HMVP candidates with the same reference picture as the AMVP target reference picture are used to construct the AMVP candidate list. Pruning is applied on the HMVP candidates. In this document, K is set to 4, while the AMVP list size remains unchanged, that is, equal to 2.

[0156] 2.9 Dual Codec Tree

[0157] In the current VVC design, the luminance and chrominance components can have different codec trees. A CTU can have two codec trees. One is for the luminance component, where the codec tree defines how to divide the CTU into codec units for the luminance component. The other is for the chrominance component, where the codec tree defines how to divide the CTU into codec units for the chrominance component.

[0158] When using a dual codec tree, a codec unit may contain only luma components or only chroma components. Since luma and chroma components have different codec trees, the codec units of luma and chroma components may not be aligned, so that a luma codec unit may correspond to several chroma codec units; and a chroma codec unit may also correspond to several luma codec units.

[0159] 2.10 Transquant Bypass Design in HEVC

[0160] In the HEVC design, it is allowed to bypass transform, quantization and loop filtering for codec units to provide a mechanism for lossless coding. When this feature is enabled, a flag called cu_transquant_bypass_flag will be sent for each codec unit. When cu_transquant_bypass_flag is true for a codec unit, the corresponding transform, quantization and loop filtering, including deblocking and sample adaptive offset, will not be performed.

[0161] In addition to the CU level flag, an SPS level flag (transquant_bypass_enabled_flag) is signaled first to indicate whether this method is enabled or disabled.

[0162] 3. Examples of technical problems solved by the disclosed embodiments

[0163] 1. In the current design of VVC test mode, the reference block of CPR should be completely located in a certain area, such as the reconstruction area of ​​the current CTU. Such a restriction significantly reduces the number of potential reference blocks and may thus lead to codec losses. Existing filling methods are either not very flexible, such as filling with intermediate gray levels, or significantly increase the complexity of the decoder, such as statistical filling based on neighboring pixels. An effective filling method is needed.

[0164] 2. In the current design of HEVC-SCC and VVC test modes, CPR is treated as an inter-frame mode when the reference picture is the current picture. However, since CPR and inter-frame may have many different characteristics, mixing them together limits the efficiency of each. Therefore, in order to benefit the coding efficiency, it is necessary to separate CPR from inter-frame mode.

[0165] 3. When dual-tree is applied, different partition structures can be selected for luma and chroma blocks. It is not clear how to handle transform quantization bypass mode.

[0166] 4. When dual-tree is applied, the current VVC design requires that all luma CUs in the collocated luma region be coded and decoded with CPR, and the BV of each 2x2 block derived from each 4x4 luma block is valid. This strict constraint on the use of CPR mode limits the coding gain of CPR.

[0167] 4. Example Embodiments and Techniques

[0168] The detailed techniques below should be considered as examples to explain the general concepts. These techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way.

[0169] The fill of the current image reference

[0170] 1. Fill with the value signaled to the decoder. This value may represent the background. Therefore, filling the value with this value is likely to hit the background.

[0171] a. In one example, the value may be signaled at the sequence level (eg, SPS). It indicates that the value is common in the sequence.

[0172] b. In one example, the value may be signaled at the slice / slice group / slice level (eg, slice group header). It indicates that the value is common in the slice or slice.

[0173] c. In one example, the value may be signaled at the CTU level (eg, CTU syntax). It indicates that the value is common in the region.

[0174] d. In one example, the value may be signaled in a region / VPS / PPS / picture header / CTU row covering multiple CTUs or multiple CUs.

[0175] e. If the value can be signaled at different levels (eg SPS and PPS), prediction of the signaling of the value can be applied. In this way, the difference between two values ​​can be signaled.

[0176] f. The value may be quantized before signaling. After parsing, the value may be dequantized at the decoder.

[0177] g. The value (which can be quantized) can be binarized using the following code:

[0178] a. Unary code;

[0179] b. Truncated unary code;

[0180] c. Fixed length code;

[0181] d. Exponential Golomb code (such as EG-0 or EG-1)

[0182] 2. Fill with adaptively updateable values.

[0183] a. In one example, for each CPR CU, a flag is sent to indicate whether to use padding with new values ​​or padding with default values. When the flag is true, a value will be sent subsequently. When the flag is false, if padding is required, padding with default values ​​will be used.

[0184] 3. Instead of encoding one value in the above example, a set of values ​​may be signaled. For each CTU / slice / slice / picture, the index of the value in the set may be signaled.

[0185] 4. Fill with the values ​​in the list

[0186] a. In one example, a list of populated values ​​is maintained. When a new value is used, it is inserted at the head of the list.

[0187] b. Alternatively, the new value may be inserted at the end of the list.

[0188] c. In one example, when inserting a value into a list, if the same value already exists in the list, no action is performed.

[0189] d. Alternatively, when inserting a value into a list, if the same value already exists in the list, the new element is not inserted, but the value is placed at the head of the list.

[0190] e. Alternatively, when inserting a value into a list, if the same value already exists in the list, the new element is not inserted, but the value is placed at the end of the list.

[0191] f. In one example, the list may be initialized with predefined values, such as 0, middle gray, maximum level, or a combination thereof.

[0192] g. In one example, the list may be initialized with the values ​​sent at the SPS / PPS / slice / slice group level.

[0193] h. In one example, a flag is sent to indicate whether to use padding with values ​​in a pad value list.

[0194] i. In one example, when a value in a populate value list is used, an index is sent to indicate which value in the list is to be used.

[0195] 5. When padding is required, the above padding method can also be applied to the conventional inter-frame coding mode.

[0196] a. In one example, different padding values ​​can be used for different codec modes.

[0197] 6. The fill values ​​disclosed above can be used for different color components.

[0198] a. In one example, different fill values ​​are used for different color components.

[0199] b. In one example, different fill values ​​are signaled for different color components.

[0200] 7. In one example, different fill values ​​may be used for different areas.

[0201] Separate design of current picture reference and inter-frame mode

[0202] 8. It is proposed that a separate Merge list should be used and maintained for the CPR codec mode.

[0203] 9. It is proposed that a separate AMVP list should be used and maintained for the CPR codec mode.

[0204] 10. It is proposed that a separate HMVP list should be used and maintained for the CPR codec mode.

[0205] 11. It is proposed that a separate affine Merge list should be used and maintained for the CPR codec mode.

[0206] 12. Since (0,0) is not a valid BV for CPR, it is proposed that the zero motion vector (ie, (0,0)) used as a default filling candidate should be interpreted or replaced with another vector for the CPR codec mode.

[0207] a. In one example, (0, 0) is interpreted or replaced by (-W, 0), where W is the width of the current CU.

[0208] b. In one example, (0, 0) is interpreted or replaced by (-2W, 0), where W is the width of the current CU.

[0209] c. In one example, (0, 0) is interpreted or replaced by (0, -H), where H is the height of the current CU.

[0210] d. In one example, (0, 0) is interpreted or replaced by (0, -2H), where H is the current copper height.

[0211] e. In one example, (0, 0) is interpreted as or replaced by (-M, -N), where M and N are predefined constant values. In one example, (0, 0) is interpreted as (-M, -N), where M and N depend on the position of the block in the CTU.

[0212] f. The unit of MV disclosed above is integer pixel.

[0213] 13. It is proposed that when an invalid motion vector (Bx, By) is regarded as a BV (ie, when used in CPR coding mode), it should be interpreted as or replaced by another vector so that it can be valid.

[0214] a. In one example, (Bx, By) is interpreted or replaced by (Bx-M, By-N), where M and N are predefined constant values.

[0215] b. In one example, when (Bx-M, By-N) is still invalid, the above process can be repeated, that is, (Bx, By) is further interpreted as or replaced by (Bx-MM, By-NN).

[0216] c. In one example, the above process may be repeated until the modified motion vector becomes a valid block vector.

[0217] d. In one example, (Bx, By) is interpreted as pointing to the top left pixel of the valid reference area of ​​the CPR.

[0218] e. In one example, (Bx, By) is interpreted as pointing to the upper left pixel of the current CTU.

[0219] Transform Quantization Bypass Design

[0220] 14. It is proposed that when transform quantization bypass codec is used for the codec unit, the ALF should be bypassed.

[0221] 15. How the transquant_bypass flag is signaled / interpreted / used may depend on the color component.

[0222] a. In one example, how the transquant_bypass flag is signaled / interpreted / used may depend on whether separate segmentation tree structures for different color components are used.

[0223] b. In one example, when separate partition tree structures for luma and chroma components are applied, a first flag for luma blocks may be signaled, and a second flag for chroma blocks may be signaled independently.

[0224] c. In one example, when separate segmentation tree structures for three color components are applied, three flags may be signaled independently.

[0225] d. In one example, depending on the color component, the transform quantization bypass enable / disable flag is interpreted as different variables, e.g., transquant_bypass_luma and transquant_bypass_chroma, to control subsequent video encoding and decoding processes.

[0226] e. In a filtering process (such as a deblocking filter, a sampling adaptive loop filter, an adaptive loop filter), whether a block of a color component should be filtered may depend on the transquant_bypass flag associated with the color component.

[0227] f. Alternatively, even if a separate partitioning tree structure is enabled, the transquant_bypass flag is signaled only for the luma component, while for the chroma block, it can inherit the transquant_bypass flag from any position within a corresponding luma block (e.g., from the center position of the corresponding luma block).

[0228] g. Furthermore, alternatively, the restrictions on the block sizes / dimensions allowed for the transquant_bypass codec can be signaled for different color components, and the above restrictions can also be signaled separately for luma and chroma.

[0229] 16. An indication may be signaled to tell whether transform / quantization bypass mode is enabled at PPS / VPS / picture header / slice header / slice group header / CTU.

[0230] a. In one example, if a separate tree partitioning structure (eg, dual tree) is applied, the indication may be signaled multiple times for one picture / slice / slice group / CTU.

[0231] b. In one example, if a separate tree partitioning structure (eg, dual tree) is applied, the indication may be signaled separately for different color components.

[0232] CPR under the twin trees

[0233] 17. Confirm that the bitstream should follow the following rules: If a chroma block is encoded or decoded using CPR mode, the BV of the chroma block derived from a selected luminance block should be valid.

[0234] 18. Confirm that the bitstream should follow the following rules: If a chroma block is encoded and decoded using the CPR mode, the reference samples of the upper left and lower right samples of the chroma block identified by its BV should be valid.

[0235] 19. To derive the BV of a chroma block, multiple luminance blocks in the collocated luminance region of a chroma block may be examined.

[0236] a. In one example, multiple blocks may be checked to select one to derive the BV of the chroma block.

[0237] b. In one example, multiple blocks may be checked to select multiple blocks to derive the BV of the chroma block.

[0238] c. The checking order / position of the luma blocks may be predefined or signaled. In addition, alternatively, it may also depend on the dimensions of the blocks (eg, block size; block shape, etc.).

[0239] d. Once the block has been encoded and decoded using CPR mode, the inspection process can be terminated.

[0240] e. Once the block has been encoded and decoded with CPR mode and the corresponding BV is valid for the chroma block, the checking process can be terminated.

[0241] f. In one example, a block (e.g., Fig.12 The CR in the bitstream is decoded using CPR mode, and then the luminance blocks covering the corner samples are checked (e.g., a0 and a3; or a0, a1, a2, a3). Once the block is decoded using CPR mode and the BV of the derived chrominance block derived from the block is a valid BV, the checking process can be terminated. Confirm that the bitstream should follow the following rules: If a chrominance block is decoded using CPR mode, the BV of the chrominance block derived from a selected luminance block or multiple selected luminance blocks should be valid.

[0242] g. In one example, a block (e.g., Fig.12 The CR in the bitstream is determined whether the chrominance block is coded or decoded using the CPR mode, and then the luminance blocks covering the corner samples (e.g., a0 and a3; or a0, a1, a2, a3) are checked. Once the block is coded or decoded using the CPR mode, the checking process can be terminated. Confirm that the bitstream should follow the following rules: If a chrominance block is coded or decoded using the CPR mode, the BV of the chrominance block derived from a selected luminance block or multiple selected luminance blocks should be valid.

[0243] h. Alternatively, first check a block covering the center of the collocated brightness area (e.g. Fig.12 The CR in the bitstream is decoded using CPR mode and returns a valid BV, and then checks the luminance blocks covering the corner samples (for example, a0 and a3; or a0, a1, a2, a3). Once the block is decoded using CPR mode and the BV of the chrominance block derived from the block is a valid BV, the checking process can be terminated. Confirm that the bitstream should follow the following rules: If a chrominance block is encoded and decoded using CPR mode, the BV of the chrominance block derived from a selected luminance block or multiple selected luminance blocks should be valid.

[0244] 20. When the check process cannot return a valid BV from the luma block (e.g., the method mentioned in item 19), an additional default BV will be checked to derive the BV for the chroma block.

[0245] a. In one example, the default BV for a chroma block is (-w, 0), where w is the width of the chroma block.

[0246] b. In one example, the default BV for a chroma block is (-2w, 0), where w is the width of the chroma block.

[0247] c. In one example, the default BV for a chroma block is (0, -h), where h is the height of the chroma block.

[0248] d. In one example, the default BV of a chroma block is (0, -2h), where h is the height of the chroma block.

[0249] e. In one example, the default BV list is {(-w, 0), (0, -h)}. The BVs in the list will be checked in turn to get a valid BV.

[0250] f. In one example, the default BV list is {(-2w, 0), (0, -2h), (-w, 0), (0, -h)}. The BVs in the list will be checked in turn to get a valid BV.

[0251] 21. It is proposed to set a valid BV for all chroma sub-blocks regardless of whether the BV derived from the corresponding luminance block is valid.

[0252] a. In one example, when the corresponding luma block of a sub-block cannot return a valid BV, other BVs (eg, a default BV) may be used instead.

[0253] b. In one example, the BV found using the above method in item 19 can be used as the default BV of the sub-block.

[0254] c. Alternatively, items 19 and 20 can be used together to find the default BV.

[0255] 22. For the above example, when BV is valid, the following conditions should be true:

[0256] a. All samples within the reference block identified by the BV should be within the restricted search range (eg, should be within the same CTU in the current VVC design).

[0257] b. All samples within the reference block identified by BV have been reconstructed.

[0258] 23. When the check process cannot return a BV from the luma block (e.g., the method mentioned in item 19), an additional default BV will be checked to derive the BV for the chroma block.

[0259] a. In one example, when Fig.12 When the luma sub-block CR positions shown are not encoded with CPR, an additional default BV is checked to derive the BV for the chroma blocks.

[0260] b. In one example, when Fig.12 When all luma sub-blocks in the shown CR, a0, a1, a2, a3 positions are not decoded with CPR, an additional default BV is checked to derive the BV of the chroma blocks.

[0261] c. In one example, the default BV for a chroma block is (-w, 0), where w is the width of the chroma block.

[0262] d. In one example, the default BV for a chroma block is (-2w, 0), where w is the width of the chroma block.

[0263] e. In one example, the default BV for a chroma block is (0, -h), where h is the height of the chroma block.

[0264] f. In one example, the default BV of a chroma block is (0, -2h), where h is the height of the chroma block.

[0265] g. In one example, the default BV list is {(-w, 0), (0, -h)}. The BVs in the list will be checked in turn to get a valid BV.

[0266] h. In one example, the default BV list is {(-2w, 0), (0, -2h), (-w, 0), (0, -h)}. The BVs in the list will be checked in turn to get a valid BV.

[0267] Fig.13 1 is a block diagram of a video processing device 1300. Device 1300 may be used to implement one or more methods described herein. Device 1300 may be implemented in a smartphone, tablet computer, computer, Internet of Things (IoT) receiver, etc. Such 1300 may include one or more processors 1302, one or more memories 1304, and video processing hardware 1306. Processor (s) 1302 may be configured to implement one or more methods described herein. Memory (s) 1304 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1306 may be used to implement some of the techniques described herein in hardware circuits.

[0268] Fig.14 1400 is a flow chart of an example method 1400 for video processing. The method 1400 includes: applying (1402) an intra codec tool or an inter codec tool to a current video block of a current video picture for conversion between a current video block and a bitstream representation of the current video block, the current video block being referenced by the intra codec tool or the inter codec tool from a reference video block at least partially located in the current video picture; determining (1404) one or more padding values ​​during the conversion, wherein the one or more padding values ​​are signaled in the bitstream representation; and performing (1406) the conversion using the one or more padding values ​​and the intra codec tool or the inter codec tool.

[0269] Fig.15 is a flow chart of an example method 1500 of video processing. The method 1500 includes: determining (1502) one or more padding values ​​during conversion between a current block and a bitstream representation of the current block; and performing (1504) the conversion based at least on the one or more padding values.

[0270] Fig.16 1 is a flow chart of an example method 1600 for video processing. The method 1600 includes: determining (1602) a first candidate list construction method for a first video block of a video having a first codec mode during a first conversion between the first video block and a bitstream representation of the video; and performing (1604) the first conversion based on at least a first candidate list obtained according to the first candidate list construction method; wherein the conversion of the first video block is based on samples in a current picture, and the first candidate list construction method is different from a second candidate list construction method, and during a second conversion between a second video block of the video having a second codec mode and the bitstream representation of the video, the second candidate list construction method is applied on the second video block.

[0271] Fig.17 1700 is a flow chart of an example method 1700 for video processing. The method 1700 includes: during a conversion between a current block and a bitstream representation of the current block, determining (1702) that the current block is encoded with a transform quantization bypass codec mode; and in response to the current block being encoded with the transform quantization bypass codec mode, performing (1704) the conversion between the current block and the bitstream representation of the current block without filtering based on an adaptive loop filter, wherein the transform quantization bypass codec mode is a codec mode in which a block is encoded without using one or more of transform, quantization, and loop filtering.

[0272] Fig.18 1800 is a flow chart of an example method 1800 for video processing. The method 1800 includes: performing (1802) processing of one or more transform quantization bypass flags based on color components of the current block during conversion between a current block and a bitstream representation of the current block, wherein the transform quantization bypass flags are associated with a transform quantization bypass codec mode, wherein the transform quantization bypass codec mode is a codec mode in which a block is encoded and decoded without using one or more of transform, quantization, and loop filtering; and performing (1804) the conversion based on the processing.

[0273] It should be appreciated that several techniques have been disclosed that will benefit video encoder and decoder embodiments incorporated into video processing devices such as smartphones, laptops, desktops, and similar devices by allowing the use of intra-picture video codecs such as CPR. Some techniques and embodiments may be described in the following clause-based format.

[0274] 1. A video processing method, comprising: applying an intra codec or an inter codec to the current video block of a current video picture for conversion between a current video block and a bitstream representation of the current video block, referring to the current video block from a reference video block at least partially located in the current video picture by the intra codec or the inter codec; determining one or more padding values ​​during the conversion, wherein the one or more padding values ​​are signaled in the bitstream representation; and performing the conversion using the one or more padding values ​​and the intra codec or the inter codec.

[0275] 2. A method according to clause 1, wherein the bitstream representation includes the one or more padding values ​​at a sequence parameter set level, or a slice level, or a slice group level, or a slice level, or a codec tree unit level, or a video parameter set level, or a picture parameter set level, or a picture header level, or a codec tree unit row level, or a region level, and the region covers multiple codec units.

[0276] 3. A method according to any one of clauses 1 to 2, wherein the one or more filler values ​​are included in the bitstream representation in a quantized format.

[0277] 4. A method according to any one of clauses 1 to 3, wherein the one or more filler values ​​are included in the bit stream representation using a binarization scheme, the binarization scheme being (a) a unary code, or (b) a truncated unary code, or (c) a fixed-length code, or (d) an exponential Golomb code.

[0278] 5. A method according to any of clauses 1 to 4, wherein the one or more fill values ​​are adaptively dependent on video characteristics.

[0279] 6. A method according to clause 5, wherein the adaptive dependency is indicated in the bitstream representation at a sequence parameter set level, or a slice level, or a slice group level, or a slice level, or a codec tree unit level, or a video parameter set level, or a picture parameter set level, or a picture header level, or a codec tree unit row level, or a region level, wherein the region covers multiple codec units.

[0280] 7. A method according to any one of clauses 1 to 6, wherein the bitstream representation includes an identification of the one or more fill values, and the identification of the one or more fill values ​​is included in a fill value list contained in the bitstream representation.

[0281] 8. A method according to any of clauses 1 to 7, wherein the one or more fill values ​​include different fill values ​​for at least some color or brightness components.

[0282] 9. A method according to any of clauses 1 to 8, wherein the one or more padding values ​​comprise different padding values ​​for at least some video image regions.

[0283] 10. A video processing method, comprising:

[0284] During a conversion between a current video block and a bitstream representation of the current video block, determining an intra candidate list for an intra codec mode of the current video block; and performing the conversion between the current video block and the bitstream representation using the intra candidate list, wherein the intra candidate list for the intra codec mode is different from a candidate list for the inter codec mode of the current video block; and wherein the candidate list is one of a Merge list, a history-based motion vector predictor list, or an affine Merge list.

[0285] 11. A method according to clause 10, wherein the intra candidate list uses one of the following motion vectors as a default fill candidate:

[0286] (a)(-W, 0), where W is the width of the current video block; or

[0287] (b)(-2W, 0), or

[0288] (c)(0, -H), where H is the height of the current video block, or

[0289] (d) (0, -2H) or

[0290] (e)(-M, -N), where M and N are predefined numbers.

[0291] 12. The method of clause 11, wherein W, H, M and N are integers.

[0292] 13. A method according to any of clauses 10 to 12, wherein said using said intra candidate list comprises: replacing or interpreting an invalid motion vector (Bx, By) in said intra candidate list as one of:

[0293] (a)(Bx-M, By-N), where M and N are predefined integers, or

[0294] (b) repeatedly subtract M and N from Bx and By until a valid motion vector is obtained, or

[0295] (c) repeatedly subtract M and N from Bx and By until a valid block vector is obtained, or

[0296] (d) interpret (Bx, By) as referring to the upper left pixel of the valid reference area of ​​the current video block, or

[0297] (e) Interpret (Bx, By) as pointing to the upper left pixel of the codec tree unit of the current video block.

[0298] 14. A video processing method, comprising: performing a determination that a codec mode of a current video block is a transform quantization bypass codec mode, wherein the current video block is encoded into a bitstream representation by omitting a transform step and a quantization step; and based on the determination, performing a conversion between the current video block and the bitstream representation according to the transform quantization bypass codec mode, wherein the conversion is performed without filtering based on an adaptive loop filter.

[0299] 15. The method of clause 14, wherein one or more flags in the bitstream representation correspond to the transform quantization bypass codec mode.

[0300] 16. The method of clause 15, wherein the number of the one or more flags corresponds to the number of luma or color components of the current video block that use separate partitioning structures in the bitstream representation.

[0301] 17. The method of clause 15, wherein a first flag of the one or more flags is signaled for a luma component of the current video block and a second flag of the one or more flags is signaled for two color components of the current video block.

[0302] 18. A method according to any of clauses 14 to 17, wherein the bitstream representation further comprises one or more transquant_bypass flags indicating the applicability of a pulse codec modulation codec mode for luma and colour components of the current video block.

[0303] 19. The method of clause 16, wherein chroma components inherit corresponding transquant_bypass flags from the position of the luma component of the current video block.

[0304] 20. The method of clause 19, wherein the current position corresponds to a center position of the luma component of the current video block.

[0305] 21. The method of clause 16, wherein the one or more flags are included at a picture parameter set level, or a video parameter set level, or a picture header level, or a slice header level, or a slice group header level, or a codec tree unit level.

[0306] 22. A video processing method, comprising:

[0307] performing a determination that a current video block that is a chroma block is coded using a current picture reference mode; and

[0308] Based on the determination, performing a conversion between the current video block and a bitstream representation of the current video block,

[0309] The conversion uses at least one of the following rules:

[0310] (1) the block vector of the chrominance block derived from the selected luminance block is valid, or

[0311] (2) The reference samples of the upper left and lower right samples of the chroma block identified by its block vector are valid.

[0312] 23. A video processing method, comprising:

[0313] selecting at least one luma block from a plurality of luma blocks in a collocated luma region of a current video block that is a chroma block; and

[0314] deriving a block vector for the chrominance block based on the at least one luminance block; and

[0315] Based on the block vector, a conversion between the current video block and a bitstream representation of the current video block is performed.

[0316] 24. The method of clause 23, wherein the at least one luma block comprises a plurality of luma blocks.

[0317] 25. The method of clause 23, wherein the order in which the plurality of luma blocks are examined to select the at least one luma block is predefined, signaled, or based on a size or shape of the current video block.

[0318] 26. The method of clause 23, wherein the current video block is encoded with a current picture reference mode, and wherein the block vector is valid.

[0319] 27. A method according to any of clauses 22 to 26, wherein a reference block is identified by the block vector and all samples within the reference block are within a restricted search range.

[0320] 28. The method of clause 27, wherein the restricted search scope is a codec tree unit.

[0321] 29. A method according to any of clauses 22 to 26, wherein a reference block is identified by the block vector and all samples within the reference block are reconstructed.

[0322] 30. A method according to clause 23, wherein the block vector is used as a default block vector for a sub-block of the chroma block.

[0323] 31. A video processing method, comprising:

[0324] determining that a block vector for a current video block that is a chroma block cannot be derived based on a luma block in a collocated luma region of the current video block;

[0325] Based on the determination, selecting a default block vector as the block vector for the chroma block; and

[0326] Based on the block vector, a conversion between the current video block and a bitstream representation of the current video block is performed.

[0327] 32. A method according to clause 31, wherein the default block vector is (-w, 0), and wherein w is the width of the chroma block.

[0328] 33. A method according to clause 31, wherein the default block vector is (0, -h), and wherein h is the height of the chroma block.

[0329] 34. A method according to clause 31, wherein the default block vector is (0, -2h), and wherein h is the height of the chroma block.

[0330] 35. A method according to clause 31, wherein the default block vector is selected from a list of default block vectors including (-w, 0) and (0, -h), and wherein w and h are the width and height of the chroma block respectively.

[0331] 36. A method according to clause 31, wherein the default block vector is selected from a list of default block vectors including (-2w, 0), (0, -2h), (-w, 0) and (0, -h), and wherein w and h are the width and height of the chroma block, respectively.

[0332] 37. A method according to any one of clauses 1 to 36, wherein the converting comprises: generating pixel values ​​of the current video block from the bitstream representation.

[0333] 38. A method as described in any of clauses 1 to 36, wherein the converting comprises: generating the bitstream representation from the current video block.

[0334] 39. A video encoder apparatus comprising a processor configured to implement the method of any of clauses 1 to 38.

[0335] 40. A video decoder device comprising a processor configured to implement the method of any of clauses 1 to 38.

[0336] 41. A computer readable medium having stored thereon code which, when executed, causes a processor to implement the method of any one of clauses 1 to 38.

[0337] Some techniques and embodiments may be described in another clause-based format as follows.

[0338] 1. A video processing method, comprising:

[0339] determining one or more padding values ​​during conversion between a current block and a bitstream representation of the current block; and

[0340] The converting is performed based at least on the one or more padding values.

[0341] 2. A method according to clause 1, wherein the one or more padding values ​​are signaled in the bitstream representation at one or more levels, and the one or more padding values ​​are the same for a level of the one or more levels.

[0342] 3. The method of clause 2, wherein the one or more levels include one or more of the following:

[0343] Sequence parameter set level, slice level, slice group level, slice level, codec tree unit level, video parameter set level, picture parameter set level, picture header level, codec tree unit row level, and region level corresponding to multiple codec tree units or multiple codec units.

[0344] 4. A method according to any of clauses 1 to 3, wherein the one or more fill values ​​of the first level are predicted from the one or more fill values ​​of the second level based on a signalled difference between the values ​​of the two levels.

[0345] 5. A method according to any one of clauses 1 to 4, wherein the one or more fill values ​​are quantized.

[0346] 6. The method of clause 5, wherein the one or more padding values ​​are quantized using one or more of:

[0347] Unary codes, truncated unary codes, fixed-length codes, and exponential-Golomb codes.

[0348] 7. The method according to any one of clauses 1 to 6, further comprising:

[0349] The one or more fill values ​​are adaptively updated.

[0350] 8. The method of clause 7, wherein adaptively updating the one or more padding values ​​comprises:

[0351] updating the one or more fill values ​​according to the first flag,

[0352] The first flag indicates whether to use a default filling value or a newly sent filling value.

[0353] 9. A method according to any of clauses 1 to 8, wherein the index of the one or more padding values ​​in the set is signaled in one or more of each codec tree unit, each slice, each slice, and each picture.

[0354] 10. A method according to clause 9, wherein the signalling is a set of the one or more padding values.

[0355] 11. A method according to any one of clauses 1 to 10, wherein the one or more fill values ​​are from a fill value list.

[0356] 12. The method according to any one of clauses 1 to 11, further comprising:

[0357] A fill value list is maintained based on the one or more fill values.

[0358] 13. The method of clause 12, wherein maintaining a list of fill values ​​based on the one or more fill values ​​comprises:

[0359] A new fill value for the conversion is inserted into the fill value list.

[0360] 14. The method of clause 13, wherein inserting a new fill value for the conversion into the fill value list comprises:

[0361] Insert the new padding value at the head or tail of the padding value list.

[0362] 15. The method of clause 12, wherein maintaining a list of fill values ​​based on the one or more fill values ​​comprises:

[0363] When a padding value for the conversion is in the padding value list before the conversion, the padding value list is maintained unchanged.

[0364] 16. The method of clause 12, wherein maintaining a list of fill values ​​based on the one or more fill values ​​comprises:

[0365] When the padding value for the conversion is in the padding value list before the conversion, the padding value for the conversion is placed at the head or the tail of the padding value list.

[0366] 17. The method according to any one of clauses 12 to 16, further comprising:

[0367] The fill value list is initialized with one or more predefined values.

[0368] 18. The method of clause 17, wherein the one or more predefined values ​​include one or more of 0, middle gray, and maximum level.

[0369] 19. The method of clause 17, wherein the one or more predefined values ​​are sent at one or more of a sequence parameter set level, a picture header level, a slice level, and a slice group level.

[0370] 20. A method according to any of clauses 12 to 19, wherein a second flag is signalled to indicate that the one or more padding values ​​belong to the padding value list.

[0371] 21. A method according to any of clauses 12 to 20, wherein an index is sent to indicate the one or more padding values ​​in the list of padding values ​​used for the conversion.

[0372] 22. A method according to any one of clauses 1 to 21, wherein the conversion is performed at least based on the one or more padding values ​​in intra block copy (IBC) mode, inter coding mode or intra coding mode.

[0373] 23. A method according to clause 22, wherein the one or more fill values ​​used in different modes are different.

[0374] 24. A method according to any one of clauses 1 to 23, wherein the one or more fill values ​​for different color components are different.

[0375] 25. A method according to any of clauses 1 to 24, wherein the one or more fill values ​​signalled for different colour components are different.

[0376] 26. A method according to any one of clauses 1 to 25, wherein the one or more fill values ​​for different regions are different.

[0377] 27. A video processing apparatus comprising a processor configured to implement the method of any one of clauses 1 to 26.

[0378] 28. The device of clause 27, wherein the device is a video encoder.

[0379] 29. The device of clause 27, wherein the device is a video decoder.

[0380] 30. A computer-readable medium having recorded thereon a program including codes, the program causing a processor to implement the method of any one of clauses 1 to 26.

[0381] Some techniques and embodiments may be described in another clause-based format as follows.

[0382] 1. A video processing method, comprising:

[0383] determining, during a first conversion between a first video block of a video and a bitstream representation of the video, the first candidate list construction method for the first video block having a first codec mode;

[0384] performing the first conversion based at least on the first candidate list obtained according to the first candidate list construction method;

[0385] The conversion of the first video block is based on samples in a current picture, and the first candidate list construction method is different from a second candidate list construction method, and the second candidate list construction method is applied to the second video block during a second conversion between a second video block of the video having a second codec mode and the bitstream representation of the video.

[0386] 2. A method according to clause 1, wherein the reference block used for prediction in the first coding mode refers to the same picture as the current block.

[0387] 3. A method according to clause 1 or 2, wherein the first coding mode is a current picture reference (CPR) mode or an intra block copy (IBC) mode.

[0388] 4. A method according to any one of clauses 1 to 3, wherein the first candidate list comprises one or more of the following:

[0389] IBC Merge candidate list, IBC spatial block vector candidate list, IBC history-based block vector candidate list.

[0390] 5. The method according to any one of clauses 1 to 4, wherein the second candidate list obtained according to the second candidate list construction method comprises one or more of the following:

[0391] Inter-frame Merge candidate list, inter-frame spatial domain Merge candidate list, inter-frame pairwise average Merge candidate list, inter-frame zero motion vector Merge candidate list, and inter-frame history-based motion vector candidate list.

[0392] 6. A method according to any of clauses 1 to 5, wherein the second codec mode comprises one or more of the following:

[0393] Non-IBC mode, Merge non-IBC mode, Advanced Motion Vector Prediction (AMVP) non-IBC mode, History-based Motion Vector Prediction (HMVP) mode, Affine mode.

[0394] 7. A method according to any of clauses 1 to 6, wherein the zero motion vector which is a default fill candidate in the first candidate list is replaced with one of the following motion vectors:

[0395] (a)(-W, 0), or

[0396] (b)(-2W, 0), or

[0397] (c)(0, -H) or

[0398] (d) (0, -2H) or

[0399] (e)(-M, -N),

[0400] Wherein, W is the width of the current block, H is the height of the current video block, and M and N are predefined numbers.

[0401] 8. A method according to clause 7, wherein M and N depend on the position of the current block in a codec tree unit (CTU).

[0402] 9. A method according to any of clauses 1 to 8, wherein performing the converting based at least on the first candidate list comprises:

[0403] Replace the invalid motion vectors in the first candidate list with valid motion vectors; and

[0404] The converting is performed based at least on the effective motion vector.

[0405] 10. A method according to clause 9, wherein replacing the invalid motion vector in the first candidate list with a valid motion vector comprises:

[0406] An updating step of replacing the invalid motion vector with an updated motion vector by respectively subtracting M and N from the two components of the invalid motion vector.

[0407] 11. A method according to clause 10, wherein replacing the invalid motion vector in the first candidate list with a valid motion vector comprises:

[0408] The updating step is repeated until the updated motion vector is the valid motion vector.

[0409] 12. A method according to any of clauses 9 to 11, wherein replacing invalid motion vectors in the first candidate list with valid motion vectors comprises:

[0410] The invalid motion vector is interpreted as pointing to the upper left pixel of the valid reference area of ​​the current block.

[0411] 13. A method according to any of clauses 9 to 12, wherein replacing invalid motion vectors in the first candidate list with valid motion vectors comprises:

[0412] The invalid motion vector is interpreted as pointing to an upper left pixel of a codec tree unit (CTU) of the current block.

[0413] 14. A video processing device comprising a processor configured to implement the method of any one of clauses 1 to 13.

[0414] 15. The device of clause 14, wherein the device is a video encoder.

[0415] 16. The device of clause 14, wherein the device is a video decoder.

[0416] 17. A computer-readable medium having recorded thereon a program including codes, the program causing a processor to implement the method of any one of clauses 1 to 13.

[0417] Some techniques and embodiments may be described in another clause-based format as follows.

[0418] 1. A video processing method, comprising:

[0419] During conversion between a current block and a bitstream representation of the current block, determining that the current block is encoded using a transform quantization bypass codec mode; and

[0420] In response to the current block being encoded with the transform quantization bypass codec mode, performing conversion between the current block and a bitstream representation of the current block without filtering based on an adaptive loop filter, wherein the transform quantization bypass codec mode is a codec mode in which a block is encoded and decoded without using one or more of transform, quantization, and loop filtering.

[0421] 2. A method according to clause 1, wherein one or more transform quantization bypass flags are signaled, interpreted or used based on color components.

[0422] 3. A video processing method, comprising:

[0423] performing processing of one or more transform quantization bypass flags based on color components of the current block during conversion between a current block and a bitstream representation of the current block, wherein the transform quantization bypass flags are associated with a transform quantization bypass codec mode, wherein the transform quantization bypass codec mode is a codec mode in which a block is encoded or decoded without using one or more of transform, quantization, and loop filtering;

[0424] The converting is performed based on the processing.

[0425] 4. The method of clause 3, wherein the processing of the one or more transform quantization bypass flags comprises at least one of:

[0426] signaling the one or more transform quantization bypass flags;

[0427] interpreting the one or more transform quantization bypass flags;

[0428] The one or more transforms are used to quantize the bypass flag.

[0429] 5. A method according to any of clauses 2 to 4, wherein the processing of the one or more transform quantization bypass flags depends on whether different partitioning tree structures are used for different color components of the current block.

[0430] 6. A method according to any one of clauses 2 to 5, wherein, when different partitioning tree structures are used for the different color components of the current block, the one or more transform quantization bypass flags correspond to the different color components of the current block respectively, and the one or more transform quantization bypass flags are signaled independently.

[0431] 7. A method according to any of clauses 5 to 6, wherein the different color components include luminance and chrominance components.

[0432] 8. A method according to any of clauses 2 to 7, wherein the one or more transform quantization bypass flags are interpreted as different variables based on the color component.

[0433] 9. A method according to any one of clauses 2 to 8, wherein a transform quantization bypass flag corresponding to a color component indicates whether the current block of the color component is filtered during the conversion between the current block and the bitstream representation of the current block.

[0434] 10. A method according to any one of clauses 2 to 9, wherein one transform quantization bypass flag corresponding to one color component is signaled, and other transform quantization bypass flags corresponding to other color components are inherited from the position within the current block of the one color component.

[0435] 11. The method of clause 10, wherein the position within the current block is a center position.

[0436] 12. A method according to any of clauses 1 to 11, wherein whether the current block of color components is encoded in transform quantization bypass coding mode depends on restriction parameters signaled separately for the different color components.

[0437] 13. A method according to clause 12, wherein the restriction parameter indicates a size or dimension of the current block of the color component.

[0438] 14. A method according to any of clauses 2 to 13, wherein the one or more transform quantization bypass flags are signalled at one or more levels.

[0439] 15. The method of clause 14, wherein the one or more stages include one or more of:

[0440] Picture parameter set level, video parameter set level, picture header level, slice header level, slice group level, and codec tree unit level.

[0441] 16. A method according to clause 14 or 15, wherein the one or more transform quantization bypass flags are signaled multiple times for a picture, or a slice, or a slice group, or a codec tree unit when the different partitioning tree structure is used.

[0442] 17. A video processing device comprising a processor configured to implement the method of any one of clauses 1 to 16.

[0443] 18. The device of clause 17, wherein the device is a video encoder.

[0444] 19. The device of clause 17, wherein the device is a video decoder.

[0445] 20. A computer-readable medium having recorded thereon a program including codes, the program causing a processor to implement the method of any one of clauses 1 to 19.

[0446] It should be understood that several methods and apparatus are disclosed for performing video processing including video encoding (e.g., generating a bitstream from pixel values ​​of a video) or video decoding (e.g., generating video pixels from a bitstream). The disclosed techniques may be incorporated into embodiments where CPR is used for video encoding and decoding. The bitstream representation of a video block may include contiguous or non-contiguous bits (e.g., header fields and network abstraction layer or NAL fields).

[0447] The implementation and functional operation of the subject matter described herein can be implemented in various systems, digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more thereof. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for data processing devices to execute or control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. The propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0448] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0449] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0450] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory can be supplemented by dedicated logic circuits, or merged into dedicated logic circuits.

[0451] Although this patent document contains many details, they should not be construed as limitations on any subject matter or the scope of the claims, but rather as descriptions of features of particular embodiments of particular inventions. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially claimed to be so, in some cases, one or more features in the claim combination may be removed from the combination, and the claim combination may be directed to subcombinations or variations of subcombinations.

[0452] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0453] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: During conversion between a current block and a bitstream representation of the current block, determining one or more padding values ​​for the reference block or the prediction block in the event that the reference block overlaps the current block or the reference block is outside a picture; as well as performing said converting based at least on said one or more fill values, wherein the one or more padding values ​​are signaled in the bitstream representation at one or more levels, and the one or more padding values ​​are the same for a level of the one or more levels, Therein, the one or more fill values ​​of the first level are predicted from the one or more fill values ​​of the second level based on a signaled difference between the values ​​of the two levels.

2. The method according to claim 1, wherein: The one or more levels include one or more of the following: Sequence parameter set level, slice level, slice group level, slice level, codec tree unit level, video parameter set level, picture parameter set level, picture header level, codec tree unit row level, and region level corresponding to multiple codec tree units or multiple codec units.

3. The method according to claim 1, wherein: The one or more fill values ​​are quantized.

4. The method according to claim 3, wherein: The one or more padding values ​​are quantized using one or more of the following: Unary codes, truncated unary codes, fixed-length codes, and exponential-Golomb codes.

5. The method according to claim 1, further comprising: The one or more fill values ​​are adaptively updated.

6. The method according to claim 5, wherein: Adaptively updating the one or more fill values ​​includes: updating the one or more fill values ​​according to the first flag, The first flag indicates whether to use a default filling value or a newly sent filling value.

7. The method according to claim 1, wherein: The index of the one or more padding values ​​in the set is signaled for one or more of each codec tree unit, each slice, each slice, and each picture.

8. The method according to claim 7, wherein: The set of one or more padding values ​​is signaled.

9. The method according to claim 1, wherein: The one or more fill values ​​are from a fill value list.

10. The method according to claim 1, further comprising: A fill value list is maintained based on the one or more fill values.

11. The method according to claim 10, wherein: Maintaining a fill value list based on the one or more fill values ​​comprises: A new fill value for the conversion is inserted into the fill value list.

12. The method according to claim 11, wherein: Inserting a new fill value for the conversion into the fill value list comprises: Insert the new padding value at the head or tail of the padding value list.

13. The method according to claim 10, wherein: Maintaining a fill value list based on the one or more fill values ​​comprises: When a padding value for the conversion is in the padding value list before the conversion, the padding value list is maintained unchanged.

14. The method according to claim 10, wherein: Maintaining a fill value list based on the one or more fill values ​​comprises: When the padding value for the conversion is in the padding value list before the conversion, the padding value for the conversion is placed at the head or the tail of the padding value list.

15. The method according to claim 10, further comprising: The fill value list is initialized with one or more predefined values.

16. The method according to claim 15, wherein: The one or more predefined values ​​include one or more of 0, middle gray, and maximum value level.

17. The method according to claim 15, wherein: The one or more predefined values ​​are sent at one or more of a sequence parameter set level, a picture header level, a slice level, and a slice group level.

18. The method according to claim 10, wherein: A second flag is signaled to indicate that the one or more padding values ​​belong to the padding value list.

19. The method according to claim 10, wherein: An index is sent to indicate the one or more padding values ​​in the list of padding values ​​for the conversion.

20. The method according to claim 1, wherein: In an intra block copy (IBC) mode, an inter coding mode or an intra coding mode, the converting is performed based at least on the one or more padding values.

21. The method according to claim 20, wherein: The one or more fill values ​​used in different modes are different.

22. The method according to claim 1, wherein: The one or more fill values ​​are different for different color components.

23. The method according to claim 1, wherein: The one or more fill values ​​signaled for different color components are different.

24. The method according to claim 1, wherein: The one or more fill values ​​are different for different regions.

25. A video processing device, comprising a processor, wherein the processor is configured to implement the method according to any one of claims 1 to 24.

26. The device of claim 25, wherein the device is a video encoder.

27. The device of claim 25, wherein the device is a video decoder.

28. A computer-readable medium having recorded thereon a program including codes, the program causing a processor to implement the method according to any one of claims 1 to 24.

Images