Extension of intra-frame coding mode in video coding

By introducing affine motion compensation and extended intra prediction mode sets in video encoding, the problems of high video encoding bandwidth and low hardware processing efficiency are solved, and more efficient video encoding and processing performance are achieved.

CN114342408BActive Publication Date: 2024-11-19DOUYIN VISION CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080062132.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-08-26
Filing Date
2020-08-26
Publication Date
2024-11-19
Estimated Expiration
2040-08-26

AI Technical Summary

Technical Problem

Existing video encoding technologies occupy a large amount of bandwidth in the Internet and digital communication networks, and bandwidth demand will continue to grow as the number of devices increases, with existing intra-block replication (IBC) and palette modes in encoding efficiency and hardware processing throughput There are quantitative challenges.

Method used

A video processing method is proposed to extend the intra prediction mode set by using affine motion compensation in video encoding and decoding, including new encoding methods such as palette mode, intra-block copy mode and quantized residual block differential pulses code modulation mode and corresponding conversion and adjustments are performed during encoding and decoding to optimize encoding performance and hardware processing efficiency.

Benefits of technology

Improves the encoding efficiency of video encoding, reduces bandwidth usage, and improves hardware processing throughput, especially significantly improves performance when processing screen content and natural video.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114342408B_ABST
    Figure CN114342408B_ABST
Patent Text Reader

Abstract

Extension of intra-frame coding modes in video coding is described. One example is a method for video processing, the method comprising: for a conversion between a current video block of a video and a bitstream representation of the video, generating an extended intra-frame prediction mode (IPM) set associated with the current video block by modifying an existing IPM set to include at least one new coding method, wherein the new coding method is different from a coding method in the existing IPM and is treated as an IPM; and performing the conversion based on the extended IPM set.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] In accordance with the applicable provisions of the Patent Law and / or the Paris Convention, this application promptly claims priority to and the benefits of International Patent Application No. PCT / CN2019 / 102586 filed on August 26, 2019. The entire disclosure of International Patent Application No. PCT / CN2019 / 102586 is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] The patent document relates to video encoding technology, decoding technology, devices and systems. Background Art

[0004] Despite advances in video compression, digital video still consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention

[0005] This document discloses methods, systems and devices related to using affine motion compensation in video encoding and decoding.

[0006] In one representative aspect, a video processing method is disclosed. The method includes: performing a conversion between a current video block and an encoded representation of the current video block, wherein during the conversion process, at least one encoding prediction method is considered as an intra-frame prediction mode (IPM) in an IPM set without using a flag to signal the at least one encoding prediction mode.

[0007] In yet another aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block and an encoded representation of the current video block, wherein during the conversion, at least one encoding prediction method is considered as an intra-frame prediction mode (IPM) in a most probable mode (MPM) list without using a flag to signal the at least one encoding prediction mode.

[0008] In yet another aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block and an encoded representation of the current video block, wherein during the conversion, in a palette mode, a context of a syntax element associated with the current video block is derived from a neighboring block used to derive an MPM list; and aligning a position of the neighboring block with a position of the current video block.

[0009] In yet another aspect, another video processing method is disclosed. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, generating an extended intra-frame prediction mode (IPM) set associated with the current video block by modifying an existing IPM set to include at least one new encoding method, wherein the new encoding method is different from the encoding method in the existing IPM and is regarded as an IPM; and performing the conversion based on the extended IPM set.

[0010] In yet another aspect, another video processing method is disclosed. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, generating a modified most probable intra-frame mode (MPM) list associated with the current video block, wherein the modified MPM list includes at least one new encoding method, the at least one new encoding method is different from an existing intra-frame prediction mode (IPM) in an existing IPM set and is considered to be an IPM; and performing the conversion based on the modified MPM list.

[0011] In yet another aspect, another video processing method is disclosed. The method includes: for conversion between a chroma block of a video and a bitstream representation of the chroma block, determining whether the chroma block is a screen content block; in response to a determination indicating that the chroma block is a screen content block, modifying an intra-frame prediction process and / or a transform process of the chroma block; and performing the conversion based on the modified intra-frame prediction process and / or transform process.

[0012] In yet another aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block of a video and a bitstream representation of the video, wherein during the conversion, the position of a left neighboring block and / or an upper neighboring block used in a context derivation of one or more syntax elements is aligned with the position of a left neighboring block and / or an upper neighboring block used in a most probable intra mode (MPM) derivation.

[0013] In yet another aspect, another video processing method is disclosed. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, using a plurality of context signals to indicate the use of a palette mode; and performing the conversion based on the indication.

[0014] In yet another representative aspect, the various techniques described herein may be embodied as a computer program product stored on a non-transitory computer-readable medium. The computer program product includes program code for executing the methods described herein.

[0015] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0016] The details of one or more embodiments are set forth in the accompanying drawings, the accompanying drawings, and the following description. Other features will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 An example of intra block copy is shown.

[0018] Figure 2 An example of a block encoded in palette mode is shown.

[0019] Figure 3 An example of using a palette predictor to signal palette entries is shown.

[0020] Figure 4 Examples of horizontal and vertical traversal scans are shown.

[0021] Figure 5 An example of encoding of a palette index is shown.

[0022] Figure 6 An example of an intra prediction mode is shown.

[0023] Figure 7 Neighbors of the current block are shown.

[0024] Figure 8 An example of the locations of sample points used to derive model parameters is shown.

[0025] Figures 9A-9D The definition of the samples used by PDPC applied to diagonal and adjacent intra modes is shown.

[0026] Fig.10 An example of four reference lines adjacent to a prediction block is shown.

[0027] Fig.11A Examples of sub-partitioning for 4x8 and 8x4 CUs are shown.

[0028] Fig. 11B Examples of sub-partitions of the CU other than 4x8, 8x4, and 4x4 are shown.

[0029] Fig.12 An example of a matrix-weighted intra prediction process is shown.

[0030] Fig.13A and 13B A 4-parameter affine model and a 6-parameter affine model are shown.

[0031] Fig.14 An example of the affine MVF for each sub-block is shown.

[0032] Fig.15A and15B An example of a SCIPU is shown.

[0033] Fig.16 An example of luma regions for a 4:2:0 color format is shown.

[0034] Fig.17 Examples of the left and top neighbors of the current block are shown.

[0035] Fig.18 is a block diagram showing an example of the structure of a computer system or other control device that can be used to implement various parts of the disclosed technology.

[0036] Fig.19 A block diagram of an example embodiment of a mobile device that can be used to implement various portions of the disclosed technology is shown.

[0037] Fig. 20 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

[0038] Fig.21 A flow chart of an example method for video processing is shown.

[0039] Fig. 22 A flow chart of an example method for video processing is shown.

[0040] Fig.23 A flow chart of an example method for video processing is shown.

[0041] Fig.24 A flow chart of an example method for video processing is shown.

[0042] Fig.25 A flow chart of an example method for video processing is shown.

[0043] Fig.26 A flow chart of an example method for video processing is shown. DETAILED DESCRIPTION

[0044] This document provides several techniques that can be embodied in digital video encoders and decoders. For clear understanding, section headings are used in this document, and the scope of the techniques and embodiments disclosed in each section is not limited to only that section.

[0045] 1. Summary

[0046] The disclosed technology relates to video coding technology. Specifically, it relates to intra-frame mode coding in video coding. It can be applied to existing video coding standards such as HEVC, or standards that will be finalized (e.g., universal video coding). It may also be applicable to future video coding standards or video codecs.

[0047] 2. Introduction

[0048] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Video, and the two organizations worked together to produce the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure that utilizes temporal prediction plus transform coding. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new approaches and applied them to reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of reducing bitrate by 50% compared to HEVC.

[0049] 2.1 Intra-frame Block Copy

[0050] In HEVC Screen Content Coding Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0), intra-block copy (IBC) (also known as current picture reference) is used. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 1 As shown, when IBC is applied, the current block is predicted by a reference block in the same picture. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. Although IBC is not efficient for sequences captured by most cameras, it shows significant coding gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. IBC can effectively eliminate the redundancy between these repeated patterns. In HEVC-SCC, if the inter-coding unit (CU) selects the current picture as its reference picture, it can apply IBC. In this case, MV is renamed as block vector (BV), and BV always has integer pixel precision. In order to be compatible with the main profile HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that similarly, in the multi-view / 3D video coding standard, the inter-view reference picture is also marked as a "long-term" reference picture.

[0051] After the BV finds its reference block, a prediction can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Transforms and quantization can then be applied as in other coding modes.

[0052] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to certain constraints, some or all pixel values ​​are undefined. Basically, there are two solutions to this problem. One is to prohibit this situation, for example in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0053] 2.2 IBC in HEVC Screen Content Coding Extension

[0054] In the screen content coding extension of HEVC, when a block uses the current picture as a reference, it should be ensured that the entire reference block is within the available reconstruction area, as shown in the following text in the specification:

[0055] The derivation of the variables offsetX and offsetY is as follows:

[0056] offsetX=(ChromaArrayType==0)? 0:(mvCLX[0]&0x7?2:0)(8-106)

[0057] offsetY=(ChromaArrayType==0)? 0:(mvCLX[1]&0x7?2:0)(8-107)

[0058] This is a requirement for bitstream consistency, i.e. when the reference picture is the current picture, the luma motion vector mvLX shall follow the following restrictions:

[0059] - When the derivation of z-scan order block availability as specified in Section 6.4.1 is processed as input (xCuee, yCurr) set to (xCb, yCb) and the adjacent luminance position (xNbY, yNbY) set equal to (xPb + (mvLX[0]>>2) - offsetX, yPb + (mvLX[1]>>2) - offsetY), the output shall be true (TRUE).

[0060] - The output shall be TRUE when the derivation of z-scan order block availability as specified in clause 6.4.1 is processed with as input (xCuee,yCurr) set to (xCb,yCb), and the adjacent luma position (xNbY,yNbY) set equal to (xPb+(mvLX[0]>>2)+nPbW-1+offsetX,yPb+(mvLX[1]>>2)+nPbH-1+offsetY)

[0061] -One or both of the following conditions should be true:

[0062] The value of -(mvLX[0]>>2)+nPbW+xB1+offsetX is less than or equal to 0.

[0063] The value of -(mvLX[1]>>2)+nPbH+yB1+offsetY is less than or equal to 0.

[0064] - The following conditions should be true:

[0065] (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCurr / CtbSizeY<=yCurr / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY(8-108)

[0066] Therefore, it will not happen that the reference block overlaps with the current block or that the reference block is outside the picture. No padding of the reference or prediction blocks is required.

[0067] 2.3.IBC in VVC Test Model

[0068] In the current VVC test model (i.e., VTM-4.0 design), the entire reference block should overlap with the current coding tree unit (CTU) and not overlap with the current block. Therefore, no padding of reference or prediction blocks is required. The IBC flag is encoded as the prediction mode of the current CU. Therefore, there are a total of three prediction modes for each CU: MODE_INTRA\MODE_INTER and MODE_IBC.

[0069] 2.3.1.IBC Merge Mode

[0070] In IBC Merge mode, the index pointing to the entry in the IBC Merge candidate list is parsed from the bitstream. The construction of the IBC Merge list can be summarized according to the following steps in sequence:

[0071] ●Step 1: Derive spatial candidates

[0072] ●Step 2: Insert HMVP candidates

[0073] ●Step 3. Insert pairwise average candidates

[0074] In the derivation of spatial merge candidates, Figure 2 Up to four Merge candidates are selected from the candidates at the depicted positions. The order of derivation is A1, B1, B0, A0 and B2. Position B2 is considered only when any prediction unit (PU) at position A1, B1, B0, A0 is not available (for example, because it belongs to another slice or block) or is not encoded in IBC mode. After adding the candidate at position A1, a redundancy check is performed on the insertion of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving the coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the attached Figure 3 The pairs connected by arrows are depicted in , and candidates are added to the list only if the corresponding candidates used for redundancy checking have different motion information.

[0075] After inserting the spatial candidates, if the IBC Merge list size is still less than the maximum IBC Merge list size, the IBC candidates from the HMVP table can be inserted. A redundancy check is performed when inserting HMVP candidates.

[0076] Finally, the pairwise average candidates are inserted into the IBC Merge list.

[0077] When the reference block identified by a Merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstruction area, or is outside the valid area subject to certain constraints, the Merge candidate is called an invalid Merge candidate.

[0078] Note that invalid merge candidates can be inserted into the IBC Merge list.

[0079] 2.3.2 IBC AMVP Model

[0080] In IBC AMVP mode, the AMVP index pointing to the entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following steps in sequence:

[0081] ●Step 1: Derive spatial candidates

[0082] ○ Check A0, A1 until a usable candidate is found.

[0083] ○ Check B0, B1, B2 until a usable candidate is found.

[0084] ●Step 2: Insert HMVP candidates

[0085] ●Step 3: Insert zero candidates

[0086] After inserting the spatial candidates, if the IBC AMVP list size is still less than the maximum IBC AMVP list size, the IBC candidates from the HMVP table may be inserted.

[0087] Finally, the zero candidate is inserted into the IBC AMVP list.

[0088] 2.4 Palette Mode

[0089] The basic idea behind the palette mode is that the samples in a CU are represented by a small set of representative color values. This set is called the palette. Samples outside the palette can also be indicated by signaling an escape symbol followed by a (possibly quantized) component value. This is done in Figure 2 Shown in.

[0090] 2.5 Palette Mode in HEVC Screen Content Coding Extension (HEVC-SCC)

[0091] In the palette mode of HEVC-SCC, prediction is used to encode the palette and index maps.

[0092] 2.5.1 Encoding of Palette Entries

[0093] To encode the palette entries, a palette predictor is maintained. The palette predictor and the maximum size of the palette are signaled in the SPS. In HEVC-SCC, palette_predictor_initializer_present_flag is introduced in the PPS. When this flag is 1, the entry for initializing the palette predictor is signaled in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each slice and each slice. Depending on the value of palette_predictor_initializer_present_flag, the palette predictor is reset to 0 or initialized using the palette predictor initializer entry signaled in the PPS. In HEVC-SCC, a palette predictor initializer of size 0 is enabled to allow palette predictor initialization to be explicitly disabled at the PPS level.

[0094] For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. Figure 3The reuse flag is sent using a run-length encoding of zero. Thereafter, the number of new palette entries is signaled using an exponential Golomb code of order 0. Finally, the component values ​​of the new palette entries are signaled.

[0095] 2.5.2 Encoding of Palette Index

[0096] like Figure 4 As shown, the palette index is encoded using horizontal and vertical traversal scans. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. For the remainder of this subsection, it is assumed that the scan is horizontal.

[0097] The palette index is encoded using the two main palette sample modes ("INDEX" and "COPY_ABOVE"). As mentioned before, escape symbols are also signaled as "INDEX" mode and are assigned an index equal to the maximum palette size. The mode is signaled using a flag other than top row or when the previous mode was "COPY_ABOVE". In "COPY_ABOVE" mode, the palette index of the sample in the previous row is copied. In "INDEX" mode, the palette index is signaled explicitly. For both "INDEX" and "COPY_ABOVE" modes, a run value is signaled that specifies the number of subsequent samples that are also encoded using the same mode. When escape symbols are part of a run in "INDEX" or "COPY_ABOVE" modes, the escape component values ​​are signaled for each escape symbol. The encoding of the palette index is in Figure 5 Shown in.

[0098] The syntax sequence is done as follows. First, the number of index values ​​for the CU is signaled. This is followed by signaling the actual index value for the entire CU using truncated binary encoding. Both the number of indices and the index values ​​are encoded in bypass mode. This groups together the bypass boxes associated with the index. The palette sample mode (if required) and run are then signaled in an interleaved manner. Finally, the component escape values ​​corresponding to the escape samples for the entire CU are grouped together and encoded in bypass mode.

[0099] After signaling the index value, a further syntax element last_run_type_flag is signaled. This syntax element, together with the index number, eliminates the need to signal the run value corresponding to the last run in the block.

[0100] In HEVC-SCC, palette mode is also enabled for 4:2:2, 4:2:0 and monochrome chroma formats. The signaling of palette entries and palette indices is almost the same for all chroma formats. In case of non-monochrome formats, each palette entry consists of 3 components. For monochrome formats, each palette entry consists of a single component. For the chroma direction of the sub-sample, the chroma sample is associated with a luma sample index that is divisible by 2. After reconstructing the palette index for the CU, if the sample has only a single component associated with it, only the first component of the palette entry is used. The only difference in the signaling is in the escape component values. For each escape sample, the number of escape component values ​​signaled may be different, depending on the number of components associated with the sample.

[0101] 2.6 Intra Prediction in VVC

[0102] 2.6.1 Intra-mode Coding with 67 Intra-prediction Modes

[0103] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VTM4 is extended from 33 used in HEVC to 65. New directional modes not present in HEVC are Figure 6 Indicated by the red dashed arrows in , planar and DC modes remain the same. These more dense directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction.

[0104] A unified 6-MPM list is proposed for intra blocks, regardless of whether MRL and ISP coding tools are applied. Figure 7 As shown, the MPM list is constructed based on the intra-frame modes of the left and upper neighboring blocks in VTM4.0.

[0105] Assuming that the pattern on the left is denoted as Left, and the pattern of the block above is denoted as Above, the unified MPM list is constructed in the following steps:

[0106] - When the intra prediction mode of a neighboring block is invalid, its intra mode is set to "Planar" by default.

[0107] -If Left and Above are the same and both are angled:

[0108] ○MPM list→{Planar,Left,Left-1,Left+1,DC,Left-2}

[0109] -If Left and Above are different and both are angles:

[0110] ○ Set Max mode to Left and Above

[0111] ○ If the difference between Left and Above modes is between 2 and 62 (inclusive)

[0112] ■MPM List → {Planar,Left,Above,DC,Max-1,Max+1}

[0113] Otherwise

[0114] ■MPM List → {Planar,Left,Above,DC,Max-2,Max+2}

[0115] -If Left and Above are different, and one of the modes of Left and Above is angular and the other is non-angular:

[0116] ○ Set Max mode to Left and Above

[0117] ○MPM List → {Planar,Max,DC,Max-1,Max+1,Max-2}

[0118] -If both Left and Above modes are non-angle modes:

[0119] ○MPM List → {Planar,DC,V,H,V-4,V+4}

[0120] Note that when a pattern index (eg, Max+2) is out of the range [0, 66], it may be replaced with another valid pattern index.

[0121] The first MPM candidate, ie, the Planar mode, is signaled separately from the remaining MPM candidates.

[0122] 2.6.2 Mode-dependent Intra-frame Smoothing (MDIS)

[0123] A four-tap intra-frame interpolation filter is used to improve the accuracy of directional intra-frame prediction. In HEVC, a two-tap linear interpolation filter has been used to generate intra-frame prediction blocks in directional prediction mode (i.e., excluding plane and DC predictors). In VTM6, a simplified 6-bit 4-tap Gaussian interpolation filter is only used for directional intra-frame mode. The non-directional intra-frame prediction process is not modified. The selection of the 4-tap filter is performed according to the MDIS condition of the directional intra-frame prediction mode that provides non-fractional displacement, that is, all directional modes except the following: 2, HOR_IDX, DIA_IDX, VER_IDX, 66.

[0124] Depending on the intra prediction mode, the following reference sample processing is performed:

[0125] 1. Directional intra prediction modes are classified into one of the following groups:

[0126] A. Vertical or horizontal mode (HOR_IDX, VER_IDX),

[0127] B. Diagonal mode, indicating angles that are multiples of 45 degrees (2, DIA_IDX, VDIA_IDX),

[0128] C. Remaining directional mode;

[0129] 2. If the directional intra prediction mode is classified as belonging to group A, no filter is applied to the reference samples to generate the prediction samples;

[0130] 3. Otherwise, if the mode belongs to group B, a [1, 2, 1] reference sample filter may be applied (depending on the MDIS condition) to the reference samples to further copy these filtered values ​​to the intra predictor according to the selected direction, but no interpolation filter is applied;

[0131] 4. Otherwise, if the mode is classified as belonging to group C, only the intra reference sample interpolation filter is applied to the reference samples to generate prediction samples that fall into fractional or integer positions between the reference samples according to the selected direction (no reference sample filtering is performed).

[0132] 2.6.3 Cross-component linear model prediction

[0133] To reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is used in VTM4. To this end, a linear model is used to predict the chrominance samples based on the reconstructed luminance samples of the same CU, as shown below:

[0134] pred C (i,j)=α·rec L ′(i,j)+β (2-1)

[0135] Among them, pred C (i, j) represents the predicted chroma sample in CU, rec L (i, j) represents the downsampled reconstructed luminance sample of the same CU. The linear model parameters α and β are derived from the relationship between the luminance value and the chrominance value of the two samples. These two values ​​are the minimum luminance sample A(X a ,Y a ) and the maximum brightness sample B(X b ,Y b ). The linear model parameters α and β are obtained according to the following equation.

[0136]

[0137]

[0138] Where X a and X a Indicates the brightness and chromaticity values ​​of the minimum brightness sample. b and X b denote the luminance value and chrominance value of the maximum luminance sample, respectively. For a square coding block, the above two equations are directly applied. For a non-square coding block, the adjacent samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary. Figure 8 The sample point of the current block and the positions of the left and upper samples are shown in CCLM mode.

[0139] In addition to the template above and the template on the left being used to calculate the linear model coefficients together, they can also be used alternately in 2 other LM modes (called LM_A and LM_L modes).

[0140] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H). In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W).

[0141] For non-square blocks, the upper template is expanded to W+W and the left template is expanded to H+H.

[0142] Note that when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used for downsampling luma samples.

[0143] This parameter calculation is part of the decoding process, not just an encoder search operation. As a result, no syntax is used to communicate the α and β values ​​to the decoder.

[0144] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A and LM_L). The chroma mode signaling and derivation process are shown in Table 2-1. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures are enabled for luminance and chrominance components in I slices, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0145] Table 2-1 – Chroma prediction modes derived from luma mode when ccm_ is enabled

[0146]

[0147] 2.6.4 Position-dependent intra prediction combination

[0148] In VTM4, the intra prediction results of the planar mode are also corrected using the Position Dependent Intra Prediction Combination (PDPC) method. PDPC is an intra prediction method that calls for unfiltered boundary reference samples and a combination of HEVC-style intra prediction and filtered boundary reference samples. PDPC is applied to the following non-signaled intra modes: planar, DC, horizontal, vertical, bottom left angle mode and its eight adjacent angle modes, top right angle mode and its eight adjacent angle modes.

[0149] The prediction sample pred(x,y) is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference sample according to the following equation 3-4:

[0150] pred(x,y)=(wL×R -1,y +wT×R x,-1 –wTL×R -1,-1 +(64–wL–wT+wTL)×pred(x,y)+32)>>6 (1-4)

[0151] Among them, R x,-1 ,R -1,y represent the reference sample points located at the top and left of the current sample point (x, y), and R -1,-1 Indicates the reference sample located at the upper left corner of the current block.

[0152] If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filters are required, while in the case of HEVC, a DC mode boundary filter or a horizontal / vertical mode edge filter is required.

[0153] Figures 9A-9D The definitions of the reference samples (Rx,-1, R-1,y and R-1,-1) for PDPC applied to various prediction modes are described. The prediction sample pred(x',y') is located at (x',y') within the prediction block. The coordinate x of the reference sample Rx,-1 is given by x=x'+y'+1, and the coordinate y of the reference sample R-1,y is similarly given by y=x'+y'+1.

[0154] The PDPC weights depend on the prediction mode and are shown in Table 2-2.

[0155] Table 2-2: Example of PDPC weights according to prediction mode

[0156] Prediction Model w wxya wxya Upper right corner 16>>((y’<<1)>>shift 16>>((x’<<1)>>shift) 0 Lower left corner 16>>((y’<<1)>>shift) 16>>((x’<<1)>>shift) 0 Upper right corner 32>>((y’<<1)>>shift) 0 0 Lower left adjacent corner 0 32>>((x’<<1)>>shift) 0

[0157] 2.6.5 Multiple Reference Line (MRL) Intra Prediction

[0158] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Fig.10 In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not taken from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (e.g., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0159] The index of the selected reference line (mrl_idx) is signaled and used to generate the intra predictor. For reference line indices greater than 0, only other reference line modes are included in the MPM list, and only the MPM index is signaled without signaling the remaining modes. The reference line index is signaled before the intra prediction mode, and in case of a non-zero reference line index being signaled, planar mode and DC mode are excluded from the intra prediction mode.

[0160] MRL is disabled for the first line of blocks within a CTU to prevent the use of extended reference samples outside the current CTU line. Likewise, PDPC is disabled when additional lines are used.

[0161] 2.6.6 Intra-frame Sub-segmentation (ISP)

[0162] The Intra Sub-Partitioning (ISP) tool divides the luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions depending on the block size. For example, the minimum block size of ISP is 4x8 (or 8x4). If the block size is larger than 4x8 (or 8x4), the corresponding block is divided by 4 sub-partitions. Fig.11A and 11B Examples of two possibilities are shown. All sub-divisions satisfy the condition of having at least 16 samples.

[0163] Table 2-3 – Entropy coding coefficient group sizes

[0164] Block size Coefficient group size 1×N,N≥16 1×16 N×1,N≥16 16×1 2×N,N≥8 2×8 N×2,N≥8 8×2 All other possible M×N situations 4×4

[0165] For each sub-division, the reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each sub-division can be used to generate a prediction for the next sub-division, and each sub-division is processed repeatedly. In addition, the first sub-division to be processed is the sub-division containing the upper left sample of the CU, and then continues to the sub-division downward (horizontal division) or to the right (vertical division). As a result, the reference samples used to generate the sub-division prediction signal are located only to the left and above the line. All sub-divisions share the same intra-frame mode. The following is an overview of the interaction of ISP with other encoding tools.

[0166] -Multi-reference line (MRL): If the MRL index of a block is not 0, the ISP coding mode will be inferred to be 0, so the ISP mode information will not be sent to the decoder.

[0167] - Entropy coding coefficient group dimensions: The dimensions of the entropy coded sub-blocks have been modified so that they have 16 samples in all possible cases, as shown in Table 2-3. Note that the new dimensions only affect ISP-generated blocks where one of the dimensions is less than 4 samples. In all other cases, the coefficient groups retain 4×4 dimensions.

[0168] - CBF encoding: It is assumed that at least one subpartition has a non-zero CBF. Thus, if n is the number of subpartitions and the first n-1 subpartitions have yielded zero CBFs, the CBF of the nth subpartition is inferred to be 1.

[0169] -MPM usage: In ISP mode coded blocks, the MPM flag will be inferred to be 1, and the MPM list is modified to exclude DC mode and prioritize horizontal intra mode for ISP horizontal partitions and vertical intra mode for vertical partitions.

[0170] - Transform size restriction: All ISP transforms with length greater than 16 points use DCT-II.

[0171] -PDPC: When the CU uses ISP coding mode, the PDPC filter will not be applied to the generated sub-partitions.

[0172] -MTS flag: If a CU uses ISP coding mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Therefore, the encoder will not perform RD tests on the different available transforms for each generated sub-partition. The transform selection for ISP mode will instead be fixed and will be selected based on the intra mode, processing order and the used block size. Therefore, no signaling is required. For example, let t H and t VThe horizontal and vertical transforms are selected for the w×h subdivisions respectively, where w is the width and h is the height. The transforms are then selected according to the following rules:

[0173] • If w=1 or h=1, there is no horizontal or vertical transformation, respectively.

[0174] ●If w=2 or w>32, then t H =DCT-II

[0175] ●If h=2 or h>32, then t V =DCT-II

[0176] ●Otherwise, select a transformation as shown in Table 2-4.

[0177] Table 2-4 – Transform selection depending on intra mode

[0178]

[0179] 2.7 Quantized Residual Block Differential Pulse Code Modulation (QR-BDPCM)

[0180] A quantized residual block differential pulse code modulation (QR-BDPCM) is proposed to encode the screen content efficiently.

[0181] The prediction direction used in QR-BDPCM can be vertical and horizontal prediction mode. The whole block is intra-predicted by copying samples in the prediction direction (horizontal or vertical prediction) similar to intra-prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is encoded. This can be described as follows: For a block of size M (rows) × N (columns), let r i,j , 0≤i≤M-1,0≤j≤N-1 is the prediction residual after intra prediction using unfiltered samples from the upper or left block boundary samples horizontally (copying the left neighboring pixel values ​​of the entire prediction block row by row) or vertically (copying the top neighbor row to each row in the prediction block). Let Q(r i,j ), 0≤i≤M-1,0≤j≤N-1 represents the residual r i,j The quantized version of , where the residual is the difference between the original block and the predicted block value. Then, block DPCM is applied to the quantized residual samples to obtain a sample with elements The modified M×N array When vertical BDPCM is signaled:

[0182]

[0183] For horizontal prediction, similar rules apply and the residual quantized samples are obtained by

[0184]

[0185] Quantize the residual samples Send to the decoder.

[0186] At the decoder side, the above calculation is inverted to produce Q(r i,j ),0≤i≤M-1,0≤j≤N-1. For vertical prediction,

[0187]

[0188] For the horizontal case,

[0189]

[0190] The inverse quantized residual Q -1 (Q(r i,j )) is added to the intra-block prediction value to produce the reconstructed sample point value.

[0191] 2.8 Matrix Weighted Intra Prediction (MIP)

[0192] The matrix-weighted intra prediction (MIP) method is a new intra prediction technique in VTM6. To predict the samples of a rectangular block of width W and height H, the matrix-weighted intra prediction (MIP) takes as input a row of H reconstructed adjacent boundary samples on the left side of the block and a row of W reconstructed adjacent boundary samples above the block. If the reconstructed samples are not available, they are generated as done in conventional intra prediction. The generation of the prediction signal is based on the following three steps, namely averaging, matrix-vector multiplication and linear interpolation, as shown in Fig.12 shown.

[0193] 2.9 Affine Motion Compensated Prediction

[0194] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VTM6, block-based affine transformation motion compensation prediction is applied. Fig.13A and 13B As shown, the affine motion field of the block is described by motion information of two control point motion vectors (4 parameters) or three control point motion vectors (6 parameters).

[0195] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0196]

[0197] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0198]

[0199] Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right control point, (mv 2x ,mv 2y ) is the motion vector control point in the lower left corner.

[0200] In order to simplify motion compensation prediction, block-based affine transformation prediction is applied. In order to derive the motion vector of each 4×4 luminance sub-block, the motion vector of the center sample of each sub-block is calculated according to the above equation, such as Fig.14 As shown, and rounded to fractional precision of 1 / 16. Then, a motion compensated interpolation filter is applied to generate a prediction for each subblock with a derived motion vector. The subblock size of the chrominance component is also set to 4×4. The MV of a 4×4 chrominance subblock is calculated as the average of the MVs of the four corresponding 4×4 luminance subblocks.

[0201] Like translational motion inter-frame prediction, there are two affine motion inter-frame prediction modes: affine Merge mode and affine AMVP mode.

[0202] 2.10 Local Dual Trees in VVC

[0203] In typical hardware video encoders and decoders, the processing throughput decreases when a picture has smaller intra blocks due to the sample processing data dependency between adjacent intra blocks. The predictor generation for an intra block requires boundary reconstruction samples from the top and left of the adjacent blocks. Therefore, intra prediction must be processed sequentially, block by block.

[0204] In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of the smallest intra CU can be further divided into four 4x4 luma intra prediction units (PUs), but the chroma components of the smallest intra CU cannot be further divided. Therefore, the worst-case hardware processing throughput occurs when processing 4x4 chroma intra blocks or 4x4 luma intra blocks.

[0205] In VTM5.0, in a single coding tree, since the chroma partitioning always follows luma and the minimum intra CU is 4x4 luma samples, the minimum chroma intra CB is 2x2. Therefore, in VTM5.0, the minimum chroma intra CB in a single coding tree is 2x2. The worst-case hardware processing throughput of VVC decoding is only 1 / 4 of that of HEVC decoding. In addition, after adopting tools including cross-component linear model (CCLM), 4-tap interpolation filter, position-dependent intra prediction combination (PDPC), and combined inter-intra prediction (CIIP), the reconstruction process of chroma intra CB is much more complicated than that of HEVC. It is challenging to achieve high processing throughput in a hardware decoder. In this section, a method to improve the worst-case hardware processing throughput is proposed.

[0206] The purpose of this method is to prohibit chroma intra CBs with less than 16 chroma samples by limiting the segmentation of chroma intra CBs.

[0207] In a single coding tree, a SCIPU is defined as a coding tree node whose chroma block size is greater than or equal to TH chroma samples and has at least one sub-luminance block less than 4TH luminance samples, where TH is set to 16 in this contribution. It is required that in each SCIPU, all CBs are inter-frame, or all CBs are non-inter-frame, that is, intra-frame or IBC. In the case of non-inter-frame SCIPU, it is further required that the chroma of the non-inter-frame SCIPU shall not be further divided, and the luminance of the SCIPU shall be allowed to be further divided. In this way, the minimum chroma intra-frame CB size is 16 chroma samples, and 2x2, 2x4 and 4x2 chroma CBs are deleted. In addition, in the case of non-intra-frame SCIPU, chroma scaling is not applied. In addition, when the luminance block is further divided and the chroma block is not divided, a local dual tree coding structure is constructed.

[0208] exist Fig.15A and 15B Two SCIPU examples are shown in Fig.15A In , a chroma CB with 8x4 chroma samples and three luma CBs (4x8, 8x8, 4x8 luma CB) form a SCIPU, because the ternary tree (TT) partitioned from the 8x4 chroma samples will result in a chroma CB with less than 16 chroma samples. Fig. 15B In the example, a 4x4 chroma sample chroma CB (on the left of the 8x4 chroma sample) and three luma CBs (8x4, 4x4, 4x4 luma CB) form a SCIPU, and another 4x4 sample chroma CB (on the right of the 8x4 chroma sample) and two luma CBs (8x4, 8x4 luma CB) form a SCIPU, because the binary tree (BT) partitioned from the 4x4 chroma sample will result in a chroma CB that is less than 16 chroma samples.

[0209] In the proposed method, if the current slice is an I slice or the current SCIPU has a 4x4 luma split in it after being further divided once, the type of the SCIPU is inferred as non-inter (because intra 4x4 is not allowed in VVC); otherwise, before parsing the CU in the SCIPU, a signaled flag is used to indicate the type of the SCIPU (intra or non-intra).

[0210] By applying the above method, the worst-case hardware processing throughput occurs when processing 4x4, 2x8, or 8x2 chroma blocks instead of 2x2 chroma blocks. The worst-case hardware processing throughput is the same as that in HEVC and 4 times that in VTM5.0.

[0211] 3. Question

[0212] In addition to the regular intra and inter prediction modes in VVC, palette, IBC is also treated as a different prediction mode. Even for HEVC SCC, palette is treated as an intra prediction mode, but the signaling is still separated from the regular intra prediction modes (e.g., DC / plane and angle modes) through the palette mode flag. For HEVC SCC, IBC is treated as an inter prediction mode by checking whether the signaled / derived reference picture is the current picture. That is, the reference picture list is modified to include the current picture as an additional reference picture.

[0213] When the codec supports palette mode, how to encode the palette flag becomes a key factor in encoding performance. The palette flag may have correlation between neighboring blocks. If we only encode the palette mode flag without considering the context of neighboring flags, we may lose coding efficiency.

[0214] Chroma intra prediction and transform may need to consider the prediction mode of the luma block.

[0215] 4. Example Embodiments

[0216] The embodiments discussed below should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow sense. In addition, these embodiments can be combined in any way.

[0217] In this document, it is proposed to treat a new coding method X (eg, X is palette / IBC / QR-BDPCM) as an additional intra prediction mode. Therefore, there is no need to separately signal an enable / disable flag for the new coding method.

[0218] 1. The set of allowed intra prediction modes (IPM) may be further extended to include new coding methods. Assuming that the index range of the existing IPM is [0, M-1], using the proposed method, the following conditions may apply:

[0219] a. The index range of IPM can be modified to [0, M-1+K], where K is the number of new encoding methods considered as part of IPM.

[0220] b. In one example, the palette mode can be marked as Idx New IPM, where Idx New In the range of [0, M-1+K].

[0221] i. In one example, the IPM index for the new coding method may be set to be no less than M, ie, the new coding method is included in the IPM set after all existing intra prediction modes.

[0222] 1) The palette mode can be set to M (i.e. Idx New Equal to M)

[0223] 2) The IBC mode can be set to M or M+1 (i.e., Idx New Equal to M, M+1)

[0224] 3) QR-BDPCM can be set to M or M+1 or M+2 (i.e., Idx New equal to M, M+1 or M+2)

[0225] ii. In one example, the new encoding method may be included in the IPM set before one of the existing intra prediction modes, i.e., Idx New Not greater than M-1.

[0226] 1) In one example, a new encoding method may be included in the IPM set before all existing intra prediction modes.

[0227] a) In one example, the IPM index for palette mode can be set to 0 (ie, Idx New equal to 0).

[0228] b) In one example, the IPM index for IBC mode can be set to 0 (ie, Idx New equal to 0).

[0229] c) In one example, the IPM index for QR-BDPCM mode may be set to 0 (ie, Idx New equal to 0).

[0230] 2) Alternatively, in addition, the original index is not less than Idx New The index of the remaining IPM can be modified, for example, increased by 1.

[0231] iii. In one example, IdxNew Can be predefined, or signaled or depend on the following:

[0232] 1) The block dimension of the current video block,

[0233] 2) the current quantization parameter of the current video block,

[0234] 3) Indication of the color format (such as 4:2:0, 4:4:4),

[0235] 4) Split / dual coding tree structure,

[0236] 5) slice / slice group type and / or picture type,

[0237] 6) Indications signaled in SPS, PPS, picture header, slice header, and slice header.

[0238] c. In one example, whether and / or how a certain encoding method is included in the allowed IPM set may depend on block dimensions and / or decoding information.

[0239] i. In one example, for MxN (e.g., 128x128 / 128x64 / 64x128) blocks, IPM corresponding to IBC may not be allowed.

[0240] ii. In one example, for MxN (e.g., 128x128 / 128x64 / 64x128) blocks, IPMs corresponding to palettes may not be allowed.

[0241] iii. In one example, for a width greater than K (eg, K=32) or a height greater than L

[0242] For blocks (eg, L=32), the IPM corresponding to the BDPCM may not be allowed.

[0243] 2. The most probable intra mode (MPM) set may include new coding methods.

[0244] a. In one example, an MPM list may be first constructed and then modified by replacing a mode in the constructed MPM list with a new encoding method (eg, IBC / palette / QR-BDPCM / MIP) different from conventional intra prediction.

[0245] i. In one example, the new encoding method can replace the mode in the first position in the MPM list.

[0246] 1) Alternatively, in one example, the new encoding method may replace the mode at the last position in the MPM list.

[0247] ii. Alternatively, furthermore, the new encoding method may be represented by an intra prediction mode index, such as the index defined in bullet point 1.

[0248] iii. Alternatively, furthermore, during the construction of the MPM list, if the neighboring blocks are encoded with the new method, the neighboring blocks may be marked as unavailable, or treated in the same way as those encoded using the inter-mode.

[0249] b. In one example, the MPM list may be constructed first, and then modified by adding a new coding mode (e.g., IBC / palette / QR-BDPCM) to a given position of the list, and the size of the MPM list may be increased by K, where K is the number of new coding modes

[0250] i. The location of the new encoding mode may depend on

[0251] 1) the block dimensions of the current block,

[0252] 2) The prediction mode of the current block,

[0253] 3) The current quantization parameter of the current block,

[0254] 4) The palette flag of the adjacent block,

[0255] 5) Intra-block copy flag of adjacent blocks,

[0256] 6) an indication of the color format (such as 4:2:0, 4:4:4),

[0257] 7) Separate or dual coding tree structure,

[0258] 8) Slice / slice group type and / or picture type,

[0259] 9) Flags in the SPS / PPS / slice header.

[0260] c. In one example, the MPM list construction process may be modified so that if a neighboring block is encoded with a new encoding method, the encoding method may be directly added as one of the MPMs.

[0261] d. In one example, the MPM list construction process may be modified so that if a neighboring block is encoded with a new encoding method, the IPM index corresponding to the encoding method may be added as one of the MPMs.

[0262] e. Whether / how a new coding method is added to the MPM list may depend on the block dimensions and / or decoding information.

[0263] i. In one example, for certain block sizes, adding an intra prediction mode (IPM) corresponding to a new coding mode to the MPM list may not be allowed.

[0264] 1) In one example, for MxN (e.g., 128x128 / 128x64 / 64x128) blocks, adding an IPM corresponding to an IBC to the MPM list may not be allowed.

[0265] 2) In one example, for MxN (e.g., 128x128 / 128x64 / 64x128) blocks, deriving the IPM corresponding to the palette from the MPM list may not be allowed.

[0266] 3) In one example, for a block with a width greater than L (eg, L=32) or a height greater than K (eg, K=32), adding an IPM corresponding to a BDPCM to the MPM list may not be allowed.

[0267] f. In one example, whether and / or how to include a new coding mode in the MPM may depend on the enabling flag of the coding mode in the SPS / PPS / VPS / slice header / slice header.

[0268] 3. The filtering process (eg, deblocking filter, SAO, ALF, bilateral filter, Hadamard domain filter) may depend on the intra prediction mode (IPM).

[0269] a. In one example, the boundary strength (BS) setting in the deblocking filter may depend on the IPM index, such as if the index indicates palette mode, then BS is set to 0.

[0270] 4. When a chroma block is determined to be a screen content block, the intra prediction process and / or transform process of the chroma block may be modified.

[0271] a. In one example, when one or more selected luma blocks are encoded in IBC mode or palette mode, the chroma blocks may be determined as screen content blocks.

[0272] i. In one example, the size of the selected luma block may be the smallest CU / PU / TU size or the unit used for motion / mode storage (such as 4×4).

[0273] ii. In one example, the selected luma block may be a CU / PU / TU covering the center, upper left, upper right, lower left, or lower right position of the corresponding luma region. Fig.16 Examples of corresponding brightness areas are shown.

[0274] iii. In one example, assuming that the upper left coordinate of the current chrominance block is (x0, y0), and the width and height of the current chrominance block are w0 and h0 respectively, the coordinates of the upper left sample point in the corresponding luminance region, the width and height of the corresponding luminance region can be scaled according to the color format.

[0275] 1) For the 4:2:0 color format, the upper left coordinate of the co-located brightness area is (2*x0, 2*y0), and its width and height are 2*w0 and 2*h0 respectively.

[0276] 2) For the 4:4:4 color format, the upper left coordinate of the co-located brightness region is (x0, y0), and its width and height are w0 and h0 respectively.

[0277] iv. Assuming the upper left coordinate of the co-located brightness region is (x, y), and its width and height are W and H, the coordinates of the center position may be:

[0278] 1)(x+W / 2,y+H / 2)

[0279] 2)(x+W / 2-1,y+H / 2-1)

[0280] 3)(x+W / 2,y+H / 2-1)

[0281] 4)(x+W / 2-1,y+H / 2).

[0282] v. Assuming that the upper left coordinate of the co-located brightness region is (x, y), and its width and height are W and H, the coordinates of the upper left position may be (x, y).

[0283] vi. Assuming the upper left coordinate of the co-located brightness region is (x, y), and its width and height are W and H, the coordinates of the upper right position may be:

[0284] 1)(x+W,y)

[0285] 2)(x+W-1,y).

[0286] vii. Assuming the upper left coordinate of the co-located brightness region is (x, y), and its width and height are W and H, the coordinates of the lower left position may be:

[0287] 1)(x,y+H)

[0288] 2)(x,y+H-1).

[0289] viii. Assuming the upper left coordinate of the co-located brightness region is (x, y), and its width and height are W and H, the coordinates of the lower right position may be:

[0290] 1)(x+W,y+H)

[0291] 2)(x+W,y+H-1)

[0292] 3)(x+W-1,y+H)

[0293] 4)(x+W-1,y+H-1).

[0294] b. In one example, when a chroma block is determined to be a screen content block, certain IPMs (eg, DC and / or palette PLANAR modes) may not be allowed.

[0295] i. Alternatively, in one example, when a chroma block is determined to be a screen content block, VER and / or HOR modes may not be allowed.

[0296] ii. Alternatively, in one example, when the chroma block is determined to be a screen content block, the export mode may not be allowed.

[0297] c. In one example, when a chroma block is determined to be a screen content block, PDPC may not be allowed.

[0298] d. In one example, when a chroma block is determined to be a screen content block, MDIS may not be allowed.

[0299] 5. It is recommended to align the positions of the left neighboring block and / or the upper neighboring block used in the context derivation of certain syntax elements with the positions of the left neighboring block and / or the upper neighboring block used in the MPM derivation.

[0300] a. In one example, the syntax elements may be associated with a partition flag, a mode constraint flag, a skip flag, a prediction mode flag, an IBC flag, a MIP flag, an affine merge flag, and an affine inter flag.

[0301] b. The positions of the left neighboring block and / or the upper neighboring block used in the context derivation of certain syntax elements may be aligned with the positions of the left neighboring block and / or the upper neighboring block used in the Merge / AMVP candidate list derivation process.

[0302] c. In one example, the positions of the left neighboring block and / or the upper neighboring block used in the context derivation of the above syntax element may be Figure 7 The left / upper neighboring block shown in .

[0303] 6. It is recommended to use more than 1 context to signal the indication of the use of palette mode.

[0304] a. In one example, the context derivation for indicating the use of the palette mode can be based on the left / upper neighboring blocks. Let A be the palette flag of the left neighboring block and B be the palette flag of the upper neighboring block.

[0305] i. In one example, a context index indicating the use of palette mode may be derived by A*2+B.

[0306] 1) Alternatively, in one example, a context index for indicating the use of palette mode may be derived from A+B.

[0307] ii. In one example, the positions used in MPM derivation to derive the left and top neighboring blocks can be used to derive the context for encoding the palette mode flag.

[0308] iii. In one example, the positions of the left and top neighboring blocks used to derive the context for encoding the palette mode flag may be Fig.17 Position shown.

[0309] 7. Whether and / or how the above methods are applied may be based on:

[0310] a. Video content (e.g. screen content or natural content)

[0311] b. Messages signaled in DPS / SPS / VPS / PPS / APS / picture header / slice header / slice group header / largest coding unit (LCU) / coding unit (CU) / LCU row / LCU group / TU / PU block / video coding unit

[0312] c. Location of CU / PU / TU / block / video coding unit

[0313] d. Block dimensions of the current block and / or its neighboring blocks

[0314] e. Block shape of the current block and / or its neighboring blocks

[0315] f. Indication of the color format (such as 4:2:0, 4:4:4, RGB, or YUV)

[0316] g. Coded tree structure (e.g. dual tree or single tree)

[0317] h. Strip / slice group type and / or picture type

[0318] i. Color components (e.g. can only be applied to luma and / or chroma components)

[0319] j. Time domain layer ID

[0320] k. Profile / Level / Standard Tier

[0321] Fig.18 1 is a schematic diagram illustrating an example of the structure of a computer system or other control device 1800 that can be used to implement various parts of the disclosed technology. Fig.16In the embodiment, computer system 1800 includes one or more processors 1805 and memory 1810 connected by interconnect 1825. Interconnect 1825 can represent any one or more separate physical buses, point-to-point connections, or both connected by appropriate bridges, adapters, or controllers. Thus, interconnect 1825 can include, for example, a system bus, a peripheral component interconnect (PCI) bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus (sometimes referred to as "FireWire").

[0322] The processor 1805 may include a central processing unit (CPU) to control the overall operation of, for example, the host. In some embodiments, the processor 1805 implements this by executing software or firmware stored in the memory 1810. The processor 1805 may be or may include one or more programmable general or special microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), etc., or a combination of these devices.

[0323] The memory 1810 may be or include the main memory of the computer system. The memory 1810 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, etc., or a combination of these devices. In use, the memory 1810 may include, among other things, a set of machine instructions that, when executed by the processor 1805, cause the processor 1805 to perform operations to implement embodiments of the disclosed technology.

[0324] Also connected to processor 1805 via interconnect 1825 is an (optional) network adapter 1815. Network adapter 1815 provides computer system 1800 with the ability to communicate with remote devices (such as storage clients and / or other storage servers) and may be, for example, an Ethernet adapter or a Fibre Channel adapter.

[0325] Fig.19A block diagram of an example embodiment of a mobile device 1900 that can be used to implement various parts of the disclosed technology is shown. The mobile device 1900 can be a laptop, a smartphone, a tablet, a camera, or other device capable of processing video. The mobile device 1900 includes a processor or controller 1901 to process data, and a memory 1902 that communicates with the processor 1901 to store and / or buffer data. For example, the processor 1901 may include a central processing unit (CPU) or a microcontroller unit (MCU). In some implementations, the processor 1901 may include a field programmable gate array (FPGA). In some implementations, the mobile device 1900 includes or communicates with a graphics processing unit (GPU), a video processing unit (VPU), and / or a wireless communication unit to implement various visual and / or communication data processing functions of a smartphone device. For example, the memory 1902 may include and store processor executable code, and when the processor 1901 executes the code, the mobile device 1900 is configured to perform various operations, such as receiving information, commands, and / or data, processing information and data, and sending or providing the processed information / data to another data device, such as an actuator or an external display. To support the various functions of the mobile device 1900, the memory 1902 can store information and data, such as instructions, software, values, images, and other data processed or referenced by the processor 1901. For example, various types of random access memory (RAM) devices, read-only memory (ROM) devices, flash memory devices, and other suitable storage media can be used to implement the storage function of the memory 1902. In some implementations, the mobile device 1900 includes an input / output (I / O) unit 1903 to interface the processor 1901 and / or memory 1902 with other modules, units, or devices. For example, the I / O unit 1903 can interface with the processor 1901 and the memory 1902 to utilize various wireless interfaces compatible with typical data communication standards, for example, between one or more computers in the cloud and user devices. In some implementations, the mobile device 1900 can interface with other devices using a wired connection through the I / O unit 1903. The I / O unit 1903 may include a wireless sensor, such as an infrared detector for detecting remote control signals, or other suitable wireless human-machine interface technology. The mobile device 1900 may also be connected to other external interfaces (e.g., data storage) and / or a visual or audio display device 1904 to retrieve and transmit data and information that may be processed by the processor, stored by the memory, or displayed on the output unit of the display device 1904 or an external device. For example, the display device 1904 may display a video frame modified based on the MVP according to the disclosed technology.

[0326] Fig. 202000 is a block diagram illustrating an example video processing system 2000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2000. System 2000 may include an input 2002 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. Input 2002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0327] System 2000 may include coding component 2004, which may implement various coding or decoding methods described herein. Coding component 2004 may reduce the average bit rate of the video from input 2002 to the output of coding component 2004 to generate the coded representation of the video. Therefore, coding techniques are sometimes referred to as video compression or video transcoding techniques. The output of coding component 2004 may be stored, or sent by the communication of the connection represented by component 2006. Component 2008 may use the storage or communication bit stream (or coding) representation of the video received at input 2002 to generate pixel values ​​or displayable video sent to display interface 2010. The process of generating user-visible video from bit stream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "coding" operations or tools, it should be understood that coding tools or operations are used at the encoder, and the operation of the corresponding decoding tools or the reverse coding result will be performed by the decoder.

[0328] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[0329] Fig.21 A flow chart of an example method for video processing is shown. The method includes performing (2102) a conversion between a current video block and an encoded representation of the current video block, wherein during the conversion process, at least one encoding prediction method is considered as an intra-frame prediction mode (IPM) in an IPM set without using a flag to signal the at least one encoding prediction mode.

[0330] Some embodiments of the disclosed technology are presented in a clause-based format.

[0331] 1. A video processing method, comprising:

[0332] A conversion between a current video block and an encoded representation of the current video block is performed, wherein during the conversion process, at least one encoding prediction method is considered as an intra prediction mode (IPM) in an IPM set without using a flag to signal the at least one encoding prediction mode.

[0333] 2. A method according to clause 1, wherein the at least one coding prediction method corresponds to one of the following: a palette mode, an intra block copy (IBC) mode or a quantized residual block differential pulse code modulation mode (QR-BDPCM).

[0334] 3. The method according to clause 2, wherein the IPM set is associated with an index range [0, M-1], further comprising:

[0335] The index range is modified to [0, M-1+K], where K is the number of at least one coding prediction method to be considered as part of the IPM set.

[0336] 4. The method according to clause 3, wherein, if at least one coding prediction method corresponds to a palette mode, further comprising:

[0337] Maps a palette mode to an index value M in the index range.

[0338] 5. The method according to clause 3, wherein, if at least one coding prediction method corresponds to IBC mode, further comprising:

[0339] Maps an IBC mode to index value M or M+1 within the index range.

[0340] 6. The method according to clause 3, wherein if at least one coding prediction method corresponds to QR-BDPCM mode, further comprising:

[0341] Maps an IBC mode to index value M, M+1, or M+2 within the index range.

[0342] 7. The method according to clause 3, wherein the at least one encoding prediction method is mapped to an index value less than or equal to M-1.

[0343] 8. A method according to any of clauses 1-7, wherein at least one encoding prediction method is selectively allowed or not allowed to be considered as an IPM based at least in part on one or more conditions.

[0344] 9. A method according to clause 8, wherein the one or more conditions relate to: a dimension of the current video block, a quantization parameter of the current video block, a color format of the current video block, a coding tree structure associated with the current video block, a type of slice / slice group / picture associated with the current video block, or a flag signaled in an SPS / PPS / picture header / slice header / slice header.

[0345] 10. A video processing method, comprising:

[0346] A conversion between a current video block and an encoded representation of the current video block is performed, wherein during the conversion, at least one encoding prediction method is considered as an intra prediction mode (IPM) in a most probable mode (MPM) list without using a flag to signal the at least one encoding prediction mode.

[0347] 11. The method according to clause 10, further comprising:

[0348] Constructing the MPM list; and

[0349] Replace the patterns in the MPM list with at least one encoding prediction method.

[0350] 12. The method according to clause 11, wherein at least one coding prediction method replaces the last mode in the list.

[0351] 13. The method according to clause 11, wherein at least one encoding prediction method replaces the first mode in the list.

[0352] 14. The method according to clause 11, wherein the constructing comprises:

[0353] If the neighboring block is encoded according to at least one encoding prediction method, the neighboring block of the current video block is marked as unavailable.

[0354] 15. The method according to clause 11, wherein constructing comprises:

[0355] If the adjacent block is encoded according to at least one encoding prediction method, the adjacent block is used as an inter-frame encoding block.

[0356] 16. A method according to clause 15, wherein at least one encoding prediction method replaces the pattern at a certain position in the list.

[0357] 17. A method according to any one of clauses 10 to 16, further comprising:

[0358] Add an IPM index corresponding to at least one prediction method to the list.

[0359] 18. A method according to any of clauses 10-17, wherein at least one encoding prediction method is selectively allowed or not allowed to be considered as an IPM based at least in part on one or more conditions.

[0360] 19. According to clause 18. A method according to claim 18, wherein the position in the list corresponding to the at least one coding prediction method relates to: the dimension of the current video block, the quantization parameter of the current video block, the color format of the current video block, the coding tree structure associated with the current video block, the type of slice / slice group / picture associated with the current video block, or a flag signaled in the SPS / PPS / picture header / slice header / slice header.

[0361] 20. A method according to any one of clauses 10 to 19, further comprising:

[0362] Parameters of the filtering step are set based in part on an index corresponding to the at least one encoding prediction method.

[0363] 21. A method according to clause 20, wherein the parameter comprises a boundary strength (BS) of a deblocking filter.

[0364] 22. A method according to clause 21, wherein if the at least one encoding prediction method is a palette mode, BS is set to zero.

[0365] 23. The method according to any one of clauses 1 to 22, further comprising:

[0366] When it is determined that the current video block is a screen content block, the IPM step or the transformation process is modified.

[0367] 24. The method of clause 23, wherein determining that the current video block is a screen content block comprises determining that one or more luma blocks are encoded in IBC mode or palette mode.

[0368] 25. A method according to clause 24, wherein position dependent intra prediction combining (PDPC) is not allowed for chroma blocks.

[0369] 26. A video processing method, comprising:

[0370] performing a conversion between a current video block and an encoded representation of the current video block, wherein during the conversion, in palette mode, a context of a syntax element associated with the current video block is derived from a neighboring block used to derive an MPM list; and

[0371] Align the position of the neighboring block with the position of the current video block.

[0372] 27. The method of clause 26, wherein the context of the syntax element associated with the current video block corresponds to encoding a flag for a palette mode.

[0373] 28. The method according to clause 26, wherein the neighboring block comprises at least one of: a left neighboring block or an upper neighboring block.

[0374] 29. A method according to clause 26, wherein the palette mode includes the use of multiple contexts, and further wherein the use of the multiple contexts is signaled in the encoded representation.

[0375] 30. A method according to any of clauses 26-29, wherein exporting the context is selectively allowed or not allowed based at least in part on one or more conditions.

[0376] 31. A method according to clause 30, wherein one or more conditions relate to: content of the current video block, DPS / SPS / VPS / PPS / APS / picture header / slice header / slice group header / largest coding unit (LCU) / coding unit (CU) / LCU row / LCU group / TU / PU block / video coding unit, position of CU / PU / TU associated with the current video block, dimensions of the current video block and / or neighboring blocks, shape of the current video block and / or neighboring blocks, color format of the current video block, coding tree structure associated with the current video block, type of slice / slice group / picture associated with the current video block, color component of the current video block, temporal layer ID of the current video block, or profile level of the current video block.

[0377] 32. A video decoding apparatus comprising a processor configured to implement one or more of the methods described in clauses 1 to 31.

[0378] 33. A video encoding apparatus comprising a processor configured to implement the method of one or more of clauses 1 to 31.

[0379] 34. A computer program product having computer code stored thereon, which code, when executed by a processor, causes the processor to implement the method of any one of clauses 1 to 31.

[0380] 35. The methods, apparatus, or systems described in this document.

[0381] Fig. 22A flow chart of an example method for video processing is shown. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, generating (2202) an extended intra-prediction mode (IPM) set associated with the current video block by modifying an existing IPM set to include at least one new encoding method, wherein the new encoding method is different from an encoding method in the existing IPM and is considered an IPM; and performing (2204) the conversion based on the extended IPM set.

[0382] In some examples, the extended IPM set is signaled with the same flag.

[0383] In some examples, the at least one new encoding method includes at least one of: a palette mode, an intra block copy IBC mode, and a quantized residual block differential pulse code modulation mode (QR-BDPCM).

[0384] In some examples, the index range of the existing IPM set is [0, M-1], and the index range of the extended IPM set is modified to [0, M-1+K], where K is the number of at least one new encoding method to be included in the extended IPM set, where M and K are integers.

[0385] In some examples, the at least one new encoding method is labeled as No. Idx New IPMs, wherein the range of the IPM index of the at least one new encoding method IdxNew is [0, M-1+K].

[0386] In some examples, the at least one new encoding method is included in the extended IPM set after all existing IPMs.

[0387] In some examples, the IPM index for the palette mode is set to M.

[0388] In some examples, the IPM index for the IBC mode is set to M or M+1.

[0389] In some examples, the IPM index for the QR-BDPCM mode is set to M, M+1, or M+2.

[0390] In some examples, the at least one new encoding method is included in the extended IPM set before one of the existing IPMs.

[0391] In some examples, the at least one new encoding method is included in the extended IPM set before all existing IPMs.

[0392] In some examples, the IPM index for the palette mode is set to 0.

[0393] In some examples, the IPM index for the IBC mode is set to 0.

[0394] In some examples, the IPM index for the QR-BDPCM mode is set to zero.

[0395] In some examples, by making the original index not less than Idx New The index of the remaining IPM is increased by 1 to modify it.

[0396] In some examples, Idx New is predefined, or signaled, or depends on at least one of the following:

[0397] 1) The block dimension of the current video block,

[0398] 2) the current quantization parameter of the current video block,

[0399] 3) Indicates color formats including 4:2:0 and 4:4:4,

[0400] 4) Split / dual coding tree structure,

[0401] 5) slice / slice group type and / or picture type,

[0402] 6) An indication signaled in at least one of a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, and a slice header.

[0403] In some examples, whether and / or how to include a new encoding method into the existing IPM set depends on the block dimensions and / or decoding information of the current video block.

[0404] In some examples, for MxN blocks, the IBC mode is not allowed to be included in the extended IPM set, where the MxN blocks include at least one of 128x128, 128x64, and 64x128 blocks.

[0405] In some examples, for MxN blocks, palette modes are not allowed to be included in the extended IPM set, where the MxN blocks include at least one of 128x128, 128x64, and 64x128.

[0406] In some examples, for blocks with a width greater than K or a height greater than L, where K=32 and L=32, the BDPCM mode is not allowed to be included in the extended IPM set.

[0407] Fig.23A flow chart of an example method for video processing is shown. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, generating (2302) a modified most probable intra-frame mode (MPM) list associated with the current video block, wherein the modified MPM list includes at least one new encoding method that is different from an existing intra-frame prediction mode (IPM) in an existing IPM set and is considered to be an IPM; and performing (2304) the conversion based on the modified MPM list.

[0408] In some examples, the at least one new encoding method includes at least one of the following: a palette mode, an intra block copy IBC mode, a quantized residual block differential pulse code modulation mode QR-BDPCM, and a matrix weighted intra prediction (MIP) mode.

[0409] In some examples, generating the modified MPM list includes: constructing an MPM list of existing intra-prediction modes; and replacing the modes in the MPM list with the at least one new encoding method to generate the modified MPM list.

[0410] In some examples, replacing the pattern in the MPM list with the at least one new encoding method includes: replacing the pattern at the first position in the MPM list with the at least one new encoding method.

[0411] In some examples, replacing the pattern in the MPM list with the at least one new encoding method includes: replacing the pattern at the last position in the MPM list with the at least one new encoding method.

[0412] In some examples, the index range of the existing IPM set is [0, M-1], and the IPM index for the palette mode is set to M, or the IPM index for the IBC mode is set to M or M+1, or the IPM index for the QR-BDPCM mode is set to M, M+1 or M+2, where M is an integer.

[0413] In some examples, during the construction of the MPM list, if a neighboring block of the current video block is encoded using the new encoding method, the neighboring block is marked as unavailable or considered as an inter-coded block.

[0414] In some examples, generating the modified MPM list includes: constructing an MPM list of existing intra-frame prediction modes; and adding the at least one new encoding method to a given position of the MPM list to generate the modified MPM list, wherein the size of the MPM list is increased by K, where K is the number of the new encoding modes.

[0415] In some examples, the position of the new coding mode depends on at least one of:

[0416] 1) the block dimension of the current video block,

[0417] 2) the prediction mode of the current video block,

[0418] 3) the current quantization parameter of the current video block,

[0419] 4) The palette flag of the adjacent block,

[0420] 5) Intra-block copy flag of adjacent blocks,

[0421] 6) Indication of color formats including 4:2:0 and 4:4:4,

[0422] 7) Separate or dual coding tree structure,

[0423] 8) slice, slice group type and / or picture type,

[0424] 9) Flag in at least one of the SPS, PPS, and slice header.

[0425] In some examples, generating the modified MPM list includes: if a neighboring block of the current video block is encoded using the at least one new encoding method, directly adding the at least one new encoding method as one of the MPMs to construct the modified MPM list.

[0426] In some examples, generating the modified MPM list includes: if a neighboring block of the current video block is encoded using the at least one new encoding method, constructing the modified MPM list, wherein an IPM index corresponding to the at least one new encoding method is directly added as one of the MPMs.

[0427] In some examples, whether and / or how to add a new coding method to the MPM list depends on the block dimensions and / or decoding information of the current video block.

[0428] In some examples, for certain block sizes, adding an IPM corresponding to the new coding mode to the MPM list is not allowed.

[0429] In some examples, for an M×N block, adding an IPM corresponding to an IBC mode to the MPM list is not allowed.

[0430] In some examples, for an MxN block, adding an IPM corresponding to a palette mode to the MPM list is not allowed.

[0431] In some examples, the MxN blocks include at least one of 128x128, 128x64, and 64x128 blocks.

[0432] In some examples, for blocks with a width greater than K or a height greater than L, an IPM corresponding to the BDPCM mode is not allowed to be included in the existing IPM set, where K=32 and L=32.

[0433] In some examples, whether and / or how to include a new coding mode in the MPM depends on an enabling flag for the new coding mode in at least one of the SPS, PPS, VPS, slice header, and slice header.

[0434] In some examples, the filtering process for the current video block depends on the IPM.

[0435] In some examples, the filtering process is associated with at least one of a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), a bilateral filter, and a Hadamard domain filter.

[0436] In some examples, a boundary strength (BS) setting in the deblocking filter depends on an index of the IPM.

[0437] In some examples, if the index represents palette mode, BS is set to 0.

[0438] Fig.24 A flowchart of an example method for video processing is shown. The method includes: for conversion between a chroma block of a video and a bitstream representation of the chroma block, determining (2402) whether the chroma block is a screen content block; in response to a determination indicating that the chroma block is a screen content block, modifying (2404) an intra-frame prediction process and / or a transform process of the chroma block; and performing (2406) the conversion based on the modified intra-frame prediction process and / or transform process.

[0439] In some examples, the chrominance blocks are determined to be screen content blocks when one or more selected luma blocks are encoded in IBC mode or palette mode.

[0440] In some examples, a selected luma block size is the smallest CU, PU or TU size or the unit used for motion or mode storage.

[0441] In some examples, a selected luma block is a CU, PU, ​​or TU covering a center, top left, top right, bottom left, or bottom right position of a corresponding luma region.

[0442] In some examples, when the upper left coordinate of the current chrominance block is (x0, y0) and the width and height of the current chrominance block are w0 and h0, respectively, the coordinates of the upper left sample point of the corresponding luminance area, the width and height of the corresponding luminance area are scaled according to the color format associated with the current chrominance block.

[0443] In some examples, for a 4:2:0 color format, the upper left coordinate of the co-located luma region is (2*x0, 2*y0), and the width and height of the co-located luma region are 2*w0 and 2*h0, respectively.

[0444] In some examples, for a 4:4:4 color format, the upper left coordinate of the co-located luma region is (x0, y0), and its width and height are w0 and h0, respectively.

[0445] In some examples, when the upper left coordinate of the co-located luminance region is (x, y) and its width and height are W and H, the coordinates of the center position are determined as one of the following:

[0446] 1)(x+W / 2,y+H / 2)

[0447] 2)(x+W / 2-1,y+H / 2-1)

[0448] 3)(x+W / 2,y+H / 2-1)

[0449] 4)(x+W / 2-1,y+H / 2).

[0450] In some examples, when the upper left coordinates of the co-located brightness region are (x, y) and its width and height are W and H, the coordinates of the upper left position are (x, y).

[0451] In some examples, when the top left coordinate of the co-located luma region is (x, y) and its width and height are W and H, the coordinate of the top right position is one of the following:

[0452] 1)(x+W,y)

[0453] 2)(x+W-1,y).

[0454] In some examples, when the upper left coordinate of the co-located luminance region is (x, y) and its width and height are W and H, the coordinate of the lower left position is one of the following:

[0455] 1)(x,y+H)

[0456] 2)(x,y+H-1).

[0457] In some examples, when the upper left coordinate of the co-located luminance region is (x, y) and its width and height are W and H, the coordinate of the lower right position is one of the following:

[0458] 1)(x+W,y+H)

[0459] 2)(x+W,y+H-1)

[0460] 3)(x+W-1,y+H)

[0461] 4)(x+W-1,y+H-1).

[0462] In some examples, certain intra prediction modes (IPMs) are not allowed when the chroma block is determined to be a screen content block.

[0463] In some examples, the certain intra prediction modes (IPM) include DC mode and / or PLANAR mode, VER and / or HOR mode or derived mode.

[0464] In some examples, when the chroma block is determined to be a screen content block, position-dependent intra prediction combining (PDPC) is not allowed.

[0465] In some examples, when the chroma block is determined to be a screen content block, mode dependent intra smoothing (MDIS) is not enabled.

[0466] Fig.25 A flow chart of an example method for video processing is shown. The method includes: performing (2502) a conversion between a current video block of a video and a bitstream representation of the video, wherein during the conversion, the positions of a left neighboring block and / or a top neighboring block used in a context derivation of one or more syntax elements are aligned with the positions of a left neighboring block and / or a top neighboring block used in a most probable intra mode (MPM) derivation.

[0467] In some examples, the syntax elements are associated with a Split Flag, a Mode Constraint Flag, a Skip Flag, a Prediction Mode Flag, an IBC Flag, a MIP Flag, an Affine Merge Flag, and an Affine Inter Flag.

[0468] In some examples, during the conversion, the positions of the left neighboring block and / or the upper neighboring block used in the context derivation of one or more syntax elements are aligned with the positions of the left neighboring block and / or the upper neighboring block used in the Merge or AMVP candidate list derivation process.

[0469] In some examples, the positions of the left neighboring block and / or the top neighboring block employed in the context derivation of one or more syntax elements are the bottom left and / or top right neighboring blocks.

[0470] Fig.26 A flow chart of an example method for video processing is shown. The method includes: for a conversion between a current video block of a video and a bitstream representation of the video, using multiple context signaling (2602) an indication of the use of a palette mode; and performing (2604) the conversion based on the indication.

[0471] In some examples, the context used to indicate the use of a palette mode is derived based on a left neighboring block and / or an above neighboring block.

[0472] In some examples, the context index used to indicate the use of the palette mode is derived by A*2+B, where A is the palette flag of the left neighboring block and B is the palette flag of the top neighboring block.

[0473] In some examples, the context index used to indicate the use of the palette mode is derived by A+B, where A is the palette flag of the left neighboring block and B is the palette flag of the top neighboring block.

[0474] In some examples, the positions of the left and top neighboring blocks used to derive a most probable intra mode (MPM) derivation are used to derive a context for encoding the palette mode flag.

[0475] In some examples, the positions of the left neighboring block and the above neighboring block used to derive the context for encoding the palette mode flag are the positions of the above-left neighboring block and / or the above-left neighboring block.

[0476] In some examples, whether and / or how to apply the above-mentioned generating process, determining process, executing process, or signaling process is based on at least one of the following:

[0477] a. Video content, which includes screen content or natural content;

[0478] b. A message signaled in at least one of a DPS, SPS, VPS, PPS, APS, picture header, slice header, slice group header, largest coding unit (LCU), coding unit (CU), LCU row, LCU group, TU, PU block, and video coding unit;

[0479] c. The position of at least one of CU, PU, ​​TU, block, and video coding unit;

[0480] d. the block dimensions of the current video block and / or its neighboring blocks;

[0481] e. the block shape of the current video block and / or its neighboring blocks;

[0482] f. an indication of a color format, wherein the color format includes 4:2:0, 4:4:4, RGB, or YUV;

[0483] g. Code tree structure, including dual tree or single tree;

[0484] h. Strip, slice group type and / or picture type;

[0485] i. Color components, including brightness components and / or chrominance components;

[0486] j. Time domain layer ID;

[0487] k. Profiles, levels, standard tiers.

[0488] In some examples, the performing the converting includes generating the bitstream representation from the current video block.

[0489] In some examples, the performing the converting includes generating the current video block from the bitstream representation.

[0490] The disclosed and other embodiments, modules and functional operations described herein may be implemented in digital electronic circuits or computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or a combination of one or more. The disclosed embodiments and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for a data processing device to execute or control its operation. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a combination of substances that affect a machine-readable propagated signal, or one or more combinations thereof. The term "data processing device" includes all devices, apparatuses and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, a device may also include code that creates an execution environment for a computer program, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0491] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0492] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0493] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory can be supplemented by dedicated logic circuits, or merged into dedicated logic circuits.

[0494] Although this patent document contains many details, they should not be interpreted as limitations on the scope of any invention or the claims, but rather as descriptions of features of certain embodiments of certain inventions. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially claimed to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the claim combination can be directed to subcombinations or variations of subcombinations.

[0495] Likewise, although operations are described in certain orders in the drawings, this should not be understood as requiring that such operations must be performed in the certain order or order shown, or that all described operations must be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments of this patent document should not be understood as requiring such separation in all embodiments.

[0496] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For a conversion between a current video block of a video and a bitstream representation of the video, generating an extended set of intra-prediction modes (IPMs) associated with the current video block by modifying an existing set of IPMs to include at least one new coding method, wherein the new coding method is different from a coding method in the existing IPMs and is considered an IPM; and performing the conversion based on the extended IPM set, The at least one new encoding method includes at least one of the following: a palette mode, an intra-block copy IBC mode, and a quantized residual block differential pulse code modulation mode (QR-BDPCM).

2. The method according to claim 1, wherein: The extended IPM set is signaled with the same flag.

3. The method according to claim 1, wherein: The index range of the existing IPM set is [0, M-1], and the index range of the extended IPM set is modified to [0, M-1 + K], where K is the number of at least one new encoding method to be included in the extended IPM set, where M and K are integers.

4. The method according to claim 3, wherein: The at least one new encoding method is marked as Idx New IPM, wherein the at least one new encoding method Idx New The range of the IPM index is [0, M-1 + K].

5. The method according to claim 4, wherein: The at least one new encoding method is included in the extended IPM set, after all existing IPMs.

6. The method according to claim 5, wherein: The IPM index for the palette mode is set to M.

7. The method according to claim 5, wherein: The IPM index for the IBC mode is set to M or M+1.

8. The method according to claim 5, wherein: The IPM index for the QR-BDPCM mode is set to M, M+1 or M+2.

9. The method according to claim 4, wherein: The at least one new encoding method is included in the extended IPM set before one of the existing IPMs.

10. The method according to claim 9, wherein: The at least one new encoding method is included in the extended IPM set, before all existing IPMs.

11. The method according to claim 10, wherein: The IPM index for the palette mode is set to 0.

12. The method according to claim 10, wherein: The IPM index for the IBC mode is set to 0.

13. The method according to claim 10, wherein: The IPM index for the QR-BDPCM mode is set to 0.

14. The method according to claim 9, wherein: By making the original index not less than Idx New The index of the remaining IPM is increased by 1 to modify it.

15. The method according to any one of claims 9 to 14, wherein: Idx New is predefined, or signaled, or depends on at least one of the following: 1) The block dimension of the current video block, 2) The current quantization parameter of the current video block, 3) Indicates color formats including 4:2:0 and 4:4:4, 4) Split / dual encoding tree structure, 5) Strip / slice group type and / or picture type, 6) An indication signaled in at least one of a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, and a slice header.

16. The method according to claim 1, wherein: Whether and / or how to include a new encoding method into the existing IPM set depends on the block dimension and / or decoding information of the current video block.

17. The method according to claim 16, wherein: For MxN blocks, the IBC mode is not allowed to be included in the extended IPM set, wherein the MxN blocks include at least one of 128x128, 128x64, and 64x128 blocks.

18. The method according to claim 16, wherein: For an MxN block, the palette mode is not allowed to be included in the extended IPM set, wherein the MxN block includes at least one of 128x128, 128x64, and 64x128.

19. The method according to claim 16, wherein: For blocks with width greater than K or height greater than L, the BDPCM mode is not allowed to be included in the extended IPM set, where K=32, L=32.

20. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein: When the instructions are executed by a processor, the processor is caused to implement the method of any one of claims 1 to 19 .

21. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising a program code for executing the method according to any one of claims 1 to 19.

22. A non-transitory computer-readable medium recording a program code for executing the method according to any one of claims 1 to 19.