Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is presented. The method comprises the steps of determining a first intra-frame prediction sample of a current video block for conversion between the current video block of a video and a bit stream of the video; determining an adjustment value associated with another video block in the video, the another video block being coded and decoded prior to the current video block; and performing conversion based on the first intra prediction sample and the adjustment value.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 370,198, filed on August 2, 2022, entitled “METHOD, APPARATUS, AND MEDIUM FOR VIDEO PROCESSING,” the entire contents of which are expressly incorporated herein by reference. Technical Field

[0003] Embodiments of the present disclosure relate generally to video coding techniques, and more particularly to prediction enhancement. Background Art

[0004] Nowadays, digital video functions are being applied to all aspects of people's lives. For video encoding / decoding, various types of video compression technologies have been proposed, such as Moving Picture Experts Group (MPEG)-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency of conventional video encoding and decoding technologies is usually very low, which is undesirable. Summary of the invention

[0005] Embodiments of the present disclosure provide a solution for video processing.

[0006] In a first aspect, a method for video processing is proposed. The method includes: determining a first intra-frame prediction sample of a current video block for a conversion between a current video block of a video and a bitstream of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; and performing the conversion based on the first intra-frame prediction sample and the adjustment value. The method according to the first aspect of the present disclosure adjusts the prediction of the current video block, such as the intra-frame prediction of the current video block, based on the adjustment value associated with another encoded and decoded block. Compared with conventional solutions, the method proposed in the first aspect can advantageously improve the coding effectiveness and coding efficiency.

[0007] In a second aspect, an apparatus for processing video data is provided. The apparatus for processing video data comprises a processor and a non-volatile memory having instructions thereon. These instructions, when executed by the processor, cause the processor to perform the method according to the first aspect of the present disclosure.

[0008] In a third aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions for causing a processor to execute the method according to the first aspect of the present disclosure.

[0009] In a fourth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; and generating a bitstream based on the first intra-frame prediction sample and the adjustment value.

[0010] In a fifth aspect, a method for storing a bitstream of a video is provided. The method includes: determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; generating a bitstream based on the first intra-frame prediction sample and the adjustment value; and storing the bitstream in a non-transitory computer-readable recording medium.

[0011] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of example embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings, in which like reference numerals generally refer to like components throughout the example embodiments of the present disclosure.

[0013] Figure 1 A block diagram illustrating an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0014] Figure 2 shows a block diagram illustrating a first example video encoder according to some embodiments of the present disclosure;

[0015] Figure 3 shows a block diagram illustrating an example video decoder according to some embodiments of the present disclosure;

[0016] Figure 4 An example of an encoder block diagram is shown;

[0017] Figure 5 67 intra prediction modes are shown;

[0018] Figure 6 Reference samples for wide-angle intra prediction are shown;

[0019] Figure 7 The discontinuity problem is shown when the orientation exceeds 45°;

[0020] Figure 8 The locations of the samples used for the derivation of α and β are shown;

[0021] Fig. 9A A schematic diagram showing the definition of samples used by PDPC applied to diagonal top right mode and adjacent angle intra mode;

[0022] Fig. 9B A schematic diagram showing the definition of samples used by PDPC for diagonal bottom left mode and adjacent angle intra mode applied to diagonal lines;

[0023] Fig. 9C A schematic diagram showing the definition of samples used by PDPC applied to diagonal adjacent diagonal top right mode and adjacent angle intra mode;

[0024] Fig.9D A schematic diagram showing the definition of samples used by PDPC applied to diagonal adjacent diagonal bottom left mode and adjacent angle intra mode;

[0025] Fig.10 A gradient method for non-vertical / non-horizontal modes is shown;

[0026] Fig.11 The value of nScale is shown with respect to nTbH and the number of modes; for all cases where nScale<0, the gradient method is used;

[0027] Fig.12 Flowcharts showing conventional PDPC and the proposed PDPC;

[0028] Fig.13 Neighboring blocks (L, A, BL, AR, AL) used in the derivation of a general MPM list are shown;

[0029] Fig.14 An example of the proposed intra reference mapping is shown;

[0030] Fig.15 An example of four reference lines adjacent to a prediction block is shown;

[0031] Fig.16A A schematic diagram showing the subdivision process according to block size is shown;

[0032] Fig. 16B A schematic diagram showing the subdivision process according to block size is shown;

[0033] Fig.17 The matrix-weighted intra prediction process is shown;

[0034] Fig.18shows a target sample, a template sample, and a reference sample of a template used in DIMD;

[0035] Fig.19 shows a set of selected pixels on which gradient analysis is performed;

[0036] Fig. 20 The convolution of a 3x3 Sobel gradient filter with a template is shown;

[0037] Fig.21 The proposed intra-block decoding process is shown;

[0038] Fig. 22 HoG computation from a template of width 3 pixels is shown;

[0039] Fig.23 shows the prediction fusion by weighted averaging of two HoG modes and planes;

[0040] Fig.24 The top and left neighboring blocks used in the derivation of CIIP weights are shown;

[0041] Fig.25 An example of GPM segmentation grouped by the same angle is shown;

[0042] Fig.26 Unidirectional prediction MV selection for geometric partitioning mode is shown;

[0043] Fig. 27 An example generation of blending weights W0 using a geometric partitioning pattern is shown;

[0044] Fig.28 A conventional angle IPM and an extended angle IPM are shown;

[0045] Fig.29 The current block and the pseudo block are shown;

[0046] Fig.30 A flow chart showing the use of a pseudo residual block at an encoder;

[0047] Fig.31 A flow chart showing the use of a pseudo residual block at a decoder;

[0048] Fig.32 A flowchart showing a method for video processing according to some embodiments of the present disclosure; and

[0049] Fig.33 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0050] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0051] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0052] In the following description and claims, unless defined otherwise, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0053] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0054] It should be understood that although the terms "first" and "second" etc. can be used to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0055] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "comprising", and / or "having" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0056] Example Environment

[0057] Figure 11 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0058] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0059] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0060] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0061] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or further standards.

[0062] Figure 2is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0063] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0064] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0065] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0066] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0067] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0068] The mode selection unit 203 may select one of a plurality of codec modes (intra-frame codec or inter-frame codec), for example, based on the error result, and provide the generated intra-frame codec block or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coding block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0069] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0070] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0071] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0072] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0073] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0074] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0075] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0076] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0077] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0078] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0079] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0080] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0081] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0082] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0083] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0084] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0085] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0086] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0087] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0088] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy-encoded video data, and the motion compensation unit 302 can determine motion information from the entropy-decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of adjacent PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in space or time.

[0089] The motion compensation unit 302 may generate motion compensated blocks, possibly performing interpolation based on interpolation filters.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0090] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0091] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coding block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0092] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0093] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0094] Some example embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video coding and decoding technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of canceling the encoding will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates.

[0095] 1. Brief Overview

[0096] The present disclosure relates to video coding technology. Specifically, it is related intra-frame prediction in image / video coding. It can be applied to existing video coding standards, such as HEVC or universal video codec (VVC). It may also be applicable to future video coding standards or video codecs.

[0097] 2. Introduction

[0098] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.264 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.264 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that utilizes temporal prediction plus transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was created to work on the VVC standard with the goal of a 50% bitrate reduction compared to HEVC.

[0099] The latest version of the VVC draft, the Universal Video Codec (Draft 10), can be found at:

[0100] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 20_Teleconference / wg11 / JVET-T2001-v1.zip

[0101] The latest reference software for VVC, called VTM, can be found at:

[0102] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-11.0

[0103] 2.1. Encoding and decoding process of typical video codecs

[0104] Figure 4An example of an encoder block diagram 400 of VVC is shown, which contains three in-loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO) and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, where the coding side information transmits the offset and filter coefficients through the signal. ALF is located at the last processing stage of each picture and can be regarded as a tool that tries to obtain and repair artifacts produced by previous stages.

[0105] 2.2. Intra-mode codec with 67 intra-prediction modes

[0106] Figure 5 67 intra prediction modes are shown. To capture arbitrary edge directions present in natural videos, the number of directional intra modes is extended from 33 used in HEVC to 65, as shown in Figure 5 As shown, planar mode and DC mode remain unchanged. These dense directional intra prediction modes are applicable to all block sizes and to both luma intra prediction and chroma intra prediction.

[0107] In HEVC, each intra-coded block has a square shape, and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate the intra predictor using the DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of a division operation for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average value for non-square blocks.

[0108] 2.2.1 Wide-angle intra-frame prediction

[0109] Although 67 modes are defined in VVC, the precise prediction direction for a given intra prediction mode index further depends on the block shape. The traditional angular intra prediction direction is defined as from 45 degrees to -135 degrees in a clockwise direction. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced mode is notified using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged.

[0110] Figure 6 The reference samples for wide angle frame prediction are shown. To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined, as Figure 6 shown.

[0111] The number of modes replaced in the wide angle direction mode depends on the aspect ratio of the block. The replaced intra prediction modes are shown in Table 2-1.

[0112] Table 2-1 Intra-frame prediction modes replaced by wide-angle mode

[0113]

[0114]

[0115] Figure 7 The figure shows the problem of discontinuity when the orientation exceeds 45 degrees. Figure 7 As shown, in the case of wide-angle intra prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and side smoothing are applied to wide-angle prediction to reduce the added gap Δp α The negative impact brought by this. If the wide-angle mode represents a non-fractional offset. There are 8 modes in the wide-angle mode that meet this condition, which are [-14, -12, -10, -6, 72, 76, 78, 80]. When predicting a block through these modes, the samples in the reference buffer are directly copied without applying any interpolation. With this modification, the number of samples that need to be smoothed is reduced. In addition, it will align the design of non-fractional modes in the traditional prediction mode and the wide-angle mode.

[0116] In VVC, 4:2:2 and 4:4:4 chroma formats are supported, as well as the 4:2:0 chroma format. The chroma derivation mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to align with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes ranging from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the entries of the mapping table to more accurately convert the prediction angles of the chroma blocks.

[0117] 2.3. Inter-frame prediction

[0118] For each inter-frame prediction CU, the motion parameters include motion vectors, reference picture indices, and reference picture list usage indices, as well as additional information required for the new coding features of VVC that will be used for inter-frame prediction sample generation. Motion parameters can be signaled in an explicit or implicit manner. When a CU is encoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector differences (delta) or reference picture indices. A merge mode is specified in which the motion parameters of the current CU are obtained from neighboring CUs, including spatial candidates and temporal candidates, as well as additional scheduling introduced in VVC. Merge mode can be applied to any inter-frame prediction CU, not just skip mode. An alternative to merge mode is explicit transmission of motion parameters, where motion vectors, corresponding reference picture indices for each reference picture list, and reference picture list usage flags, as well as other required information, are explicitly signaled per CU.

[0119] 2.4. Intra-block copy (IBC)

[0120] Intra-block copy (IBC) is a tool adopted in the HEVC extension on SCC. It is well known that it significantly improves the coding efficiency of screen content materials. Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block, which has been reconstructed within the current picture. The luminance block vector of the CU encoded and decoded by IBC has integer precision. The chrominance block vector is also rounded to integer precision. When used in conjunction with AMVR, the IBC mode can switch between 1-pixel and 4-pixel motion vector precision. The CU encoded and decoded by IBC is regarded as a third prediction mode in addition to the intra or inter prediction mode. The IBC mode is applicable to CUs with a width and height that are both less than or equal to 64 luminance samples. On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks on blocks with a width or height of no more than 16 luminance samples. For non-merge mode, a block vector search is first performed using a hash-based search. If the hash search returns no valid candidates, a local search based on block matching will be performed.

[0121] In hash-based search, the hash key matching (32-bit CRC) between the current block and the reference blocks is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4×4 sub-blocks. For current blocks of larger size, the hash key is determined to match the hash key of the reference block when all hash keys in all 4×4 sub-blocks match the hash keys in the corresponding reference positions. If multiple reference blocks are found whose hash keys match the hash key of the current block, the block vector cost of each matching reference is calculated and the one with the smallest cost is selected.

[0122] In the block matching search, the search range is set to cover the previous CTU and the current CTU.

[0123] At the CU level, the IBC mode is signaled through a flag, which can be signaled as IBC AMVP mode or IBC Skip / Merge mode as shown below:

[0124] – IBC skip / merge mode: The merge candidate index is used to indicate which block vector from the list of neighboring candidate IBC codec blocks is used to predict the current block. The merge list includes spatial candidates, HMVP candidates, and pairwise candidates.

[0125] – IBC AMVP mode: Block vector differences are encoded and decoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (in case of IBC codec). When either neighbor is not available, the default block vector is used as predictor. A flag is signaled to indicate the block vector predictor index.

[0126] 2.5. Cross-component linear model prediction

[0127] In order to reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC. For this CCLM prediction mode, chroma samples are predicted based on the reconstructed luma samples of the same CU by using the following linear model:

[0128] pred C (i, j) = α·rec L ′(i,j)+β (2-1)

[0129] where pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i, j) represents the downsampled reconstructed luma samples of the same CU.

[0130] The CCLM parameters (α and β) are derived using up to four neighboring chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is W×H, W' and H' are set to

[0131] - When LM mode is applied, W'=W, H'=H;

[0132] -When using LM-T mode, W'=W+H;

[0133] - When LM-L mode is applied, H'=H+W.

[0134] The upper neighboring position is denoted as S[0, -1] ... S[W'-1, -1], and the left neighboring position is denoted as S[-1, 0] ... S[-1, H'-1]. Then, four samples are selected as:

[0135] - When LM mode is applied and both the upper and left neighboring samples are available, S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4];

[0136] - When the LMT mode is applied or only upper neighbor samples are available, S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1];

[0137] -S[-1,H' / 8],S[-1,3*H' / 8],S[-1,5*H' / 8],S[-1,7*H' / 8] when LM-L mode is applied or only left neighbor samples are available.

[0138] The four neighboring brightness samples at the chosen position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented by y 0 A ,y 1 A ,y 0 B and 1 B Then x A 、x B ,y A and B Export as:

[0139] X a =(x 0 A +x 1 A +1)>>1;X b =(x 0 B +x 1 B +1)>>1;Y a =(y 0 A +y 1 A+1)>>1;Y b =(y 0 B +y 1 B +1)>>1(2-2)Finally, the parameters α and β of the linear model are obtained according to the following formula.

[0140]

[0141] B=Y b -α·X b (2-4)

[0142] Figure 8 The locations of the samples used for the derivation of α and β are shown. Figure 8 Examples of the positions of the left and upper samples and the samples of the current block involved in CCLM mode are shown.

[0143] The division operation is implemented using a lookup table to calculate the parameter. In order to reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed as exponents. For example, diff is approximated by a 4-bit significant part and an exponent. Therefore, the table for 1 / diff is reduced to 16 elements for 16 values ​​of the significant bit, as shown below:

[0144] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,0} (2-5)

[0145] This will help reduce the complexity of the calculations and the memory size required to store the required tables.

[0146] In addition to the upper and left templates being used together to calculate linear model coefficients, they can also be used alternately in two other LM modes, called LM_T and LM_L modes.

[0147] In LM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0148] In LM mode, the left template and the upper template are used to calculate the linear model coefficients.

[0149] To match the chroma sampling positions for 4:2:0 video sequences, two types of downsampling filters are applied to luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0150]

[0151] Note that when the upper reference line is at a CTU boundary, only one luma line (normal line buffer in intra prediction) is used for downsampling luma samples.

[0152] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to communicate the α and β values ​​to the decoder.

[0153] For chroma intra mode coding and decoding, a total of 8 intra modes are allowed for chroma intra mode coding and decoding. These modes include five traditional intra modes and three cross-component linear model modes (LM, LM_T and LM_L). The chroma mode signaling and derivation process are shown in Table 2-3. Chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0154] Table 2-2 Chroma prediction mode derived from luma mode when CCLM is enabled

[0155]

[0156] As shown in Table 2-3, a single binarization table is used regardless of the value of sps_cclm_enabled_flag.

[0157] Table 2-3 Unified binarization table for chroma prediction mode

[0158] Value of intra_chroma_pred_mode Binary value string 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0159] In Table 2-3, the first binary value indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next binary value indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next 1 binary value indicates whether it is LM_L (0) or LM_T (1). For this case, when sps_cclm_enabled_flag is 0, the first binary value of the corresponding intra_chroma_pred_mode binarization table can be discarded before entropy encoding and decoding. Or, in other words, the first binary value is inferred to be 0 and therefore not encoded and decoded. This single binarization table is used for both cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two binary values ​​in Table 2-4 are context encoded and decoded with their own context models, and the remaining binary values ​​are bypass encoded and decoded. In addition, to reduce luma-chroma latency in dual trees, when a 64×64 luma codec tree node is split with Not Split (and ISP is not used for 64×64 CUs) or QT, the chroma CUs in the 32×32 / 32×16 chroma codec tree nodes are allowed to use CCLM in the following way:

[0160] – If a 32×32 chroma node is not split or split QT split, all chroma CUs in the 32×32 node can use CCLM

[0161] – If a 32×32 chroma node is split with horizontal BT and the 32×16 child nodes are not split or use vertical BT split, all chroma CUs in the 32×16 chroma node can use CCLM.

[0162] Under all other luma and chroma codec tree partitioning conditions, CCLM is not allowed for chroma CUs.

[0163] 2.6. Position-dependent intra prediction combination

[0164] In VVC, the intra prediction results for DC, planar, and several angular modes are further modified by the Position Dependent Intra Prediction Combination (PDPC) method. PDPC is an intra prediction method that calls for a combination of boundary reference samples and HEVC-style intra prediction with filtered boundary reference samples. PDPC is applied to the following intra modes without signaling: planar, DC, intra angles less than or equal to horizontal, and intra angles greater than or equal to vertical and less than or equal to 80. PDPC is not applied if the current block is in BDPCM mode or the MRL index is greater than 0.

[0165] The prediction sample pred(x',y') is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference sample according to the following equation 2-8:

[0166] pred(x',y')=Clip(0,(1< <BitDepth)–1,(wL×R -1,y’ +wT×R x’,-1 +(64-wL-wT)×pred(x',y')+32)>>6) (2-8)

[0167] Where R x,-1 ,R -1,y Respectively represent the reference samples located at the top and left boundaries of the current sample (x, y).

[0168] If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filtering is required, as required in the case of HEVC DC mode boundary filtering or horizontal / vertical mode edge filtering. The PDPC process is identical for DC mode and planar mode. For angular modes, if the current angular mode is HOR_IDX or VER_IDX, the left or top reference samples are not used, respectively. The PDPC weights and scaling factors depend on the prediction mode and block size. PDPC is applied to blocks with width and height both greater than or equal to 4.

[0169] Figures 9A to 9D The reference samples (R x,-1 and R -1,y ) definition. Fig. 9A A schematic diagram showing the definition of samples used by PDPC applied to diagonal top-right mode and adjacent angle intra mode. Fig. 9B A schematic diagram showing the definition of samples used by PDPC applied to the diagonal bottom-left mode and the adjacent angle intra mode. Fig. 9C A schematic diagram showing the definition of samples used by PDPC applied to the diagonal adjacent diagonal top-right mode and the adjacent angle intra mode. Fig.9D A schematic diagram showing the definition of samples used by PDPC applied to the diagonal adjacent diagonal bottom-left mode and the adjacent angular intra mode.

[0170] The prediction sample pred(x',y') is located at (x',y') in the prediction block. For example, the reference sample R x,-1 The coordinate x of is given by: x = x' + y' + 1, with reference to sample R -1,y The coordinate y of is similarly given by y=x'+y'+1 for the diagonal pattern. For other angular patterns, the reference sample R x,-1 and R -1,y Can be at a fractional sample position. In this case, the sample value at the nearest integer sample position will be used.

[0171] 2.7. Gradient PDPC

[0172] Fig.10 An example graph 1000 gradient mode for non-vertical / non-horizontal mode is shown. Fig.10 As shown in Figure 1, the gradient-based approach is extended to non-vertical / non-horizontal modes. Here, the gradient is calculated as r(-1,y)–r(-1+d,-1), where d is the horizontal displacement depending on the angular direction. A few points to note here:

[0173] The gradient term r(-1,y)–r(-1+d,-1) needs to be calculated once per row because it does not depend on the x position.

[0174] The calculation of d is already part of the original intra prediction process which can be reused, so there is no need to calculate d separately. Therefore, the accuracy of d is 1 / 32 pixel.

[0175] When d is in fractional position, two-tap (linear) filtering is used, i.e., if dPos is the displacement with 1 / 32 pixel precision, dInt is the (rounded down) integer part (dPos>>5), and dFract is the fractional part with 1 / 32 pixel precision (dPos&31), then r(-1+d) is calculated as:

[0176] r(-1+d)=(32–dFrac)*r(-1+dInt)+dFrac*r(-1+dInt+1)

[0177] As described in a, 2-tap filtering is performed once per row (if necessary).

[0178] Finally, the prediction signal is calculated:

[0179] p(x,y)=Clip(((64–wL(x))*p(x,y)+wL(x)*(r(-1,y)-r(-1+d,-1))+32)>>6)

[0180] Where wL(x)=32>>((x<<1)>>nScale2), and nScale2=(log2(nTbH)+log2(nTbW)–2)>>2, which are the same as the vertical / horizontal mode. In short, the same process is applied compared to the vertical / horizontal mode (in fact, d=0 indicates the vertical / horizontal mode).

[0181] Second, in non-vertical / non-horizontal modes, when (nScale < 0) or when PDPC cannot be applied due to unavailability of auxiliary reference samples, the gradient-based method is activated. Fig.11 The values ​​of nScale are shown in relation to TB size and angular mode to better visualize the use of the gradient method. In addition, Fig.12 , a flow chart 1200 of current and proposed PDPC is shown.

[0182] 2.8. Auxiliary MPM

[0183] The secondary MPM list is introduced. The existing primary MPM (PMPM) list includes 6 entries, while the secondary MPM (SMPM) list includes 16 entries. First, a general MPM list with 22 entries is constructed, and then the first 6 entries in the general MPM list are included in the PMPM list, and the remaining entries form the SMPM list. The first entry in the general MPM list is the flat mode. Fig.13 As shown, the remaining entries consist of the intra modes of the left (L), above (A), lower left (BL), upper right (AR), and upper left (AL) neighboring blocks, directional modes with additional offsets from the first two available directional modes of the neighboring blocks, and a default mode.

[0184] If the CU block is vertically oriented, the order of neighboring blocks is A, L, BL, AR, AL; otherwise, it is L, A, BL, AR, AL.

[0185] First parse the PMPM flag, if it is equal to 1, parse the PMPM index to determine which entry of the PMPM list is selected, otherwise parse the SPMPM flag to determine whether to parse the SMPM index or the remaining modes.

[0186] 2.9.6 Tap Intra-frame Interpolation Filter

[0187] In order to improve the prediction accuracy, it is proposed to replace the 4-tap cubic interpolation filter with a 6-tap interpolation filter, and the filter coefficients are derived based on the same polynomial regression model (whose polynomial order is 6).

[0188] The filter coefficients are shown below,

[0189] {0,0,256,0,0,0}, / / 0 / 32 position

[0190] {0, -4, 253, 9, -2, 0}, / / 1 / 32 position

[0191] {1, -7, 249, 17, -4, 0}, / / 2 / 32 position

[0192] {1, -10, 245, 25, -6, 1}, / / 3 / 32 position

[0193] {1, -13, 241, 34, -8, 1}, / / 4 / 32 position

[0194] {2, -16, 235, 44, -10, 1}, / / 5 / 32 position

[0195] {2, -18, 229, 53, -12, 2}, / / 6 / 32 position

[0196] {2, -20, 223, 63, -14, 2}, / / 7 / 32 position

[0197] {2, -22, 217, 72, -15, 2}, / / 8 / 32 position

[0198] {3, -23, 209, 82, -17, 2}, / / 9 / 32 position

[0199] {3, -24, 202, 92, -19, 2}, / / 10 / 32 position

[0200] {3, -25, 194, 101, -20, 3}, / / 11 / 32 position

[0201] {3, -25, 185, 111, -21, 3}, / / 12 / 32 position

[0202] {3, -26, 178, 121, -23, 3}, / / 13 / 32 position

[0203] {3, -25, 168, 131, -24, 3}, / / 14 / 32 position

[0204] {3, -25, 159, 141, -25, 3}, / / 15 / 32 position

[0205] {3, -25, 150, 150, -25, 3}, / / half pixel position

[0206] The reference samples used for interpolation come from the reconstructed samples or the padding samples in HEVC, so no conditional check on the reference sample availability is required.

[0207] It is proposed to use a 4-tap cubic interpolation filter instead of using a round-to-nearest operation to derive extended intra reference samples. Fig.14 An example diagram 1400 on the proposed intra-frame reference mapping is shown. Fig.14 As shown in the example in , a four-tap interpolation filter is used to derive the value of the reference sample P, while in JEM-3.0 or HM, P is directly set to X1.

[0208] 2.10. Multiple Reference Line (MRL) Intra Prediction

[0209] Fig.15 An example of four reference lines adjacent to the prediction block is shown. Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Fig.15In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0210] The index of the selected reference row (mrl_idx) is signaled and used to generate intra predictions. For reference row indices greater than 0, only additional reference row modes are included in the MPM list, and only the MPM index is signaled without the remaining modes. The reference row index is signaled before the intra prediction mode, and if the reference row index is non-zero, the intra prediction mode does not include the planar mode.

[0211] MRL is disabled for the first row of a block within a CTU to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC will be disabled when additional rows are used. For MRL mode, the derivation of DC values ​​in DC intra-frame prediction mode for non-zero reference row indexes is aligned with the derivation of reference row index 0. MRL requires storage of 3 neighboring luma reference rows with a CTU to generate predictions. The downsampling filtering of the Cross Component Linear Model (CCLM) tool also requires 3 neighboring luma reference rows. The definition of MRL using the same 3 rows is consistent with CCLM to reduce the storage requirements of the decoder.

[0212] 2.11. Intra-frame sub-partitioning (ISP)

[0213] Intra sub-partitioning (ISP) divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally depending on the block size. For example, the minimum block size of ISP is 4×8 (or 8×4). If the block size is larger than 4×8 (or 8×4), the corresponding block will be divided into four sub-partitions. We note that M×128 (M≤64) and 128×N (N≤64) ISP blocks may generate potential problems for 64×64 VDPU. For example, an M×128 CU in the single-tree case has one M×128 luma TB and two corresponding Chroma TB. If the CU uses ISP, then the luma TB will be divided into 4 M×32TBs (can only be split horizontally), each of which is smaller than a 64×64 block. However, in the current ISP design, the chroma block is indivisible. Therefore, the size of both chroma components will be larger than a 32×32 block. Similarly, a 128×NCU using ISP can also create a similar situation. Therefore, these two situations are problems for the 64×64 decoder pipeline. Therefore, the CU size that can use ISP is limited to a maximum value of 64×64. Figures 16A to 16B Examples of two possibilities are shown. Fig.16AA schematic diagram 1600 is shown of the process of subdivision according to block size. Fig. 16B A schematic diagram 1650 showing the process of subdivision according to block size is shown. All subdivisions satisfy the condition of having at least 16 samples.

[0214] In ISP, 1×N / 2×N sub-block prediction allows to rely on the reconstructed values ​​of previously decoded 1×N / 2×N sub-blocks of the codec block, making the minimum prediction width of the sub-block become four samples. For example, an 8×N (N>4) codec block, when using ISP vertical partitioning codec, is split into two prediction regions, each of size 4×N, and four transforms of size 2×N. Similarly, a 4×N codec block using vertical partitioning ISP codec uses the full 4×N block for prediction; four transforms are used, each of size 1×N. Although 1×N and 2×N transform sizes are allowed, it can be asserted that the transforms of these blocks within the 4×N region can be performed in parallel. For example, when a 4×N prediction region contains four 1×N transforms, there is no transform in the horizontal direction; the transform in the vertical direction can be performed as a single 4×N transform in the vertical direction. Similarly, when a 4×N prediction region contains two 2×N transform blocks, the transform operations of the two 2×N blocks in each direction (horizontally and vertically) can be performed in parallel. Therefore, there is no added latency when processing these smaller blocks compared to processing intra blocks for a 4x4 regular codec.

[0215] Table 2-4 Entropy encoding and decoding coefficient group size

[0216] Block size Coefficient group size 1×N, N≥16 1×16 N×1, N≥16 16×1 2×N, N≥8 2×8 N×2, N>8 8×2 All other possible M×N situations 4×4

[0217] For each subpartition, the reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each subpartition can be used to generate the prediction of the next subpartition, and each subpartition is processed repeatedly. In addition, the first subpartition to be processed is the subpartition containing the samples at the top left of the CU, and then continues down (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the subpartition prediction signal are only located to the left and above the line. All subpartitions share the same intra-frame mode. The following is a summary of the interaction of ISP with other codec tools.

[0218] – Multiple Reference Line (MRL): If the MRL index of a block is not 0, the ISP codec mode will be inferred to be 0, so the ISP mode information will not be sent to the decoder.

[0219] – Entropy Codec Coefficient Group Size: As shown in Table 2-4, the size of the entropy codec sub-blocks has been modified to have 16 samples in all possible cases. It is worth noting that the new size only affects blocks where one of the ISP dimensions is less than 4 samples. In all other cases, the coefficient groups maintain dimensionality.

[0220] – CBF codec: It is assumed that at least one subpartition has a non-zero CBF. Thus, if n is the number of subpartitions, and the first n-1 subpartitions already yield zero CBF, the CBF of the nth subpartition is inferred to be 1.

[0221] – Transform size restriction: All ISP transforms with length greater than 16 points use DCT-II.

[0222] –MTS flag: If the CU uses ISP codec mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Therefore, the encoder will not perform RD tests on the different available transforms for each result sub-partition. The transform selection for ISP mode will instead be fixed and will be chosen based on the intra mode used, the processing order and the block size. Therefore, no signaling is required. For example, let t H and t V and are the horizontal and vertical transforms selected for the w×h sub-partition respectively, where w is the width and h is the height.

[0223] The transformation is then selected according to the following rules:

[0224] - If w=1 or h=1, there is no horizontal transform or vertical transform, respectively.

[0225] – If w ≥ 4 and w ≤ 16, t H =DST-VII, otherwise t H =DCT-II

[0226] – If h≥4 and h≤16, t V =DST-VII, otherwise t V =DCT-II

[0227] In ISP mode, all 67 intra prediction modes are allowed. PDPC is also applied if the corresponding width and height are at least 4 samples long. In addition, the conditions for the reference sample filtering process (reference smoothing) and intra interpolation filter selection no longer exist, and in ISP mode, fractional position interpolation always applies a cubic (DCT-IF) filter.

[0228] 2.12. Matrix Weighted Intra Prediction (MIP)

[0229] The matrix weighted intra prediction (MIP) method is a new intra prediction technique added to VVC. In order to predict the samples of a rectangular block of width W and height H, the matrix weighted intra prediction (MIP) takes a row of H reconstructed adjacent boundary samples on the left side of the block and a row of W reconstructed adjacent edge samples above the block as input. If the reconstructed samples are not available, they are generated as in traditional intra prediction. Fig.17 17 shows a matrix weighted intra prediction process 1700. The generation of the prediction signal is based on the following three steps, namely averaging, matrix-vector multiplication and linear interpolation, as shown in FIG. Fig.17 shown.

[0230] 2.12.1 Average Neighboring Samples

[0231] Among the boundary samples, four samples or eight samples are selected by averaging based on the block size and shape. Specifically, input boundary bdry top and bdry left It is reduced to a smaller size by averaging the neighboring boundary samples according to a predefined rule that depends on the size of the block. and Then, the two reduced boundaries and Connect to the reduced boundary vector bdry red , so for blocks of shape, the size of the reduced boundary vector is 4×4, while for blocks of all other shapes, the size of the reduced edge vector is 8. If mode refers to a MIP mode, then this cascade is defined as follows:

[0232]

[0233] 2.12.2 Matrix multiplication

[0234] Taking the average sample as input, a matrix-vector multiplication is performed and then an offset is added. The result is a downscaled prediction signal generated on a subsampled set of samples in the original block. red Generate a reduced prediction signal pred red , the reduced prediction is of width W red and height H red Here, W red and H red Defined as:

[0235]

[0236] The reduced prediction signal pred is calculated by computing the matrix-vector product and adding an offset red :

[0237] predred =A.bdry red +b (2-12)

[0238] Here, A is a matrix that has W if W = H = 4z. red ·H red rows and 4 columns, in all other cases there are 8 columns. b is W red ·H red The matrix A and the offset vector b are taken from one of the sets S0, S1, S2. The index idx = idx (W, H) is defined as follows:

[0239]

[0240] Here, each coefficient of matrix A is represented with 8 bits of precision. Set S0 consists of 16 matrices i∈{0,...,15}, each matrix has 16 rows and 4 columns, and 16 offset vectors i∈{0,...,16}, each offset vector has a size of 16. The set of matrices and offset vectors is used for blocks of size 4×4. Set S1 consists of 8 matrices i∈{0,...,7}, each matrix has 8 rows and 8 columns, and 8 offset vectors i∈{0,...,7}, each offset vector has a size of 16. The set S2 consists of 6 matrices i∈{0,...,5}, each matrix has 64 rows and 8 columns, and 6 offset vectors i∈{0,...,5}, and the size of each offset vector is 64.

[0241] 2.12.3 Interpolation

[0242] The prediction signals at the remaining positions are generated from the prediction signals on the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction. The interpolation is performed first in the horizontal direction and then in the vertical direction, regardless of the shape of the block or the size of the block.

[0243] 2.12.4 MIP mode signaling and coordination with other codec tools

[0244] For each codec unit (CU) in intra mode, a flag is sent to indicate whether the MIP mode is to be applied. If the MIP mode is to be applied, the MIP mode (predModeIntra) is signaled. For the MIP mode, the transposed flag (isTransposed) is used to determine whether the mode is transposed, and the MIP mode identifier (modeId) is used to determine the matrix used for a given MIP mode, which is derived as follows:

[0245] isTransposed=predModeIntra&1

[0246] modeId=predModeIntra>>1 (2-14)

[0247] The MIP codec mode is coordinated with other codecs by taking into account the following aspects:

[0248] – Enable LFNST on MIPs on large blocks. Here the LFNST transform in planar mode is used

[0249] – The reference sample export of MIP is exactly the same as that of the traditional intra prediction mode

[0250] – For the upsampling step used in MIP prediction, the original reference samples are used instead of the downsampled samples

[0251] – Perform clipping before upsampling instead of after upsampling

[0252] – Regardless of the maximum transform size, the number of MIP modes allowed to reach 64×64 is 32 for sizeId=0, 16 for sizeId=1, and 12 for sizeId=2.

[0253] 2.13. Decoder-side intra-frame mode export

[0254] In JEM-2.0, intra modes are extended from 35 modes in HEVC to 67, and they are derived at the encoder and explicitly signaled to the decoder. A lot of overhead is spent on intra mode encoding in JEM-2.0. For example, in all intra coding configurations, intra mode signaling overhead can be as high as 5% to 10% of the total bit rate. This contribution proposes a decoder-side intra mode derivation method to reduce intra mode encoding and decoding overhead while maintaining prediction accuracy.

[0255] To reduce the overhead of intra-mode signaling, this contribution presents a decoder-side intra-mode derivation (DIMD) approach. In the proposed approach, instead of explicitly signaling the intra-mode, the information is derived from the neighboring reconstructed samples of the current block at both the encoder and the decoder. The intra-mode derived by DIMD is used in two ways:

[0256] 1) For 2N CUs, when the corresponding CU-level DIMD flag is turned on, the DIMD mode is used as the intra mode for intra prediction;

[0257] 2) For NN CU, DIMD mode is used to replace a candidate in the existing MPM list to improve the efficiency of intra-mode coding.

[0258] 2.13.1 Template-based intra-mode derivation

[0259] Fig.18 Target samples, template samples, and reference samples of the template used in DIMD are shown. Fig.18 As shown, the target represents the current block (block size is N) whose intra prediction mode is to be estimated. The template (composed of Fig.18 The pattern area in the image specifies a set of reconstructed samples that are used to derive the intra mode. The template size is expressed as the number of samples in the template that extend above and to the left of the target block, i.e., L. In the current implementation, the template size for 4×4 and 8×8 blocks is 2 (i.e., L=2), and the template size for 16×16 and larger blocks is 4 (i.e., L=4). According to the definition of JEM-2.0, the template reference (represented by Fig.18 The region representation in ( ) refers to a set of neighboring samples above and to the left of the template. Unlike template samples that are always from the reconstructed region, the reference samples of the template may not have been reconstructed when encoding / decoding the target block. In this case, the existing reference sample replacement algorithm of JEM-2.0 is utilized to replace the unavailable reference samples with available reference samples.

[0260] For each intra prediction mode, DIMD calculates the absolute difference (SAD) between the reconstructed template samples and its predicted samples obtained from the reference samples of the template. The intra prediction mode that produces the smallest SAD is selected as the final intra prediction model for the target block.

[0261] ·2.13.2 DIMD in 2N×2NCU frame

[0262] For intra 2N×2N CUs, DIMD is used as an additional intra mode, which is adaptively selected by comparing the DIMD intra mode with the best normal intra mode (i.e., explicitly signaled). Each intra 2N×2N CU has a flag to indicate the use of DIMD. If the flag is 1, the intra mode derived by DIMD is used to predict the CU; otherwise, DIMD is not applied and the intra mode explicitly signaled in the bitstream is used to predict the CU. When DIMD is enabled, chroma components always reuse the same intra mode derived for luma components, i.e., DM mode.

[0263] In addition, for each DIMD-encoded CU, blocks in the CU can be adaptively selected to derive their intra modes at the PU level or the TU level. Specifically, when the DIMD flag is 1, another CU-level DIMD control flag signals the level at which DIMD is performed. If the flag is 0, it means that DIMD is performed at the PU level, and all TUs in the PU use the same derived intra mode for intra prediction; otherwise (i.e., the DIMD control flag is 1), it means that DIMD is performed at the TU level, and each TU in the PU derives its own intra mode.

[0264] In addition, when DIMD is enabled, the number of angular directions increases to 129, and DC mode and planar mode remain unchanged. To accommodate the increased granularity of angular intra modes, the accuracy of frame interpolation value filtering for DIMD-coded CUs is increased from 1 / 32 pixel to 1 / 64 pixel. In addition, in order to use the derived intra mode of a DIMD-coded CU as an MPM candidate mode for neighboring intra blocks, the 129 directions of the DIMD-coded CU are converted to "normal" intra modes (i.e., 65 angular intra directions) before being used as MPMs.

[0265] ·2.13.3 DIMD of N×NCU within a frame

[0266] In the proposed method, the intra modes of an N×N intra CU are always signaled. However, to improve the efficiency of intra mode encoding and decoding, the intra modes derived from DIMD are used as MPM candidates for predicting the intra modes of the four PUs in the CU. In order not to increase the overhead of MPM index signaling, the DIMD candidate is always placed first in the MPM list and the last existing MPM candidate mode is deleted. In addition, a pruning operation is performed so that if a DIMD candidate is redundant, it will not be added to the MPM list.

[0267] ·2.13.4DIMD intra-frame pattern search algorithm

[0268] To reduce the encoding / decoding complexity, a straightforward fast intra mode search algorithm is used for DIMD. First, an initial estimation process is performed to provide a good starting point for the intra mode search. Specifically, an initial candidate list is created by selecting N fixed modes from the allowed intra modes. Then, the SAD is calculated for all candidate intra modes and the one that minimizes the SAD is selected as the starting intra mode. To achieve a good complexity / performance trade-off, the initial candidate list consists of 11 intra modes, including DC, planar, and every 4 modes of the 33 angular intra directions defined by HEVC, i.e., intra modes 0, 1, 2, 6, 10…30, 34.

[0269] If the starting intra mode is DC or planar, it is used as the DIMD mode. Otherwise, based on the starting intra mode, a refinement process is then applied where the best intra mode is determined with a single iterative search. It works by comparing the SAD values ​​of three intra modes separated by a given search interval in each iteration and keeping the intra mode that minimizes the SAD. Then, the search interval is reduced to half and the intra mode selected from the previous iteration is used as the center intra mode for the current iteration. For the current DIMD implementation with 129 angular intra directions, a maximum of 4 iterations are used in the refinement process to find the best DIMD intra mode.

[0270] 2.14. Decoder-side intra-mode derivation

[0271] In this contribution, a method is proposed to avoid sending the luma intra prediction mode in the bitstream. This is done by deriving the luma intra mode using previously encoded / decoded pixels in the same way at the encoder and decoder. This process defines a new codec mode called DIMD, whose selection is to use a simple flag to signal in the bitstream for intra-coded blocks. DIMD competes with other coding modes at the encoder, including the classic intra coding mode (wherein the intra prediction mode is encoded). Note that in this contribution, DIMD is only applied to luma. For chroma, the classic intra coding mode is applied. As done for other codec modes (classic intra, inter, merge, etc.), the rate distortion cost is calculated for the DIMD mode, and then it is compared with the codec cost of other modes to decide whether to select it as the final codec mode for the current block.

[0272] On the decoder side, the DIMD flag is parsed first. If true, the intra prediction mode is derived using the same previously coded neighboring pixels during reconstruction. If not, the intra prediction mode is parsed from the bitstream as in classic intra coding mode.

[0273] 2.14.1 Intra-frame prediction mode export

[0274] 2.14.1.1 Gradient analysis

[0275] To derive the intra prediction mode for a block, first a set of neighboring pixels is selected on which the gradient analysis is performed. For normative purposes, these pixels should be in the pool of decoded / reconstructed pixels. Fig.19 A set of selected pixels on which the gradient analysis is performed is shown. Fig.19 As shown, the template around the current block is selected by T pixels to the left and T pixels above. In the proposal, T=2 is set.

[0276] Next, a gradient analysis is performed on the pixels of the template. This allows the determination of the principal direction for the template, which is assumed (and is the core premise of our method) to have a very high probability of being the same as the one of the current block. Therefore, a simple 3×3 Sobel gradient filter is used, which is defined by the following matrix that will be convolved with the template:

[0277] as well as For each pixel of the template, each of these two matrices with a 3×3 window centered at the current pixel is multiplied point-wise and composed of its 8 direct neighbors, and the results are summed. Thus, two values ​​G are obtained in the horizontal and vertical directions, respectively x (From M x The multiplication of) and G corresponding to the gradient at the current pixel y (From M y multiplication of ). Fig. 20 The convolution process is shown. Fig. 20 The convolution of a 3x3 Sobel gradient filter with a template is shown. The blue pixel is the current pixel. The red pixels (including blue) are pixels for which gradient analysis is possible. The gray pixels are pixels for which gradient analysis is not possible due to the lack of some neighbors. The purple pixels are available (reconstructed) pixels outside the considered template that are used in the gradient analysis of the red pixels. In the case where a purple pixel is not available (e.g. because the block is too close to the border of the picture), the gradient analysis of all the red pixels using that purple pixel is not performed.

[0278] 2.14.1.2 Gradient histogram and mode export

[0279] For each red pixel, use G x and G y The strength (G) and orientation (O) of the gradient are calculated as:

[0280] G=|G x |+|G y | and

[0281] Note that a fast implementation of the atan function is proposed. The orientation of the gradient is then converted into an intra-frame angular prediction mode, which is used to index a histogram (initialized to zero first). Once all red pixels in the template have been processed, the histogram will contain the cumulative values ​​of the gradient strengths (for each intra-frame angular mode). The mode showing the highest peak in the histogram is selected as the intra-frame prediction mode for the current block. If the maximum value in the histogram is 0 (meaning that gradient analysis cannot be performed, or the area constituting the template is flat), the DC mode is selected as the intra-frame prediction mode for the current block.

[0282] For the block at the top of the CTU, the gradient analysis of the pixels in the top part of the template is not performed. The DIMD flag is encoded and decoded using three possible contexts, depending on the neighboring blocks on the left and above, similar to the Skip flag encoding and decoding. Context 0 corresponds to the case where neither the neighboring blocks on the left nor the above are encoded and decoded with DIMD mode, context 1 corresponds to the case where only one neighboring block is encoded and decoded with DIMD, and context 2 corresponds to the case where both neighboring blocks are DIMM encoded and decoded. The initial symbol probability for each context is set to 0.5.

[0283] 2.14.2. Prediction with 130 Intra Modes

[0284] One advantage of DIMD over the classic intra mode codec is that the derived intra modes can have higher precision, allowing more accurate prediction at no additional cost since it is not transmitted in the bitstream. The derived intra modes cover 129 angular modes, so there are 130 modes in total including DC (in our contribution, derived intra modes are never planar). The classic intra codec modes remain unchanged, i.e., prediction and mode codec still use 67 modes.

[0285] To accommodate prediction using 129 modes, wide-angle intra prediction and simplified PDPC are modified as necessary. Note that only the prediction process uses the extended intra modes, which means that for any other purpose (such as deciding whether to filter the reference samples), the modes are converted back to 67-mode precision.

[0286] 2.14.3 Other specification changes

[0287] In DIMD mode, the luma intra mode is derived during the reconstruction process (just before the block is reconstructed). This is done to avoid dependencies on reconstructed pixels during parsing. However, by doing so, the luma intra mode of the block will be undefined for the chroma components of the block as well as the luma components of neighboring blocks. This causes problems because:

[0288] For chroma, a fixed mode candidate list is defined. In general, if the luma mode is equal to one of the chroma candidates, that candidate will be replaced by the vertical diagonal (VDIA_IDX) intra mode. Since in DIMD, luma mode is not available, the initial chroma mode candidate list is not modified.

[0289] In classic intra mode where luma intra prediction modes are to be parsed from the bitstream, the MPM list is constructed using luma intra modes of neighboring blocks, which may be unavailable if these blocks are encoded using DIMD. In this case, in our contribution, during MPM list construction, DIMM-encoded blocks are treated as inter blocks, which means that they are effectively treated as unavailable.

[0290] 2.15.DIMD

[0291] Three angular modes are selected from the Histogram of Gradients (HoG) computed from the neighboring pixels of the current block. Once the three modes are selected, their predictors are computed normally and then their weighted average is used as the final predictor for the block. To determine the weights, the corresponding magnitude in the HoG is used for each of the three modes. DIMD mode is used as an alternative prediction mode and is always checked in FullRD mode.

[0292] The current version of DIMD has some modifications in signaling, HoG calculation, and prediction fusion. The purpose of these modifications is to improve the codec performance and address the complexity issues raised in the last meeting (i.e., throughput of 4x4 blocks). The following sections describe the modifications for each aspect.

[0293] 2.15.1 Signaling

[0294] Fig.21 A proposed intra-block decoding process 2100 is shown. Fig.21 The order of parsing flags / indexes integrated with the proposed DIMD in VTM5 is shown.

[0295] It can be seen that the DIMD flag of the block is first parsed using a single CABAC context, which is initialized to a default value of 154.

[0296] If flag == 0, parsing will continue normally.

[0297] Otherwise (if flag == 1), only the ISP index is parsed, and the following flags / indexes are inferred to be zero: BDPCM flag, MIP flag, MRL index. In this case, the entire IPM parsing is also skipped.

[0298] During the parsing phase, when a regular non-DIMD block queries the IPM of its DIMD neighbor, the pattern PLANAR_IDX is used as a virtual IPM for the DIMD block.

[0299] 2.15.2 Texture analysis

[0300] Fig. 22An example graph 2200 of HoG calculation from a template with a width of 3 pixels is shown. Texture analysis of DIMD includes the calculation of the Histogram of Gradients (HoG) ( Fig. 22 ). The gradient histogram is calculated by applying horizontal and vertical Sobel filtering to the pixels in a template of width 3 around the block. However, if the above template pixels belong to different CTUs, they will not be used for texture analysis.

[0301] Once calculated, the IPMs corresponding to the two highest histogram bins are selected for the block.

[0302] In previous versions, all pixels in the template centerline participated in the HoG computation [1]. However, the current version improves the throughput of this process by applying the Sobel filter more sparsely over 4x4 blocks. For this purpose, only one pixel from the left and one pixel from the top are used. This Fig. 22 Shown in.

[0303] Besides reducing the number of operations for gradient computation, this property also simplifies the selection of the best 2 patterns from the HoG, since the resulting HoG cannot have more than two non-zero amplitudes.

[0304] 2.15.3 Prediction Fusion

[0305] The current version of the method also uses a fusion of three predictors for each block. However, the choice of prediction mode is different and utilizes a combined hypothesis intra prediction method, where planar mode is considered to be used in combination with other modes when calculating intra prediction candidates. In the current version, the two IPMs corresponding to the two highest HoG stripes are combined with planar mode.

[0306] Prediction fusion is applied as a weighted average of the three predictors described above. For this purpose, the weight of the plane is fixed to 21 / 64 (~1 / 3). Then, the remaining weight of 43 / 64 (~2 / 3) is shared between the two HoG IPMs in proportion to the amplitude of their HoG stripes. Fig.23 Visualize the process 2300. Fig.23 Illustration of prediction fusion via weighted averaging of two HoG modes and planes.

[0307] 2.16. Combined Inter and Intra Prediction (CIIP)

[0308] In VVC, when encoding and decoding a CU in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is signaled to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As its name indicates, CIIP prediction combines the inter prediction signal with the intra prediction signal. The inter prediction signal P in CIIP mode inter is derived using the same inter-frame prediction process applied to the normal merge mode; and the intra-frame prediction signal P intra is derived according to the conventional intra prediction process in planar mode. Then, the intra and inter prediction signals are combined using weighted averaging, where the weight values ​​are calculated according to the coding mode of the top and left neighboring blocks as follows (e.g. Fig.24 As shown, Fig.24 shows the top and left neighboring blocks used in the CIIP weight derivation):

[0309] - if the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntratop to 0;

[0310] - If the left neighbor is available and intra-coded, set isIntraLeft to 1, otherwise set isIntralLeft to 0;

[0311] -If (isIntraLeft+isIntraTop) is equal to 2, set wt to 3;

[0312] - Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, set wt to 2;

[0313] - Otherwise, set wt to 1.

[0314] The CIIP forecast is formed as follows:

[0315] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (2-17)

[0316] 2.17. Geometric Partitioning Model (GPM)

[0317] In VVC, geometric partitioning mode is supported for inter prediction. The geometric partitioning mode is signaled using a CU-level flag as a merge mode. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size, the geometric partitioning mode supports a total of 64 partitions, excluding 8x64 and 64x8.

[0318] Fig.25 An example of GPM segmentation grouped by the same angle is shown. When this mode is used, the CU is divided into two parts by geometrically positioned straight lines (such as Fig.25 ). The position of the partition line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows unidirectional prediction, that is, each part has a motion vector and a reference index. Unidirectional prediction motion constraints are applied to ensure that, as with traditional bidirectional prediction, only two motion compensated predictions are required per CU. The unidirectional prediction motion for each partition is derived using the process described in 2.17.1.

[0319] If a geometric partitioning mode is used for the current CU, a geometric partitioning index indicating the partitioning mode (angle and offset) of the geometric partitioning and two merge indices (one for each partition) are further transmitted through a signal. The number of maximum GPM candidate sizes is explicitly transmitted through a signal in the SPS, and the syntax binarization for the GPM merge indices is specified. After predicting each part of the geometric partitioning, the sample values ​​along the edges of the geometric partitioning are adjusted using the fusion process with adaptive weights in 2.17.2. This is the prediction signal for the entire CU, and the transform and quantization process will be applied to the entire CU like other prediction modes. Finally, the motion field of the CU predicted using the geometric partitioning mode is stored as described in 2.17.3.

[0320] ·2.17.1 One-way prediction candidate list construction

[0321] The unidirectional prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process in 3.4.1. Denote n as the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector (where X equals the parity of n) of the nth extended merge candidate is used as the nth unidirectional prediction motion vector of the geometric partitioning mode. These motion vectors are Fig.26 (which shows the unidirectional prediction MV selection for geometric partitioning mode) is marked with "x". In the case where the corresponding LX motion vector of the nth extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector for geometric partitioning mode.

[0322] ·2.17.2 Blending along geometric partition edges

[0323] After predicting each part of the geometric partition using its own motion, fusion is applied to the two prediction signals to derive samples around the edges of the geometric partition. The fusion weights for each position of the CU are derived based on the distance between the individual position and the partition edge.

[0324] The distance from a position (x, y) to the edge of the partition is derived as:

[0325]

[0326] where i, j are the indices for the angle and offset of the geometry partition, which depend on the geometry partition index transmitted with the signal. x，j and ρ y，j The sign of depends on the angle index i.

[0327] The weights for each part of the geometric partition are derived as follows:

[0328] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) (2-22)

[0329]

[0330] w1(x,y)=1-w0(x,y) (2-24)

[0331] partIdx depends on the angle index i. Fig. 27 An example generation of blending weights w0 using geometric partitioning mode is shown. Fig. 27 An example of weight w0 is shown in .

[0332] 2.17.3 Motion field storage for geometric partitioning mode

[0333] Mv1 from the first part of the geometric partitioning, Mv2 from the second part of the geometric partitioning, and the combination Mv of Mv1 and Mv2 are stored in the motion field of the geometric partitioning mode codec CU.

[0334] The type of motion vector stored for each individual position in the motion field is determined as:

[0335] sType=abs(motionIdx)<32?2(motionIdx≤0?(1-partIdx):partIdx)

[0336] (2-25)

[0337] Where motionIdx is equal to d(4x+2, 4y+2), which is recalculated from equation (2-18). partIdx depends on the angle index i.

[0338] If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise, if sType is equal to 2, the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process:

[0339] 1) If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bi-directional prediction motion vector.

[0340] Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored.

[0341] 2.18. Multi-hypothesis prediction

[0342] In Multiple Hypothesis Prediction (MHP), up to two additional predictors are signaled on top of the Inter AMVP mode, Normal Merge mode, Affine Merge and MMVD modes. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0343] p n+1 =(1-a n+1 ) n +α n+1 h n+1

[0344] The weight factor α is specified according to the following table:

[0345] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0346] For inter-frame AMVP mode, only MHP is applied if unequal weights in BCW are selected in bidirectional prediction mode. Additional assumptions can be merge mode or AMVP mode. In the case of merge mode, the motion information is indicated by the merge index and the merge candidate list is the same as in the geometric partitioning mode. In the case of AMVP mode, the reference index, MVP index and MVD are signaled.

[0347] 3. Question

[0348] The current design of intra prediction can only utilize the redundancy of samples in the current block and samples adjacent to the current block. Non-local texture information is not fully utilized.

[0349] 4. Detailed solution

[0350] The following detailed embodiments should be considered as examples to explain the overall concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way.

[0351] In this disclosure, the term decoder-side intra mode derivation (DIMD) or template-based intra prediction mode (TIMD) refers to a codec tool that uses previously decoded blocks to derive intra prediction modes.

[0352] In the present disclosure, a “regular intra prediction mode (IPM) candidate set” is used to indicate the allowed IPMs for an intra-coded block (e.g., 35 modes in HEVC, 67 modes in VVC), and a “regular intra prediction mode” may refer to an IPM in the regular IPM candidate set.

[0353] In this disclosure, the "extended intra prediction mode (IPM) candidate set" includes all regular IPMs and extended IPMs (such as Fig.28 in the example). Fig.28 A conventional angle IPM (indicated by a black arrow) and an extended angle IPM (indicated by a dashed line) are shown.

[0354] In the following discussion, RightShift(x, n) can be defined as

[0355]

[0356] RightShift(x,n) can also be defined as RightShift(x,n)=(x+offset0)>>n.

[0357] Concept of the proposed method

[0358] 1. In one example at the decoder, the first intra-frame prediction sample may be added with an adjustment value to generate a second intra-frame prediction sample, and the second intra-frame prediction sample will be added with a residual value (which may be obtained by inverse transformation / dequantization) to generate a reconstructed sample value.

[0359] a. Accordingly, at the encoder, the first intra-frame prediction sample may be added with the adjustment value to generate the second intra-frame prediction sample.

[0360] The second intra-predicted sample value will be subtracted from the original sample value to obtain a residual value which can be quantized / transformed later.

[0361] 2. In one example, the adjustment value may be a pseudo residual sample value (PR) generated by subtracting a pseudo prediction value (PP) from a sample value in a pseudo block (PS). In a formulation, PR = PS - PP.

[0362] 3. In one example, the pseudo block may be in the same picture / slice / tile of the current block. Fig.29 An example of a current block and a pseudo block is shown. Assuming that the upper left position of the current block is (x0, y0) and the upper left position of the pseudo block is (x1, y1), then (x1, y1) = (x0 + BVx, y0 + BVy), where BV = (BVx, BVy) represents a block vector (BV) measuring the displacement between the current block and the pseudo block.

[0363] a. The pseudo block should have the same width and height as the current block.

[0364] b. In an alternative example, the pseudo block may be in a different picture / slice / tile of the current block. In one example, the pseudo block may be in a reference picture of the current block.

[0365] i. In one example, BV is replaced by a motion vector (MV).

[0366] 4. In one example, a pseudo block adjacent to / non-adjacent to a pseudo block and / or its pseudo reference samples may include reconstructed samples.

[0367] a. For example, the samples can be before or after filtering.

[0368] b. For example, samples can be before or after value mapping, such as luma mapping with chroma scaling (LMCS).

[0369] 5. In one example, pseudo blocks and / or pseudo reference samples that are adjacent / non-adjacent to the pseudo block may be in the region.

[0370] a. In one example, the area may not overlap with the current block.

[0371] b. In one example, the area may be inside the picture / slice / tile containing the current block.

[0372] c. In one example, the region may be in one or more predefined CTUs or CTU rows.

[0373] d. In one example, this area may be the same as the valid area of ​​the prediction block for IBC.

[0374] Use of Pseudo Residuals

[0375] 6. In one example, pseudo residual sample values ​​may be generated by applying intra prediction on a pseudo block, where pseudo reference samples adjacent / non-adjacent to the pseudo block may be used.

[0376] a. In one example, the intra prediction may be any type of intra prediction, such as planar, DC, angular intra prediction, wide-angle intra prediction, PDPC, multiple reference lines (MRL), intra sub-partitioning (ISP), MIP, cross-component linear model (CCLM), intra fusion mode, etc.

[0377] b. In one example, the intra prediction mode (the same as the intra prediction mode applied to the current block) may be applied.

[0378] to generate pseudo residual sample values.

[0379] c. In one example, intra prediction applied to the pseudo block may be performed in the same manner as applied to the current block, except that pseudo reference samples are used instead of reference samples of the current block.

[0380] i. Alternatively, the intra prediction applied to the pseudo block may be done in a different manner than that applied to the current block.

[0381] 1) For example, the intra-frame interpolation filter can be shorter.

[0382] 2) For example, PDPC can be omitted.

[0383] 7. Fig.30 An example of the proposed method at an encoder is shown. Fig.30 A flow chart for using a pseudo residual block at an encoder is shown.

[0384] a. Obtain the "true" residual block (TR) by subtracting the predicted block (P) from the original block (O). In the formulation, TR(x, y) = O(x, y) - P(x, y), where (x, y) is the coordinate of the sample relative to the top left of the current block.

[0385] b. Obtain a pseudo residual block (PR) by subtracting the pseudo prediction block (PP) from the pseudo block (PS). In the formulation, PR(x,y)=PS(x,y)-PP(x,y), where (x,y) is the coordinate of the sample relative to the upper left of the pseudo block.

[0386] c. Obtain the residual block (FR) by subtracting the pseudo residual block (PR) from the "true" residual block (TR). In a formalized manner, FR(x,y) = TR(x,y) - PR(x,y).

[0387] d. The residual block (FR) may be further processed by operations such as transform and / or quantization.

[0388] 8. Fig.31 An example of the proposed method at a decoder is shown. Fig.31 A flow chart for using a pseudo residual block at a decoder is shown.

[0389] a. Obtain a pseudo residual block (PR) by subtracting the pseudo prediction block (PP) from the pseudo block (PS). Formulated in a manner, PR(x,y)=PS(x,y)-PP(x,y), where (x,y) is the coordinate of the sample relative to the upper left of the pseudo block.

[0390] b. The reconstructed residual block (RR) is obtained from ongoing operations such as inverse transform and / or dequantization.

[0391] c. Obtain the "true" reconstructed residual block (TRR) by adding the reconstructed residual block (RR) and the pseudo residual block (PR). In a formulation, TRR(x,y)=RR(x,y)+PR(x,y).

[0392] d. Obtain the reconstructed block (R) by adding the "true" reconstructed residual block (TRR) and the prediction block (P). In a formulation, R(x,y)=TRR(x,y)+P(x,y).

[0393] e. The reconstructed block (R) may be further processed by the following operations: filtering and / or mapping and / or outputting and / or storing, etc.

[0394] 9. The proposed method can be implemented in some different variants, such as with a different calculation order or clipping order.

[0395] a. In one example at the encoder, FR(x,y)=TR(x,y)-PR(x,y)=O(x,y)-(PR(x,y)+P(x,y)).

[0396] b. In one example at the decoder, R(x,y)=TRR(x,y)+P(x,y)=RR(x,y)+(PR(x,y)+P(x,y)).

[0397] c. In one example, (PR(x,y)+P(x,y)) can be clipped before using in the above equation.

[0398] d. In one example, (PR(x,y)+P(x,y)) can be clipped to the valid sample range.

[0399] 10. In one example, the pseudo residuals may be weighted.

[0400] a. For example, FR(x,y)=TR(x,y)-PR(x,y)*w, where w is a weight value such as 0.5.

[0401] b. For example, FR(x,y)=O(x,y)-(PR(x,y)*w+P(x,y)), where w is a weight value such as 0.5.

[0402] c. For example, TRR(x,y)=RR(x,y)+PR(x,y)*w, where w is a weight value such as 0.5.

[0403] d. For example, R(x,y)=RR(x,y)+(PR(x,y)*w+P(x,y)), where w is a weight value such as 0.5.

[0404] e. For example, FR(x,y)=TR(x,y)-RightShift(PR(x,y), s), where s is an integer such as 1.

[0405] f. For example, FR(x,y)=O(x,y)-(Right Shift(PR(x,y),s)+P(x,y)), where s is an integer such as 1.

[0406] g. For example, TRR(x,y)=RR(x,y)+RightShift(PR(x,y), s), where s is an integer such as 1.

[0407] h. For example, R(x,y)=RR(x,y)+((Right Shift(PR(x,y),s)+P(x,y)), where s is an integer such as 1.

[0408] Signaling

[0409] 11. In one example, the location of the dummy blocks may be signaled from the encoder to the decoder.

[0410] a. In one example, BV may be signaled.

[0411] i. In one example, a BV prediction may be signaled to select from the candidates, and a BV difference (BVD) is signaled.

[0412] b. In one example, the BV may be signaled to select from the candidate(s) in merge mode.

[0413] c. In one example, BV may be signaled in the same manner as BV signaling in IBC mode.

[0414] i. In one example, different arithmetic coding context models may be used for IBC mode and pseudo blocks.

[0415] 12. In one example, the location of the pseudo-blocks can be derived implicitly at the decoder.

[0416] a. In one example, template matching can be used to derive the location of the pseudo-blocks.

[0417] 13. In one example, intra prediction of a block applying the proposed method claimed in point 1 may be signaled.

[0418] a. In one example, intra prediction may be signaled in the same way as an intra-coded block to which the proposed method claimed in point 1 is not applied.

[0419] b. In one example, intra prediction may be encoded and decoded using at least one most probable mode (MPM).

[0420] c. In one example, intra prediction may be derived, such as using DIMD or TIMD.

[0421] d. In one example, for a block to which the proposed method claimed in point 1 is applied, a specific intra prediction method may not be allowed.

[0422] i. In one example, if a specific intra prediction method is not allowed, signaling the specific intra prediction method may be skipped.

[0423] ii. The specific intra prediction method can be any type of intra prediction, such as planar, DC, angular intra prediction, wide-angle intra prediction, PDPC, multiple reference lines (MRL), intra sub-partitioning (ISP), MIP, cross-component linear model (CCLM), intra fusion mode, etc.

[0424] 14. In one example, whether to apply the proposed method claimed in point 1 can be signaled in a syntax element (SE).

[0425] a. In one example, SE can be signaled in SPS / PPS / sequence header / picture header / slice header / CTU / CU / TU / PU / etc.

[0426] b. In one example, the SE is conditionally signaled.

[0427] i. For example, the SE for a block is signaled only if the block is intra-coded.

[0428] ii. For example, the SE for a block is signaled only if the block is IBC coded.

[0429] c. In one example, SE can be encoded and decoded with at least one arithmetic context model.

[0430] i. The arithmetic context model may depend on neighboring blocks.

[0431] d. In one example, SE can be bypassed codec.

[0432] e. In one example, SE can be encoded and decoded in a hierarchical manner.

[0433] i. If SEs signaled to higher layers (eg, in SPS) are to be disabled, SEs at lower levels (eg, in CUs) may not be signaled.

[0434] f. In one example, if it is signaled that the proposed method claimed in point 1 is applied, at least one BV and / or at least one intra prediction method is signaled.

[0435] g. In one example, if it is signaled that the proposed method claimed in point 1 is applied, the signaling of at least one BV and / or the signaling of at least one intra prediction method is modified accordingly.

[0436] General claims

[0437] 15. It may be signaled in the bitstream whether and / or how the methods disclosed above are applied.

[0438] a. In one example, they can be signaled at sequence level / picture group level / picture level / slice level / tile group level, such as in sequence header / picture header / SPS / VPS / DPS / DCI / PPS / PPS / slice header / tile group header.

[0439] b. In one example, PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of regions may be signaled at those containing more than one sample or pixel.

[0440] 16. Whether and / or how to apply the method disclosed above may depend on the coded information, such as block size, color format, single-tree partitioning / dual-tree partitioning, color component, slice / picture type.

[0441] Embodiments of the present disclosure relate to prediction enhancement. As used herein, the term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a codec unit (CU), a prediction unit (PU), a transform unit (TU), a prediction block (PB), a transform block (TB), or a video processing unit including multiple samples or pixels. A block may be rectangular or non-rectangular.

[0442] Fig.32A flow chart of a method 3200 for video processing according to an embodiment of the present disclosure is shown. The method 3200 is implemented for conversion between a current video block of a video and a bit stream of the video.

[0443] At block 3210, a first intra-frame prediction sample of the current video block is determined. At block 3220, an adjustment value associated with another video block in the video is determined. The other video block is encoded and decoded before the current video block. At block 3230, a conversion is performed based on the first intra-frame prediction sample and the adjustment value.

[0444] The method 3200 enables adjusting the prediction of the current video block based on the adjustment value associated with the previously coded block. Thus, coding efficiency and coding effectiveness can be improved.

[0445] In some embodiments, converting includes decoding the current video block from a bitstream.

[0446] In some embodiments, performing the conversion includes: determining an updated residual value by applying at least one of an inverse transform or dequantization to the residual value in the bitstream; determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; determining a reconstructed sample of the current video block based on the second intra-frame prediction sample and the updated residual value; and decoding the current video block based on the reconstructed sample.

[0447] In some embodiments, the adjustment value is determined based on at least one of the following: PR(x,y)=PS(x,y)-PP(x,y), and R(x,y)=TRR(x,y)+P(x,y)=RR(x,y)+(PR(x,y)+P(x,y)). (x,y) represents the coordinates of the sample relative to the upper left sample of another video block, PR represents the residual block corresponding to the other video block, PR(x,y) represents the adjustment value corresponding to the sample (x,y), PS(x,y) represents the sample of the other video block, PP(x,y) represents the sample of the prediction block of the other video block, R(x,y) represents the sample of the reconstructed video block of the current video block, TRR(x,y) represents the sample of the first reconstructed residual block, P(x,y) represents the sample of the prediction block of the current video block, and RR(x,y) represents the sample of the second reconstructed residual block.

[0448] In some embodiments, the second reconstructed residual block is from an ongoing operation. In some embodiments, the ongoing operation includes at least one of: inverse transformation and dequantization.

[0449] In some embodiments, the first sample of the first reconstructed residual block is determined based on at least one of: a sum of a second sample of the second reconstructed residual block and an adjustment value, a weighted sum of the second sample and the adjustment value, and a sum of the second sample and a shifted adjustment value. For example, the shifted adjustment value can be represented as RightShift(PR(x, y), s).

[0450] In some embodiments, the sample of the reconstructed video block is determined based on at least one of: a sum of a third sample of the first reconstructed residual block and a fourth sample of the prediction block of the current video block, a weighted sum of a second sample of the second reconstructed residual block, the adjustment value, and the fourth sample of the prediction block of the current video block, and a sum of the second sample, the shifted adjustment, and the fourth sample.

[0451] In some embodiments, the method 3200 further comprises: applying a further operation to the reconstructed video block, the further operation comprising at least one of: a filtering operation, a mapping operation, an output operation, and a storage operation.

[0452] In some embodiments, the method 3200 further includes: updating the following item by a cropping operation: the sum of samples of a residual block corresponding to another video block and samples of a prediction block of the current video block.

[0453] In some embodiments, the sum is clipped to a predefined sample range.

[0454] In some embodiments, converting includes encoding the current video block into a bitstream.

[0455] In some embodiments, performing the conversion includes: determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; determining a residual value based on the sample in the bitstream and the second intra-frame prediction sample; determining an updated residual value by applying at least one of quantization or transformation to the residual value; and including the updated residual value in the bitstream.

[0456] In some embodiments, the adjustment value is determined based on at least one of the following: PR(x,y)=PS(x,y)-PP(x,y), and FR(x,y)=TR(x,y)-PR(x,y)=O(x,y)-(PR(x,y)+P(x,y)). (x,y) represents the coordinates of the sample relative to the upper left sample of another video block, PR represents the residual block corresponding to the other video block, PR(x,y) represents the adjustment value corresponding to the sample (x,y), PS(x,y) represents the sample of the other video block, PP(x,y) represents the sample of the prediction block of the other video block, FR(x,y) represents the sample of the first residual block of the current video block, TR(x,y) represents the sample of the second residual block of the current video block, P(x,y) represents the sample of the prediction block of the current video block, and O(x,y) represents the sample of the original video block of the other video block.

[0457] In some embodiments, the first operation is applied to a first residual block of the current video block. In some embodiments, the first operation includes at least one of: transform and quantize.

[0458] In some embodiments, the fifth sample of the first residual block is determined based on at least one of: a difference between a sixth sample of the second residual block of the current video block and the adjustment value, a weighted difference between the sixth sample and the adjustment, and a difference between the sixth sample and the shifted adjustment.

[0459] In some embodiments, the sample of the second residual block is determined based on at least one of: a difference between a seventh sample of the original video block and an eighth sample of the second residual block, a difference between the seventh sample and a weighted sum of the adjustment value and the eighth sample, and a difference between the seventh sample and a sum of the shifted adjustment value and eight samples.

[0460] In some embodiments, the method 3200 further includes: updating the following item by a cropping operation: the sum of samples of a residual block corresponding to another video block and samples of a prediction block of the current video block.

[0461] In some embodiments, the sum is clipped to a predefined sample range.

[0462] In some embodiments, determining the adjustment value based on the other video block comprises: determining a prediction of the other video block; determining sample values ​​of the other video block; and determining the adjustment value based on the prediction and the sample values ​​of the other video block.

[0463] In some embodiments, the another video block comprises a dummy block.

[0464] In some embodiments, the prediction of the other video block comprises intra-prediction of the other video block.

[0465] In some embodiments, intra prediction of another video block is based on reference samples of another video block, which may or may not be adjacent to the other video block.

[0466] In some embodiments, intra prediction of another video block is in at least one of the following modes: planar intra prediction mode, DC intra prediction mode, angular intra prediction mode, wide-angle intra prediction mode, position-dependent intra prediction combination (PDPC) mode, multiple reference line (MRL) mode, intra sub-partitioning (ISP) mode, matrix weighted intra prediction (MIP) mode, cross-component linear model (CCLM) mode, and intra fusion mode.

[0467] In some embodiments, determining the intra prediction for the other video block includes applying an intra prediction mode to the other video block, the intra prediction mode being applied to the current video block.

[0468] In some embodiments, intra-frame prediction of another video block is determined based on a first intra-frame prediction codec tool and a first reference sample of the other video block, and intra-frame prediction of the current video block is determined based on a second intra-frame prediction codec tool and a second reference sample of the current video block, and the second reference sample is different from the first reference sample.

[0469] In some embodiments, the first intra prediction tool and the second intra prediction tool are the same or different.

[0470] In some embodiments, the first filter of the first intra prediction codec is shorter than the second filter of the second intra prediction codec.

[0471] In some embodiments, the first intra prediction codec does not have a position dependent intra prediction combination (PDPC) mode.

[0472] In some embodiments, the other video block and the current video block are in the same video region. In some embodiments, the other video block and the current video block are in different video regions. For example, a video region may include at least one of the following: a picture, a strip, and a tile.

[0473] In some embodiments, the other video block is in a reference picture of the current video block.

[0474] In some embodiments, the other video block and the current video block have the same width and the same height.

[0475] In some embodiments, the method 3200 further includes determining another video block based on a block vector (BV) associated with the current video block.

[0476] In some embodiments, the reconstructed samples associated with the other video block are in at least one of: the other video block, and a reference sample of the other video block, the reference sample being adjacent or non-adjacent to the other video block.

[0477] In some embodiments, the process is previously applied to a reference sample of another video block. In some embodiments, the process comprises at least one of: a filtering process or a value mapping process. In some embodiments, the value mapping process comprises a luma mapping with a chroma scaling process.

[0478] In some embodiments, the method 3200 further comprises: determining a reference sample of another video block; and applying a process to the reference sample of another video block. In some embodiments, the process comprises at least one of: a filtering process and a value mapping process. In some embodiments, the value mapping process comprises a luminance mapping with a chroma scaling process.

[0479] In some embodiments, the other video block or at least one of the reference samples of the other video block is in the video region, the reference samples being adjacent or non-adjacent to the other video block.

[0480] In some embodiments, the video region does not overlap with the current video block.

[0481] In some embodiments, the video region is inside at least one of: a picture, a slice, and a tile that contains the current video block.

[0482] In some embodiments, the video region is in at least one predefined coding tree unit (CTU) or at least one CTU row.

[0483] In some embodiments, the video region is a valid region for a prediction block for intra block copy (IBC).

[0484] In some embodiments, the location information of the other video block is included in the bitstream.

[0485] In some embodiments, the position information comprises at least one of: a block vector (BV) associated with the current video block, a prediction of the BV, and a BV difference (BVD) associated with the current video block.

[0486] In some embodiments, the method 3200 further includes determining a BV from a plurality of candidate BVs for the current video block, the plurality of candidate BVs being in a merge mode.

[0487] In some embodiments, the codec tool for including a BV in a bitstream is used to include another BV for an intra block copy (IBC) mode in the bitstream.

[0488] In some embodiments, a first arithmetic coding context model is used for another video block and a second arithmetic coding context model is used for the IBC mode, the first arithmetic coding context model being different from the second arithmetic coding context model.

[0489] In some embodiments, method 3200 further includes: determining position information of another video block.

[0490] In some embodiments, determining the position information of the other video block includes determining the position information based on template matching.

[0491] In some embodiments, intra prediction of the current video block is included in the bitstream.

[0492] In some embodiments, a codec tool for including an intra prediction of a current video block in a bitstream is used to include another intra prediction of a video block that was coded without using the adjustment value.

[0493] In some embodiments, intra prediction of the current video block is based on at least one of: a Most Probable Mode (MPM), a decoder-side Intra Mode Derivation (DIMD), and a Template-based Intra Prediction Mode (TIMD).

[0494] In some embodiments, the first intra prediction mode is disabled for the current video block.In some embodiments, an indication of the first intra prediction mode is included in the bitstream.

[0495] In some embodiments, the first intra-frame prediction mode includes at least one of the following: a planar intra-frame prediction mode, a DC intra-frame prediction mode, an angular intra-frame prediction mode, a wide-angle intra-frame prediction mode, a position-dependent intra-frame prediction combination (PDPC) mode, a multiple reference line (MRL) mode, an intra-frame sub-partitioning (ISP) mode, a matrix-weighted intra-frame prediction (MIP) mode, a cross-component linear model (CCLM) mode, and an intra-frame fusion mode.

[0496] In some embodiments, information about applying the method is included in the bitstream. In some embodiments, the information includes at least one of the following: whether to apply the method, and how to apply the method. In some embodiments, the information is in a syntax element (SE).

[0497] In some embodiments, the SE is in at least one of the following: a sequence parameter set (SPS), a picture parameter set (PPS), a sequence header, a picture header, a slice header, a codec tree unit (CTU), a codec unit (CU), a transform unit (TU), and a prediction unit (PU).

[0498] In some embodiments, the SE is encoded and decoded using at least one arithmetic context model.

[0499] In some embodiments, the at least one arithmetic context model is based on at least one neighboring block of the current video block.

[0500] In some embodiments, the SE is bypassed by the codec.

[0501] In some embodiments, SE is hierarchically encoded and decoded.

[0502] In some embodiments, if a first SE of a first level is to be disabled, a second SE of a second level lower than the first level is not present in the bitstream.

[0503] In some embodiments, the first level comprises a sequence parameter set (SPS) level and the second level comprises a codec unit (CU) level.

[0504] In some embodiments, the information is included in the bitstream based on at least one condition being satisfied.

[0505] In some embodiments, the at least one condition includes at least one of: a first condition that the current video block is intra-coded, and a second condition that the current video block is intra-block copy (IBC) coded.

[0506] In some embodiments, if the information indicates that the method applies to the current video block, at least one of a block vector (BV) associated with the current video block and an indication of an intra-prediction mode is included in the bitstream.

[0507] In some embodiments, if the information indicates that the method is applied to the current video block, at least one of the following is modified: a first codec tool for including a block vector (BV) associated with the current video block in the bitstream, and a second codec tool for including an indication of an intra-frame prediction mode in the bitstream.

[0508] In some embodiments, the information is at one of the following levels: sequence level, group of pictures level, picture level, slice level, and tile group level.

[0509] In some embodiments, the information is in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a decoding parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, and a tile group header.

[0510] In some embodiments, the information is in a region comprising multiple samples or multiple pixels.

[0511] In some embodiments, the region includes one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, and a sub-picture.

[0512] In some embodiments, method 3200 further comprises: determining the information based on the encoded and decoded information.

[0513] In some embodiments, the encoded information includes at least one of a block size, a color format, a single tree partition or a dual tree partition, a color component, a slice type, and a picture type.

[0514] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. In the method, a first intra-frame prediction sample of a current video block of the video is determined. An adjustment value associated with another video block in the video is determined. The other video block is encoded and decoded before the current video block. A bitstream is generated based on the first intra-frame prediction sample and the adjustment value.

[0515] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. In the method, a first intra-frame prediction sample of a current video block of the video is determined. An adjustment value associated with another video block in the video is determined. The other video block is encoded and decoded before the current video block. A bitstream is generated based on the first intra-frame prediction sample and the adjustment value. The bitstream is stored in a non-transitory computer-readable recording medium.

[0516] Embodiments of the present disclosure may be described according to the following clauses, features of which may be combined in any reasonable way.

[0517] Clause 1. A method for video processing, comprising: determining a first intra-frame prediction sample of a current video block of a video and a bitstream of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; and performing the conversion based on the first intra-frame prediction sample and the adjustment value.

[0518] Clause 2. The method of clause 1, wherein the converting comprises decoding the current video block from the bitstream.

[0519] Clause 3. A method according to Clause 2, wherein performing the conversion includes: determining an updated residual value by applying at least one of an inverse transform and a dequantization to the residual value in the bitstream; determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; determining a reconstructed sample of the current video block based on the second intra-frame prediction sample and the updated residual value; and decoding the current video block based on the reconstructed sample.

[0520] Clause 4. A method according to clause 2 or clause 3, wherein the adjustment value is determined based on at least one of the following: PR(x, y) = PS(x, y) - PP(x, y), and R(x, y) = TRR(x, y) + P(x, y) = RR(x, y) + (PR(x, y) + P(x, y)), wherein (x, y) represents the coordinates of the sample relative to the upper left sample of the other video block, PR represents the residual block corresponding to the other video block, PR(x, y) represents the adjustment value corresponding to the sample (x, y), PS(x, y) represents the sample of the other video block, PP(x, y) represents the sample of the prediction block of the other video block, R(x, y) represents the sample of the reconstructed video block of the current video block, TRR(x, y) represents the sample of the first reconstructed residual block, P(x, y) represents the sample of the prediction block of the current video block, and RR(x, y) represents the sample of the second reconstructed residual block.

[0521] Clause 5. The method of clause 4, wherein the second reconstructed residual block is from an ongoing operation.

[0522] Clause 6. The method of clause 5, wherein the ongoing operation comprises at least one of: inverse transform and dequantization.

[0523] Clause 7. A method according to any one of clauses 4 to 6, wherein the first sample of the first reconstructed residual block is determined based on at least one of: the sum of the second sample of the second reconstructed residual block and an adjustment value, the weighted sum of the second sample and the adjustment value, and the sum of the second sample and a shifted adjustment value.

[0524] Clause 8. A method according to any one of clauses 4 to 7, wherein the samples of the reconstructed video block are determined based on at least one of: the sum of a third sample of the first reconstructed residual block and a fourth sample of the prediction block of the current video block; a weighted sum of a second sample of the second reconstructed residual block, the adjustment value and a fourth sample of the prediction block of the current video block, and a sum of the second sample, the shifted adjustment and the fourth sample.

[0525] Clause 9. The method of any one of clauses 4 to 8, further comprising: applying a further operation to the reconstructed video block, the further operation comprising at least one of: a filtering operation, a mapping operation, an output operation, and a storage operation.

[0526] Clause 10. The method according to any one of clauses 4 to 9 further comprises: updating, by a cropping operation, the following item: the sum of the samples of the residual block corresponding to the other video block and the samples of the prediction block of the current video block.

[0527] Clause 11. The method of clause 10, wherein the sum is clipped to a predefined sample range.

[0528] Clause 12. The method of clause 1, wherein the converting comprises encoding the current video block into the bitstream.

[0529] Clause 13. A method according to clause 12, wherein performing the conversion comprises: determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; determining a residual value based on the sample in the bitstream and the second intra-frame prediction sample; determining an updated residual value by applying at least one of quantization and transformation to the residual value; and including the updated residual value in the bitstream.

[0530] Clause 14. A method according to clause 12 or clause 13, wherein the adjustment value is determined based on at least one of the following: PR(x, y) = PS(x, y) - PP(x, y), and FR(x, y) = TR(x, y) - PR(x, y) = O(x, y) - (PR(x, y) + P(x, y)), where (x, y) represents the coordinates of the sample relative to the upper left sample of the other video block, PR represents the residual block corresponding to the other video block, PR(x, y) represents the adjustment value corresponding to the sample (x, y), PS(x, y) represents the sample of the other video block, PP(x, y) represents the sample of the prediction block of the other video block, FR(x, y) represents the sample of the first residual block of the current video block, TR(x, y) represents the sample of the second residual block of the current video block, P(x, y) represents the sample of the prediction block of the current video block, and O(x, y) represents the sample of the original video block of the other video block.

[0531] Clause 15. The method of clause 14, wherein a first operation is applied to the first residual block of the current video block.

[0532] Clause 16. The method of clause 15, wherein the first operation comprises at least one of: transform and quantize.

[0533] Clause 17. A method according to any one of clauses 14 to 17, wherein the fifth sample of the first residual block is determined based on at least one of: a difference between a sixth sample of the second residual block of the current video block and the adjustment value; a weighted difference between the sixth sample and the adjustment, and a difference between the sixth sample and the shifted adjustment.

[0534] Clause 18. A method according to any one of clauses 14 to 17, wherein the sample of the second residual block is determined based on at least one of: a difference between a seventh sample of the original video block and an eighth sample of the second residual block, a difference between the seventh sample and: a weighted sum of the adjustment value and the eighth sample, and a difference between the seventh sample and: a sum of the shifted adjustment value and the eighth sample.

[0535] Clause 19. The method according to any one of clauses 14 to 18, further comprising: updating the following item by a cropping operation: the sum of the samples of the residual block corresponding to the other video block and the samples of the prediction block of the current video block.

[0536] Clause 20. The method of clause 19, wherein the sum is clipped to a predefined sample range.

[0537] Clause 21. A method according to any one of clauses 1 to 20, wherein determining the adjustment value based on the other video block comprises: determining a prediction of the other video block; determining sample values ​​of the other video block; and determining the adjustment value based on the prediction and the sample values ​​of the other video block.

[0538] Clause 22. The method of clause 21, wherein the another video block comprises a dummy block.

[0539] Clause 23. The method of clause 21 or clause 22, wherein the prediction of the other video block comprises an intra prediction of the other video block.

[0540] Clause 24. The method of clause 23, wherein the intra prediction of the other video block is based on reference samples of the other video block, the reference samples being adjacent or non-adjacent to the other video block.

[0541] Clause 25. A method according to clause 23 or clause 24, wherein the intra-frame prediction of the other video block is in at least one of the following modes: planar intra-frame prediction mode, DC intra-frame prediction mode, angular intra-frame prediction mode, wide-angle intra-frame prediction mode, position-dependent intra-frame prediction combination (PDPC) mode, multiple reference line (MRL) mode, intra-frame sub-partitioning (ISP) mode, matrix-weighted intra-frame prediction (MIP) mode, cross-component linear model (CCLM) mode, and intra-frame fusion mode.

[0542] Clause 26. The method of any of clauses 23 to 25, wherein determining the intra-prediction of the other video block comprises applying an intra-prediction mode to the other video block, the intra-prediction mode being applied to the current video block.

[0543] Clause 27. A method according to any one of clauses 23 to 26, wherein the intra-frame prediction of the other video block is determined based on a first intra-frame prediction codec tool and a first reference sample of the other video block, and the intra-frame prediction of the current video block is determined based on a second intra-frame prediction codec tool and a second reference sample of the current video block, and the second reference sample is different from the first reference sample.

[0544] Clause 28. The method of clause 27, wherein the first intra prediction tool and the second intra prediction tool are the same or different.

[0545] Clause 29. The method of clause 27, wherein a first filter of the first intra prediction codec is shorter than a second filter of the second intra prediction codec.

[0546] Clause 30. The method of clause 27, wherein the first intra prediction codec does not have a position dependent intra prediction combining (PDPC) mode.

[0547] Clause 31. The method of any one of clauses 1 to 30, wherein the another video block and the current video block are in a same video region.

[0548] Clause 32. The method of any one of clauses 1 to 30, wherein the another video block and the current video block are in different video regions.

[0549] Clause 33. The method of clause 31 or clause 32, wherein the video region comprises at least one of: a picture, a strip, and a tile.

[0550] Clause 34. The method of clause 32, wherein the another video block is in a reference picture of the current video block.

[0551] Clause 35. The method of any one of clauses 1 to 34, wherein the another video block and the current video block have the same width and the same height.

[0552] Clause 36. The method of any one of clauses 1 to 35, further comprising determining the another video block based on a block vector (BV) associated with the current video block.

[0553] Clause 37. A method according to any one of clauses 1 to 36, wherein the reconstructed samples associated with the other video block are in at least one of: the other video block, and reference samples of the other video block, the reference samples being adjacent or non-adjacent to the other video block.

[0554] Clause 38. The method of clause 37, wherein a process was previously applied to the reference sample of the other video block.

[0555] Clause 39. The method of clause 38, further comprising: determining the reference sample of the other video block; and applying a process to the reference sample of the other video block.

[0556] Clause 40. The method of clause 38 or clause 39, wherein the process comprises at least one of: a filtering process, and a value mapping process.

[0557] Clause 41. The method of clause 40, wherein the value mapping process comprises luma mapping with a chroma scaling process.

[0558] Clause 42. The method of any of clauses 1 to 41, wherein at least one of the other video block and a reference sample of the other video block is in a video region, the reference sample being adjacent or non-adjacent to the other video block.

[0559] Clause 43. The method of clause 42, wherein the video region does not overlap with the current video block.

[0560] Clause 44. The method of clause 42 or clause 43, wherein the video region is inside at least one of: a picture, a slice, and a tile that contains the current video block.

[0561] Clause 45. The method of any one of clauses 42 to 44, wherein the video region is in at least one predefined codec tree unit (CTU) or at least one CTU row.

[0562] Clause 46. A method according to any of clauses 42 to -45, wherein the video region is a valid region of a prediction block for intra block copy (IBC).

[0563] Clause 47. The method of any one of clauses 1 to 46, wherein position information of the another video block is included in the bitstream.

[0564] Clause 48. The method of clause 47, wherein the position information comprises at least one of: a block vector (BV) associated with the current video block, a prediction of the BV, and a BV difference (BVD) associated with the current video block.

[0565] Clause 49. The method of clause 48, further comprising: determining the BV from a plurality of candidate BVs for the current video block, the plurality of candidate BVs being in a merge mode.

[0566] Clause 50. The method of clause 48 or clause 49, wherein the codec tool for including the BV in the bitstream is used to include another BV for intra block copy (IBC) mode in the bitstream.

[0567] Clause 51. The method of clause 50, wherein a first arithmetic coding context model is used for the another video block and a second arithmetic coding context model is used for the IBC mode, the first arithmetic coding context model being different from the second arithmetic coding context model.

[0568] Clause 52. The method of any one of clauses 1 to 46, further comprising: determining position information of the another video block.

[0569] Clause 53. The method of clause 52, wherein determining the position information of the another video block comprises determining the position information based on template matching.

[0570] Clause 54. The method of any one of clauses 1 to 53, wherein intra prediction of the current video block is included in the bitstream.

[0571] Clause 55. The method of clause 54, wherein a codec for including the intra prediction of the current video block in the bitstream is used to include another intra prediction of the video block that is encoded without using the adjustment value.

[0572] Clause 56. A method according to clause 54 or clause 55, wherein the intra-frame prediction of the current video block is based on at least one of the following: most probable mode (MPM), decoder-side intra-frame mode derivation (DIMD), and template-based intra-frame prediction mode (TIMD).

[0573] Clause 57. The method of any one of clauses 54 to 56, wherein a first intra-prediction mode is disabled for the current video block.

[0574] Clause 58. The method of clause 57, wherein the indication of the first intra-prediction mode is included in the bitstream.

[0575] Clause 59. A method according to clause 57 or clause 58, wherein the first intra-frame prediction mode includes at least one of the following: a planar intra-frame prediction mode, a DC intra-frame prediction mode, an angular intra-frame prediction mode, a wide-angle intra-frame prediction mode, a position-dependent intra-frame prediction combination (PDPC) mode, a multiple reference line (MRL) mode, an intra-frame sub-partitioning (ISP) mode, a matrix-weighted intra-frame prediction (MIP) mode, a cross-component linear model (CCLM) mode, and an intra-frame fusion mode.

[0576] Clause 60. A method according to any one of clauses 1 to 59, wherein information about applying the method is included in the bitstream.

[0577] Clause 61. The method according to clause 60, wherein the information includes at least one of the following: whether to apply the method, and how to apply the method.

[0578] Clause 62. A method according to clause 60 or clause 61, wherein the information is in a syntax element (SE).

[0579] Clause 63. A method according to clause 62, wherein the SE is in at least one of the following: a sequence parameter set (SPS), a picture parameter set (PPS), a sequence header, a picture header, a slice header, a codec tree unit (CTU), a codec unit (CU), a transform unit (TU) and a prediction unit (PU).

[0580] Clause 64. The method of clause 62 or clause 63, wherein the SE is encoded and decoded using at least one arithmetic context model.

[0581] Clause 65. The method of clause 64, wherein the at least one arithmetic context model is based on at least one neighboring block of the current video block.

[0582] Clause 66. The method of clause 62 or 63, wherein the SE is bypassed for coding.

[0583] Clause 67. The method of clause 62 or 63, wherein the SE is hierarchically encoded.

[0584] Clause 68. The method of clause 67, wherein if a first SE of a first level is to be disabled, a second SE of a second level, the second level being lower than the first level, is not present in the bitstream.

[0585] Clause 69. The method of clause 68, wherein the first level comprises a sequence parameter set (SPS) level and the second level comprises a codec unit (CU) level.

[0586] Clause 70. The method of any one of clauses 60 to 63, wherein the information is included in the bitstream based on at least one condition being satisfied.

[0587] Clause 71. A method according to clause 70, wherein the at least one condition includes at least one of the following: a first condition, the first condition is that the current video block is intra-frame encoded and decoded, and a second condition, the second condition is that the current video block is intra-frame block copy (IBC) encoded and decoded.

[0588] Clause 72. A method according to any one of clauses 60 to 71, wherein if the information indicates that the method is applied to the current video block, an indication of an intra-frame prediction mode and at least one of a block vector (BV) associated with the current video block are included in the bitstream.

[0589] Clause 73. A method according to any one of clauses 60 to 71, wherein if the information indicates that the method is applied to the current video block, at least one of the following is modified: a first codec tool, the first codec tool is used to include a block vector (BV) associated with the current video block in the bitstream, and a second codec tool, the second codec tool is used to include an indication of an intra-frame prediction mode in the bitstream.

[0590] Clause 74. The method of any of clauses 60 to 73, wherein the information is at one of the following levels: sequence level, group of pictures level, picture level, slice level, and tile group level.

[0591] Clause 75. A method according to any one of clauses 60 to 74, wherein the information is in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a decoding parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header and a tile group header.

[0592] Clause 76. A method according to any one of clauses 60 to 75, wherein the information is in a region comprising a plurality of samples or a plurality of pixels.

[0593] Clause 77. A method according to clause 76, wherein the area includes one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, and a sub-picture.

[0594] Clause 78. The method of any one of clauses 60 to 77, further comprising: determining the information based on encoded and decoded information.

[0595] Clause 79. The method of clause 78, wherein the encoded information comprises at least one of: block size, color format, single tree partitioning or dual tree partitioning, color component, slice type, and picture type.

[0596] Clause 80. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of clauses 1 to 79.

[0597] Clause 81. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 79.

[0598] Item 82. A non-transitory computer-readable recording medium storing a bitstream of a video, the bitstream of the video being generated by a method performed by a video processing device, wherein the method comprises: determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; and generating the bitstream based on the first intra-frame prediction sample and the adjustment value.

[0599] Item 83. A method for storing a bitstream of a video, comprising: determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; generating the bitstream based on the first intra-frame prediction sample and the adjustment value; and storing the bitstream in a non-transitory computer-readable recording medium.

[0600] Example Device

[0601] Fig.33A block diagram of a computing device 3300 in which various embodiments of the present disclosure may be implemented is shown. The computing device 3300 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0602] It should be understood that Fig.33 The computing device 3300 shown in the figure is for illustrative purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0603] like Fig.33 As shown, computing device 3300 includes a general computing device 3300. Computing device 3300 may include at least one or more processors or processing units 3310, memory 3320, storage unit 3330, one or more communication units 3340, one or more input devices 3350, and one or more output devices 3360.

[0604] In some embodiments, the computing device 3300 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet computer, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, and includes the accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 3300 can support any type of interface to the user (such as a "wearable" circuit device, etc.).

[0605] The processing unit 3310 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 3320. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the computing device 3300. The processing unit 3310 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0606] The computing device 3300 typically includes various computer storage media. Such media can be any media accessible by the computing device 3300, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 3320 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 3330 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk or other media that can be used to store information and / or data and can be accessed in the computing device 3300.

[0607] The computing device 3300 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Fig.33 Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0608] The communication unit 3340 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 3300 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 3300 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0609] Input device 3350 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 3360 may be one or more of various output devices, such as a display, speaker, printer, etc. With the aid of communication unit 3340, computing device 3300 may also communicate with one or more external devices (not shown), such as storage devices and display devices, computing device 3300 may also communicate with one or more devices that enable a user to interact with computing device 3300, or any device that enables computing device 3300 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), if necessary. Such communication may be performed via an input / output (I / O) interface (not shown).

[0610] In some embodiments, some or all components of the computing device 3300 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (such as the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a remote server. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructure may provide services through a shared data center, although they appear as a single access point for users. Therefore, the cloud computing architecture may be used to provide the components and functions described herein from a service provider at a remote location. Alternatively, they may be provided by a conventional server, or installed directly or otherwise on a client device.

[0611] In an embodiment of the present disclosure, the computing device 3300 may be used to implement video encoding / decoding. The memory 3320 may include one or more video codec modules 3325 having one or more program instructions. These modules can be accessed and executed by the processing unit 3310 to perform the functions of the various embodiments described herein.

[0612] In an example embodiment of performing video encoding, the input device 3350 may receive video data as input 3370 to be encoded. The video data may be processed by, for example, the video codec module 3325 to generate an encoded bitstream. The encoded bitstream may be provided as output 3380 via the output device 3360.

[0613] In an example embodiment of performing video decoding, the input device 3350 may receive an encoded bitstream as input 3370. The encoded bitstream may be processed by, for example, the video codec module 3325 to generate decoded video data. The decoded video data may be provided as output 3380 via the output device 3360.

[0614] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be understood by those skilled in the art that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining a first intra-frame prediction sample of the current video block; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; as well as The converting is performed based on the first intra prediction sample and the adjustment value. 2 . The method of claim 1 , wherein the converting comprises decoding the current video block from the bitstream.

3. The method of claim 2, wherein performing the conversion comprises: determining updated residual values ​​by applying at least one of an inverse transform and dequantization to residual values ​​in the bitstream; Determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; determining, based on the second intra-prediction samples and the updated residual value, a reconstructed sample of the current video block; as well as Based on the reconstructed samples, the current video block is decoded.

4. The method according to claim 2 or claim 3, wherein the adjustment value is determined based on at least one of the following: PR(x,y)=PS(x,y)-PP(x,y), and R(x,y)=TRR(x,y)+P(x,y)=RR(x,y)+(PR(x,y)+P(x,y)), Wherein (x, y) represents the coordinates of the sample relative to the upper left sample of the other video block, PR represents the residual block corresponding to the other video block, PR(x, y) represents the adjustment value corresponding to the sample (x, y), PS(x, y) represents the sample of the other video block, PP(x, y) represents the sample of the prediction block of the other video block, R(x, y) represents the sample of the reconstructed video block of the current video block, TRR(x, y) represents the sample of the first reconstructed residual block, P(x, y) represents the sample of the prediction block of the current video block, and RR(x, y) represents the sample of the second reconstructed residual block. The method of claim 4 , wherein the second reconstructed residual block is from an ongoing operation. The method of claim 5 , wherein the ongoing operation comprises at least one of: inverse transform and dequantization.

7. The method according to any one of claims 4 to 6, wherein the first sample of the first reconstructed residual block is determined based on at least one of the following: The sum of the second sample of the second reconstructed residual block and the adjustment value, a weighted sum of the second sample and the adjustment value, and The sum of the second sample and the shifted adjustment value.

8. The method according to any one of claims 4 to 7, wherein the samples of the reconstructed video block are determined based on at least one of: a sum of a third sample of the first reconstructed residual block and a fourth sample of the prediction block of the current video block; a weighted sum of the second sample of the second reconstructed residual block, the adjustment value, and a fourth sample of the prediction block of the current video block, and The sum of the second sample, the shift-adjusted and the fourth sample.

9. The method according to any one of claims 4 to 8, further comprising: Applying a further operation to the reconstructed video block, the further operation comprising at least one of: Filtering operation, Mapping operation, Output operations, and Storage operations.

10. The method according to any one of claims 4 to 9, further comprising: Through the cropping operation, the following items are updated: the sum of the samples of the residual block corresponding to the other video block and the samples of the prediction block of the current video block. The method of claim 10 , wherein the sum is clipped to a predefined sample range.

12. The method of claim 1, wherein the converting comprises encoding the current video block into the bitstream.

13. The method of claim 12, wherein performing the conversion comprises: Determining a second intra-frame prediction sample based on the first intra-frame prediction sample and the adjustment value; Determining a residual value based on samples in the bitstream and the second intra-frame prediction sample; determining an updated residual value by applying at least one of quantization and transformation to the residual value; as well as The updated residual value is included in the bitstream.

14. The method according to claim 12 or claim 13, wherein the adjustment value is determined based on at least one of the following: PR(x,y)=PS(x,y)-PP(x,y), and FR(x,y)=TR(x,y)-PR(x,y)=O(x,y)-(PR(x,y)+P(x,y)), Wherein (x, y) represents the coordinates of the sample relative to the upper left sample of the other video block, PR represents the residual block corresponding to the other video block, PR(x, y) represents the adjustment value corresponding to the sample (x, y), PS(x, y) represents the sample of the other video block, PP(x, y) represents the sample of the prediction block of the other video block, FR(x, y) represents the sample of the first residual block of the current video block, TR(x, y) represents the sample of the second residual block of the current video block, P(x, y) represents the sample of the prediction block of the current video block, and O(x, y) represents the sample of the original video block of the other video block.

15. The method of claim 14, wherein a first operation is applied to the first residual block of the current video block. The method of claim 15 , wherein the first operation comprises at least one of: transform and quantize.

17. The method according to any one of claims 14 to 17, wherein the fifth sample of the first residual block is determined based on at least one of the following: a difference between a sixth sample of the second residual block of the current video block and the adjustment value; a weighted difference between the sixth sample and the adjustment, and The difference between the sixth sample and the shifted adjustment.

18. The method according to any one of claims 14 to 17, wherein the samples of the second residual block are determined based on at least one of the following: a difference between a seventh sample of the original video block and an eighth sample of the second residual block, a difference between the seventh sample and: a weighted sum of the adjustment value and the eighth sample, and The difference between the seventh sample and: the sum of the shifted adjustment value and the eighth sample.

19. The method according to any one of claims 14 to 18, further comprising: The following items are updated by the cropping operation: the sum of the samples of the residual block corresponding to the other video block and the samples of the prediction block of the current video block.

20. The method of claim 19, wherein the sum is clipped to a predefined sample range.

21. The method of any one of claims 1 to 20, wherein determining the adjustment value based on the another video block comprises: determining a prediction for the another video block; determining sample values ​​of the another video block; as well as Based on the prediction and the sample values ​​of the another video block, the adjustment value is determined.

22. The method of claim 21, wherein the another video block comprises a dummy block.

23. The method of claim 21 or claim 22, wherein the prediction of the other video block comprises an intra prediction of the other video block.

24. The method of claim 23, wherein the intra prediction of the other video block is based on reference samples of the other video block, the reference samples being adjacent or non-adjacent to the other video block.

25. The method of claim 23 or claim 24, wherein the intra prediction of the other video block is in at least one of the following modes: Planar intra prediction mode, DC intra prediction mode, Angular intra prediction mode, Wide-angle intra prediction mode, Position-dependent intra prediction combination (PDPC) mode, Multiple Reference Line (MRL) mode, Intra-frame sub-division (ISP) mode, Matrix Weighted Intra Prediction (MIP) mode, Cross-Component Linear Model (CCLM) mode, and Intra-frame blending mode.

26. The method of any one of claims 23 to 25, wherein determining the intra prediction for the another video block comprises: An intra-prediction mode is applied to the other video block, the intra-prediction mode being applied to the current video block.

27. The method according to any one of claims 23 to 26, wherein The intra prediction of the other video block is determined based on a first intra prediction codec and a first reference sample of the other video block, and The intra prediction of the current video block is determined based on a second intra prediction codec and a second reference sample of the current video block, where the second reference sample is different from the first reference sample.

28. The method of claim 27, wherein the first intra prediction tool and the second intra prediction tool are the same or different.

29. The method of claim 27, wherein a first filter of the first intra prediction codec is shorter than a second filter of the second intra prediction codec.

30. The method of claim 27, wherein the first intra prediction codec does not have a position dependent intra prediction combining (PDPC) mode.

31. The method of any one of claims 1 to 30, wherein the another video block and the current video block are in a same video region.

32. The method of any one of claims 1 to 30, wherein the another video block and the current video block are in different video regions.

33. The method of claim 31 or claim 32, wherein a video region comprises at least one of: a picture, a slice, and a tile.

34. The method of claim 32, wherein the another video block is in a reference picture of the current video block.

35. The method of any one of claims 1 to 34, wherein the another video block and the current video block have the same width and the same height.

36. The method according to any one of claims 1 to 35, further comprising: The another video block is determined based on a block vector (BV) associated with the current video block.

37. The method of any one of claims 1 to 36, wherein the reconstructed samples associated with the other video block are at least one of: the other video block, and The reference sample of the other video block is adjacent to or not adjacent to the other video block.

38. The method of claim 37, wherein a process was previously applied to the reference sample of the other video block.

39. The method of claim 38, further comprising: determining the reference sample of the another video block; as well as A process is applied to the reference sample of the other video block.

40. The method of claim 38 or claim 39, wherein the process comprises at least one of: The filtering process, and Value mapping process.

41. The method of claim 40, wherein the value mapping process comprises luma mapping with a chroma scaling process.

42. The method according to any one of claims 1 to 41, wherein at least one of the other video block and a reference sample of the other video block is in a video area, the reference sample being adjacent or non-adjacent to the other video block.

43. The method of claim 42, wherein the video region does not overlap with the current video block.

44. The method of claim 42 or claim 43, wherein the video region is inside at least one of: a picture, a slice, and a tile that contains the current video block.

45. The method according to any one of claims 42 to 44, wherein the video region is in at least one predefined codec tree unit (CTU) or at least one CTU row.

46. ​​The method of any one of claims 42 to -45, wherein the video region is a valid region of a prediction block for intra block copy (IBC).

47. The method of any one of claims 1 to 46, wherein position information of the another video block is included in the bitstream.

48. The method of claim 47, wherein the location information comprises at least one of: a block vector (BV) associated with the current video block, The prediction of the BV, and A BV difference (BVD) associated with the current video block.

49. The method of claim 48, further comprising: The BV is determined from a plurality of candidate BVs for the current video block, the plurality of candidate BVs being in a merge mode.

50. The method of claim 48 or claim 49, wherein the codec tool for including the BV in the bitstream is used to include another BV for intra block copy (IBC) mode in the bitstream.

51. The method of claim 50, wherein a first arithmetic coding context model is used for the another video block and a second arithmetic coding context model is used for the IBC mode, the first arithmetic coding context model being different from the second arithmetic coding context model.

52. The method according to any one of claims 1 to 46, further comprising: Position information of the another video block is determined.

53. The method of claim 52, wherein determining the position information of the another video block comprises: The position information is determined based on template matching.

54. The method of any one of claims 1 to 53, wherein intra prediction of the current video block is included in the bitstream.

55. The method of claim 54, wherein a codec for including the intra prediction of the current video block in the bitstream is used to include another intra prediction of the video block that is encoded without using the adjustment value.

56. The method of claim 54 or claim 55, wherein the intra prediction of the current video block is based on at least one of: Most Probable Model (MPM), Decoder-side intra mode derivation (DIMD), and Template-based intra prediction mode (TIMD).

57. The method of any one of claims 54 to 56, wherein a first intra-prediction mode is disabled for the current video block.

58. The method of claim 57, wherein an indication of the first intra-prediction mode is included in the bitstream.

59. The method of claim 57 or claim 58, wherein the first intra prediction mode comprises at least one of: Planar intra prediction mode, DC intra prediction mode, Angular intra prediction mode, Wide-angle intra prediction mode, Position-dependent intra prediction combination (PDPC) mode, Multiple Reference Line (MRL) mode, Intra-frame sub-division (ISP) mode, Matrix Weighted Intra Prediction (MIP) mode, Cross-Component Linear Model (CCLM) mode, and Intra-frame blending mode.

60. A method according to any one of claims 1 to 59, wherein information on application of the method is included in the bitstream.

61. The method of claim 60, wherein the information includes at least one of: whether to apply the method, and How to apply the described method.

62. The method of claim 60 or claim 61, wherein the information is in a syntax element (SE).

63. The method of claim 62, wherein the SE is in at least one of: a sequence parameter set (SPS), a picture parameter set (PPS), a sequence header, a picture header, a slice header, a codec tree unit (CTU), a codec unit (CU), a transform unit (TU), and a prediction unit (PU).

64. The method of claim 62 or claim 63, wherein the SE is encoded and decoded using at least one arithmetic context model.

65. The method of claim 64, wherein the at least one arithmetic context model is based on at least one neighboring block of the current video block.

66. The method of claim 62 or 63, wherein the SE is bypassed for coding.

67. The method of claim 62 or 63, wherein the SE is hierarchically encoded and decoded.

68. The method of claim 67, wherein if a first SE of a first level is to be disabled, a second SE of a second level, which is lower than the first level, is not present in the bitstream.

69. The method of claim 68, wherein the first level comprises a sequence parameter set (SPS) level and the second level comprises a codec unit (CU) level.

70. The method of any one of claims 60 to 63, wherein the information is included in the bitstream based on at least one condition being satisfied.

71. The method of claim 70, wherein the at least one condition comprises at least one of the following: a first condition, wherein the first condition is that the current video block is intra-coded, and The second condition is that the current video block is coded and decoded by intra block copy (IBC).

72. A method according to any one of claims 60 to 71, wherein if the information indicates that the method is applied to the current video block, at least one of an indication of an intra-frame prediction mode and a block vector (BV) associated with the current video block is included in the bitstream.

73. The method of any one of claims 60 to 71, wherein if the information indicates that the method is applied to the current video block, at least one of the following is modified: a first codec for including a block vector (BV) associated with the current video block in the bitstream, and A second codec is provided for including an indication of an intra prediction mode in the bitstream.

74. The method of any one of claims 60 to 73, wherein the information is at one of the following levels: sequence level, group of pictures level, picture level, slice level, and tile group level.

75. A method according to any one of claims 60 to 74, wherein the information is in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a decoding parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header and a tile group header.

76. A method according to any one of claims 60 to 75, wherein the information is in a region comprising a plurality of samples or a plurality of pixels.

77. The method of claim 76, wherein the region comprises one of a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, and a sub-picture.

78. The method of any one of claims 60 to 77, further comprising: The information is determined based on the encoded and decoded information.

79. The method of claim 78, wherein the encoded information comprises at least one of: block size, color format, single tree partitioning or dual tree partitioning, color component, slice type, and picture type.

80. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 79.

81. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 79.

82. A non-transitory computer-readable recording medium storing a bit stream of a video, wherein the bit stream of the video is generated by a method performed by a video processing device, wherein the method comprises: Determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; as well as The bitstream is generated based on the first intra prediction samples and the adjustment value.

83. A method for storing a bitstream of a video, comprising: Determining a first intra-frame prediction sample of a current video block of the video; determining an adjustment value associated with another video block in the video, the other video block being encoded and decoded before the current video block; generating the bitstream based on the first intra prediction sample and the adjustment value; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images